Confirm AutoML is available

    Who this is for:

    Architecture / Concept Overview: Confirm AutoML is available

    AutoML orchestrates a multi-trial search across algorithms and hyperparameters, logging every run to MLflow.

    %%{init: {"theme":"base","themeVariables":{"background":"#0B0E14","primaryTextColor":"#E0E6ED","lineColor":"#5D6470","darkMode":true,"primaryColor":"#2E4A4A","secondaryColor":"#374151","secondaryTextColor":"#E0E6ED","tertiaryColor":"#111827","tertiaryTextColor":"#E0E6ED","edgeLabelBackground":"#1f2937"}}}%% flowchart LR classDef source fill:#3F4B59,stroke:#9CA3AF,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef ingestion fill:#5A4B36,stroke:#C9A86B,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef processing fill:#535072,stroke:#8E82B4,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef storage fill:#2E4A4A,stroke:#5FAFA8,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef serving fill:#3D5550,stroke:#6BB7AA,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef governance fill:#5A3F52,stroke:#C28BB0,stroke-width:2px,rx:8,ry:8,color:#E0E6ED DS[Training Dataset] -->|Profile| EDA[Data Exploration] EDA -->|Preprocess| FP[Feature Pipeline] FP -->|Search| TRIALS[Hyperparameter Trials] TRIALS -->|Log| MLF[MLflow Experiment] MLF -->|Select| BEST[Best Model] BEST -->|Register| UC[Unity Catalog] DS:::source EDA:::ingestion FP:::processing TRIALS:::processing MLF:::governance BEST:::serving UC:::governance

    *AutoML pipeline: data profiling, feature preprocessing, hyperparameter search, and model registration.*

    %%{init: {"theme":"base","themeVariables":{"background":"#0B0E14","primaryTextColor":"#E0E6ED","lineColor":"#5D6470","darkMode":true,"primaryColor":"#2E4A4A","secondaryColor":"#374151","secondaryTextColor":"#E0E6ED","tertiaryColor":"#111827","tertiaryTextColor":"#E0E6ED","edgeLabelBackground":"#1f2937"}}}%% graph TD classDef source fill:#3F4B59,stroke:#9CA3AF,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef ingestion fill:#5A4B36,stroke:#C9A86B,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef processing fill:#535072,stroke:#8E82B4,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef storage fill:#2E4A4A,stroke:#5FAFA8,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef serving fill:#3D5550,stroke:#6BB7AA,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef governance fill:#5A3F52,stroke:#C28BB0,stroke-width:2px,rx:8,ry:8,color:#E0E6ED AUTOML[AutoML] --> CLF[Classification] AUTOML --> REG[Regression] AUTOML --> FC[Forecasting] CLF --> LR[Logistic Regression] CLF --> RF[Random Forest] CLF --> XGB[XGBoost] CLF --> LGBM[LightGBM] REG --> LINR[Linear Regression] REG --> RFR[Random Forest Regressor] REG --> XGBR[XGBoost Regressor] FC --> PROPHET[Prophet] FC --> ARIMA[ARIMA] AUTOML:::governance CLF:::processing REG:::processing FC:::processing LR:::source RF:::source XGB:::source LGBM:::source LINR:::ingestion RFR:::ingestion XGBR:::ingestion PROPHET:::storage ARIMA:::storage

    *Algorithm families explored by AutoML for classification, regression, and forecasting tasks.*

    Key Terms

    Prerequisites and Setup

    • Databricks Runtime for ML (latest version).
    • A Delta table registered in Unity Catalog with at least a few hundred rows.
    • USE CATALOG, USE SCHEMA, and SELECT privileges on the training table.

    Step-by-Step Implementation

      Configuration Reference

      Confirm AutoML is available configuration options
      ParameterDefaultDescription
      target_col—Column to predict
      primary_metrictask-dependentMetric to optimise (f1, log_loss, rmse, smape, etc.)
      timeout_minutes120Wall-clock time limit for the search
      max_trialsNone (unlimited)Maximum number of trials to run
      exclude_cols[]Columns to exclude from features
      time_col—Timestamp column (forecasting only)
      horizon—Number of periods to forecast
      frequency—Time granularity (d, h, m, etc.)

      Monitoring, Cost, and Security Considerations

      Common Pitfalls and Recommended Patterns

        Frequently Asked Questions