.github/workflows/mlops.yml
Who this is for:
Architecture / Concept Overview: .github/workflows/mlops.yml
A mature MLOps workflow connects source control, automated training, model validation, deployment, and monitoring into a continuous loop.
%%{init: {"theme":"base","themeVariables":{"background":"#0B0E14","primaryTextColor":"#E0E6ED","lineColor":"#5D6470","darkMode":true,"primaryColor":"#2E4A4A","secondaryColor":"#374151","secondaryTextColor":"#E0E6ED","tertiaryColor":"#111827","tertiaryTextColor":"#E0E6ED","edgeLabelBackground":"#1f2937"}}}%%
flowchart LR
classDef source fill:#3F4B59,stroke:#9CA3AF,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef ingestion fill:#5A4B36,stroke:#C9A86B,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef processing fill:#535072,stroke:#8E82B4,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef storage fill:#2E4A4A,stroke:#5FAFA8,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef serving fill:#3D5550,stroke:#6BB7AA,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef governance fill:#5A3F52,stroke:#C28BB0,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
GIT[Git Repository] -->|Trigger| CI[CI/CD Pipeline]
CI -->|Deploy| BUNDLE[Databricks Asset Bundle]
BUNDLE -->|Run| TRAIN[Training Job]
TRAIN -->|Log| MLF[MLflow Experiment]
MLF -->|Validate| GATE[Validation Gate]
GATE -->|Pass| REG[Register Model in UC]
REG -->|Alias| CHAMP[Champion Alias]
CHAMP -->|Deploy| EP[Serving Endpoint]
EP -->|Monitor| MON[Lakehouse Monitor]
MON -->|Alert| RETRAIN[Retrain Trigger]
RETRAIN -->|Commit| GIT
GIT:::source
CI:::ingestion
BUNDLE:::ingestion
TRAIN:::processing
MLF:::governance
GATE:::processing
REG:::governance
CHAMP:::serving
EP:::serving
MON:::governance
RETRAIN:::source
*End-to-end MLOps loop: code changes trigger training, validated models are deployed, and monitoring feeds back into retraining.*
%%{init: {"theme":"base","themeVariables":{"background":"#0B0E14","primaryTextColor":"#E0E6ED","lineColor":"#5D6470","darkMode":true,"primaryColor":"#2E4A4A","secondaryColor":"#374151","secondaryTextColor":"#E0E6ED","tertiaryColor":"#111827","tertiaryTextColor":"#E0E6ED","edgeLabelBackground":"#1f2937"}}}%%
graph TD
classDef source fill:#3F4B59,stroke:#9CA3AF,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef ingestion fill:#5A4B36,stroke:#C9A86B,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef processing fill:#535072,stroke:#8E82B4,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef storage fill:#2E4A4A,stroke:#5FAFA8,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef serving fill:#3D5550,stroke:#6BB7AA,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef governance fill:#5A3F52,stroke:#C28BB0,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
ENVS[Deployment Environments] --> DEV[Development]
ENVS --> STAGING[Staging]
ENVS --> PROD[Production]
DEV --> DEV_CAT[dev_catalog]
STAGING --> STG_CAT[staging_catalog]
PROD --> PROD_CAT[prod_catalog]
DEV_CAT --> DEV_EP[Dev Endpoint]
STG_CAT --> STG_EP[Staging Endpoint]
PROD_CAT --> PROD_EP[Production Endpoint]
ENVS:::governance
DEV:::processing
STAGING:::storage
PROD:::serving
DEV_CAT:::processing
STG_CAT:::storage
PROD_CAT:::serving
DEV_EP:::ingestion
STG_EP:::source
PROD_EP:::source
*Multi-environment deployment strategy with separate catalogs per environment.*
Key Terms
Prerequisites and Setup
- A Git repository with Databricks Asset Bundle configuration.
- CI/CD platform (GitHub Actions, Azure DevOps, GitLab CI, or Jenkins).
- Service principal with
CAN_MANAGEon jobs andCREATE MODELon the target catalog. - Separate catalogs for dev, staging, and production in Unity Catalog.
Step-by-Step Implementation
Configuration Reference
| Parameter | Default | Description |
|---|---|---|
bundle.name | — | Unique name for the Databricks Asset Bundle |
targets.<env>.workspace.root_path | — | Workspace path for deployed resources per environment |
schedule.cron | — | Cron expression for scheduled retraining |
validation_threshold | custom | Minimum metric improvement to promote a model |
scale_to_zero_enabled | true | Keep at least one replica for production SLA |