Compute Policies: Controlling Cluster Configurations
Who this is for:
Architecture / Concept Overview: Compute Policies: Controlling Cluster Configurations
%%{init: {"theme":"base","themeVariables":{"background":"#0B0E14","primaryTextColor":"#E0E6ED","lineColor":"#5D6470","darkMode":true,"primaryColor":"#2E4A4A","secondaryColor":"#374151","secondaryTextColor":"#E0E6ED","tertiaryColor":"#111827","tertiaryTextColor":"#E0E6ED","edgeLabelBackground":"#1f2937"}}}%%
graph TD
classDef source fill:#3F4B59,stroke:#9CA3AF,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef ingestion fill:#5A4B36,stroke:#C9A86B,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef processing fill:#535072,stroke:#8E82B4,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef storage fill:#2E4A4A,stroke:#5FAFA8,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef serving fill:#3D5550,stroke:#6BB7AA,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef governance fill:#5A3F52,stroke:#C28BB0,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
Admin[Platform Admin] --> PolicyDef[Policy Definition]
PolicyDef --> Fixed[Fixed Attributes]
PolicyDef --> Range[Range Constraints]
PolicyDef --> Allowlist[Allowlist Values]
PolicyDef --> Regex[Regex Patterns]
Fixed --> Cluster[Cluster Creation]
Range --> Cluster
Allowlist --> Cluster
Regex --> Cluster
Cluster --> Validation[Policy Validation Engine]
Admin:::governance
PolicyDef:::processing
Fixed:::storage
Range:::storage
Allowlist:::storage
Regex:::storage
Cluster:::ingestion
Validation:::serving
*Policy definition model: admins define constraints using fixed values, ranges, allowlists, and regex patterns that the validation engine enforces at cluster creation.*
%%{init: {"theme":"base","themeVariables":{"background":"#0B0E14","primaryTextColor":"#E0E6ED","lineColor":"#5D6470","darkMode":true,"primaryColor":"#2E4A4A","secondaryColor":"#374151","secondaryTextColor":"#E0E6ED","tertiaryColor":"#111827","tertiaryTextColor":"#E0E6ED","edgeLabelBackground":"#1f2937"}}}%%
flowchart LR
classDef source fill:#3F4B59,stroke:#9CA3AF,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef ingestion fill:#5A4B36,stroke:#C9A86B,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef processing fill:#535072,stroke:#8E82B4,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef storage fill:#2E4A4A,stroke:#5FAFA8,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef serving fill:#3D5550,stroke:#6BB7AA,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef governance fill:#5A3F52,stroke:#C28BB0,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
User[User Request] --> PolicySelect[Select Policy]
PolicySelect --> Override[User Overrides]
Override --> Validate[Policy Validation]
Validate --> |Pass| Create[Cluster Created]
Validate --> |Fail| Reject[Request Rejected]
Create --> Track[Usage Tracked]
User:::source
PolicySelect:::ingestion
Override:::processing
Validate:::governance
Create:::serving
Reject:::source
Track:::storage
*Cluster creation flow: users select a policy, customize allowed parameters, and the policy engine validates before cluster creation proceeds.*
Key Terms
Prerequisites and Setup
- Databricks workspace with Premium tier
- Workspace admin access for policy creation and assignment
- Understanding of cluster configuration parameters
- Knowledge of team compute needs and cost constraints
- Groups configured for policy assignment
Step-by-Step Implementation
Configuration Reference
| Constraint Type | Syntax | Behavior |
|---|---|---|
fixed | {"type": "fixed", "value": X} | Parameter locked to value; hidden from UI |
range | {"type": "range", "minValue": X, "maxValue": Y} | Numeric parameter bounded |
allowlist | {"type": "allowlist", "values": [...]} | Parameter limited to listed values |
blocklist | {"type": "blocklist", "values": [...]} | Listed values prohibited |
regex | {"type": "regex", "pattern": "..."} | String must match pattern |
unlimited | {"type": "unlimited"} | No constraint (user has full control) |
defaultValue | Added to any type | Pre-populated but overridable |
hidden | Added to fixed type | Hidden from cluster creation UI |