Cost Management: Dashboards, Policies, and Billing Analysis

    Who this is for:

    Architecture / Concept Overview: Cost Management: Dashboards, Policies, and Billing Analysis

    %%{init: {"theme":"base","themeVariables":{"background":"#0B0E14","primaryTextColor":"#E0E6ED","lineColor":"#5D6470","darkMode":true,"primaryColor":"#2E4A4A","secondaryColor":"#374151","secondaryTextColor":"#E0E6ED","tertiaryColor":"#111827","tertiaryTextColor":"#E0E6ED","edgeLabelBackground":"#1f2937"}}}%% flowchart LR classDef source fill:#3F4B59,stroke:#9CA3AF,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef ingestion fill:#5A4B36,stroke:#C9A86B,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef processing fill:#535072,stroke:#8E82B4,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef storage fill:#2E4A4A,stroke:#5FAFA8,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef serving fill:#3D5550,stroke:#6BB7AA,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef governance fill:#5A3F52,stroke:#C28BB0,stroke-width:2px,rx:8,ry:8,color:#E0E6ED Compute[All Compute] --> Metering[DBU Metering] Metering --> BillingTable[system.billing.usage] BillingTable --> Dashboard[Cost Dashboard] BillingTable --> Alerts[Budget Alerts] BillingTable --> Chargeback[Chargeback Reports] Dashboard --> Optimize[Optimization Actions] Alerts --> Notify[Team Notifications] Compute:::source Metering:::ingestion BillingTable:::storage Dashboard:::serving Alerts:::governance Chargeback:::processing Optimize:::serving Notify:::source

    *Cost management pipeline: compute metering feeds system tables, which power dashboards, alerts, and chargeback reporting for optimization decisions.*

    %%{init: {"theme":"base","themeVariables":{"background":"#0B0E14","primaryTextColor":"#E0E6ED","lineColor":"#5D6470","darkMode":true,"primaryColor":"#2E4A4A","secondaryColor":"#374151","secondaryTextColor":"#E0E6ED","tertiaryColor":"#111827","tertiaryTextColor":"#E0E6ED","edgeLabelBackground":"#1f2937"}}}%% graph TD classDef source fill:#3F4B59,stroke:#9CA3AF,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef ingestion fill:#5A4B36,stroke:#C9A86B,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef processing fill:#535072,stroke:#8E82B4,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef storage fill:#2E4A4A,stroke:#5FAFA8,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef serving fill:#3D5550,stroke:#6BB7AA,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef governance fill:#5A3F52,stroke:#C28BB0,stroke-width:2px,rx:8,ry:8,color:#E0E6ED Policies[Compute Policies] --> Tags[Enforced Tags] Tags --> UsageData[Usage + Tags] UsageData --> TeamCosts[Per-Team Costs] UsageData --> ProjectCosts[Per-Project Costs] UsageData --> SkuCosts[Per-SKU Costs] TeamCosts --> Report[Chargeback Report] ProjectCosts --> Report SkuCosts --> Optimization[Optimization Targets] Policies:::governance Tags:::processing UsageData:::storage TeamCosts:::ingestion ProjectCosts:::ingestion SkuCosts:::ingestion Report:::serving Optimization:::source

    *Chargeback flow: compute policies enforce tags that enrich billing data, enabling cost attribution to teams and projects.*

    %%{init: {"theme":"base","themeVariables":{"background":"#0B0E14","primaryTextColor":"#E0E6ED","lineColor":"#5D6470","darkMode":true,"primaryColor":"#2E4A4A","secondaryColor":"#374151","secondaryTextColor":"#E0E6ED","tertiaryColor":"#111827","tertiaryTextColor":"#E0E6ED","edgeLabelBackground":"#1f2937"}}}%% flowchart LR classDef source fill:#3F4B59,stroke:#9CA3AF,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef ingestion fill:#5A4B36,stroke:#C9A86B,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef processing fill:#535072,stroke:#8E82B4,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef storage fill:#2E4A4A,stroke:#5FAFA8,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef serving fill:#3D5550,stroke:#6BB7AA,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef governance fill:#5A3F52,stroke:#C28BB0,stroke-width:2px,rx:8,ry:8,color:#E0E6ED Identify[Identify Waste] --> Idle[Idle Clusters] Identify --> Oversized[Oversized Compute] Identify --> Untagged[Untagged Resources] Idle --> Terminate[Auto-Terminate] Oversized --> Rightsize[Right-Size] Untagged --> Enforce[Policy Enforcement] Terminate --> Savings[Cost Savings] Rightsize --> Savings Enforce --> Savings Identify:::governance Idle:::source Oversized:::source Untagged:::source Terminate:::processing Rightsize:::processing Enforce:::processing Savings:::serving

    *Cost optimization cycle: identify waste categories (idle, oversized, untagged), apply remediation, and measure savings.*

    Key Terms

    Prerequisites and Setup

    • Databricks Premium tier for system table access
    • Account admin access for budget configuration
    • SQL warehouse for querying billing system tables
    • Compute policies configured with mandatory tagging
    • Finance team input on budget thresholds and chargeback requirements

    Step-by-Step Implementation

      Configuration Reference

      Cost Management: Dashboards, Policies, and Billing Analysis configuration options
      FeatureScopeAccess RequiredData Freshness
      system.billing.usageAccountAccount admin or delegated~6 hours
      system.billing.list_pricesAccountAccount admin or delegatedDaily
      Budget AlertsAccountAccount adminReal-time evaluation
      Custom Tags (billing)WorkspaceVia compute policiesPer usage record
      Cost DashboardWorkspaceSQL warehouse accessBased on query schedule
      Log Delivery (billing)AccountAccount admin~6 hours

      Monitoring, Cost, and Security Considerations

      Common Pitfalls and Recommended Patterns

        Frequently Asked Questions