Audit Logs: Monitoring Who Does What in Databricks
Who this is for:
Architecture / Concept Overview: Audit Logs: Monitoring Who Does What in Databricks
%%{init: {"theme":"base","themeVariables":{"background":"#0B0E14","primaryTextColor":"#E0E6ED","lineColor":"#5D6470","darkMode":true,"primaryColor":"#2E4A4A","secondaryColor":"#374151","secondaryTextColor":"#E0E6ED","tertiaryColor":"#111827","tertiaryTextColor":"#E0E6ED","edgeLabelBackground":"#1f2937"}}}%%
flowchart LR
classDef source fill:#3F4B59,stroke:#9CA3AF,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef ingestion fill:#5A4B36,stroke:#C9A86B,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef processing fill:#535072,stroke:#8E82B4,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef storage fill:#2E4A4A,stroke:#5FAFA8,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef serving fill:#3D5550,stroke:#6BB7AA,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef governance fill:#5A3F52,stroke:#C28BB0,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
Events[Workspace Events] --> AuditService[Audit Service]
AuditService --> SystemTables[System Tables]
AuditService --> LogDelivery[Log Delivery]
SystemTables --> SQLQueries[SQL Analytics]
SystemTables --> Dashboards[Security Dashboards]
LogDelivery --> S3[Cloud Storage]
S3 --> SIEM[SIEM Platform]
Events:::source
AuditService:::ingestion
SystemTables:::storage
LogDelivery:::processing
SQLQueries:::serving
Dashboards:::serving
S3:::storage
SIEM:::governance
*Audit log distribution: events flow to system tables for SQL analysis and to cloud storage for SIEM integration.*
%%{init: {"theme":"base","themeVariables":{"background":"#0B0E14","primaryTextColor":"#E0E6ED","lineColor":"#5D6470","darkMode":true,"primaryColor":"#2E4A4A","secondaryColor":"#374151","secondaryTextColor":"#E0E6ED","tertiaryColor":"#111827","tertiaryTextColor":"#E0E6ED","edgeLabelBackground":"#1f2937"}}}%%
graph TD
classDef source fill:#3F4B59,stroke:#9CA3AF,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef ingestion fill:#5A4B36,stroke:#C9A86B,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef processing fill:#535072,stroke:#8E82B4,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef storage fill:#2E4A4A,stroke:#5FAFA8,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef serving fill:#3D5550,stroke:#6BB7AA,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef governance fill:#5A3F52,stroke:#C28BB0,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
Workspace[Workspace Events] --> WsAudit[Workspace Audit]
Account[Account Events] --> AccAudit[Account Audit]
UC[Unity Catalog Events] --> UCaudit[UC Audit]
WsAudit --> Combined[system.access.audit]
AccAudit --> Combined
UCaudit --> Combined
Combined --> Retention[365 Day Retention]
Workspace:::source
Account:::source
UC:::source
WsAudit:::ingestion
AccAudit:::ingestion
UCaudit:::ingestion
Combined:::storage
Retention:::governance
*Audit event sources: workspace operations, account administration, and Unity Catalog data access converge into a unified system table with 365-day retention.*
Key Terms
Prerequisites and Setup
- Databricks Premium tier (system tables require Premium+)
- Account admin access for log delivery configuration
- SQL warehouse or serverless compute for querying system tables
- Cloud storage bucket for log delivery (with immutability/WORM if compliance-required)
- SIEM platform for centralized security monitoring (Splunk, Sentinel, Chronicle)
Step-by-Step Implementation
Configuration Reference
| Setting | Scope | Configuration | Retention |
|---|---|---|---|
| System Tables | Account | Auto-enabled (Premium+) | 365 days |
| Log Delivery (JSON) | Account | Account API | Customer-managed |
| Log Delivery (CSV) | Account | Account API | Customer-managed |
| Workspace Filter | Account | Per log delivery config | N/A |
| Verbose Mode | Workspace | Workspace settings | Same as base |
| Unity Catalog Events | Account | Auto-enabled with UC | 365 days |