Provisioning Databricks via AWS Marketplace (Step-by-Step Guide)
Who this is for:
Architecture / Concept Overview: Provisioning Databricks via AWS Marketplace (Step-by-Step Guide)
%%{init: {"theme":"base","themeVariables":{"background":"#0B0E14","primaryTextColor":"#E0E6ED","lineColor":"#5D6470","darkMode":true,"primaryColor":"#2E4A4A","secondaryColor":"#374151","secondaryTextColor":"#E0E6ED","tertiaryColor":"#111827","tertiaryTextColor":"#E0E6ED","edgeLabelBackground":"#1f2937"}}}%%
flowchart LR
classDef source fill:#3F4B59,stroke:#9CA3AF,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef ingestion fill:#5A4B36,stroke:#C9A86B,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef processing fill:#535072,stroke:#8E82B4,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef storage fill:#2E4A4A,stroke:#5FAFA8,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef serving fill:#3D5550,stroke:#6BB7AA,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef governance fill:#5A3F52,stroke:#C28BB0,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
MP[AWS Marketplace] -->|Subscribe| ACCT[Databricks Account]
ACCT -->|Configure| CRED[Credential Config]
ACCT -->|Configure| STOR[Storage Config]
ACCT -->|Configure| NET[Network Config]
CRED --> WS[Workspace Deployment]
STOR --> WS
NET --> WS
WS -->|Ready| UI[Workspace UI]
MP:::source
ACCT:::ingestion
CRED:::governance
STOR:::storage
NET:::storage
WS:::processing
UI:::serving
*Provisioning flow from AWS Marketplace subscription through account configuration to workspace deployment.*
%%{init: {"theme":"base","themeVariables":{"background":"#0B0E14","primaryTextColor":"#E0E6ED","lineColor":"#5D6470","darkMode":true,"primaryColor":"#2E4A4A","secondaryColor":"#374151","secondaryTextColor":"#E0E6ED","tertiaryColor":"#111827","tertiaryTextColor":"#E0E6ED","edgeLabelBackground":"#1f2937"}}}%%
graph TD
classDef source fill:#3F4B59,stroke:#9CA3AF,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef ingestion fill:#5A4B36,stroke:#C9A86B,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef processing fill:#535072,stroke:#8E82B4,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef storage fill:#2E4A4A,stroke:#5FAFA8,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef serving fill:#3D5550,stroke:#6BB7AA,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef governance fill:#5A3F52,stroke:#C28BB0,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
S1[1. Subscribe on Marketplace] --> S2[2. Create Account Admin]
S2 --> S3[3. Create Cross-Account Role]
S3 --> S4[4. Register Credential Config]
S4 --> S5[5. Create Root S3 Bucket]
S5 --> S6[6. Register Storage Config]
S6 --> S7[7. Create Network Config]
S7 --> S8[8. Deploy Workspace]
S8 --> S9[9. Validate Access]
S1:::source
S2:::governance
S3:::governance
S4:::ingestion
S5:::storage
S6:::storage
S7:::storage
S8:::processing
S9:::serving
*Sequential provisioning steps from Marketplace subscription to validated workspace.*
Key Terms
Prerequisites and Setup
- AWS account with billing administrator access (to subscribe on Marketplace)
- AWS CLI configured with admin credentials
- IAM permissions to create roles, policies, S3 buckets, and VPC resources
- An email address for the Databricks account administrator
- VPC with two private subnets in different AZs (for customer-managed networking)
Step-by-Step Implementation
Configuration Reference
| Configuration | API Endpoint | Required Fields | Notes |
|---|---|---|---|
| Credentials | /accounts/{id}/credentials | role_arn | One per AWS account |
| Storage | /accounts/{id}/storage-configurations | bucket_name | One per workspace |
| Network | /accounts/{id}/networks | vpc_id, subnet_ids, security_group_ids | Optional for managed VPC |
| Workspace | /accounts/{id}/workspaces | credentials_id, storage_configuration_id, aws_region | Combines all configs |
| Private Access | /accounts/{id}/private-access-settings | public_access_enabled, allowed_vpc_endpoint_ids | For Private Link |