w.secrets.put_secret(scope="ai-keys", key="openai", string_value="sk-...")
Who this is for:
Architecture / Concept Overview: w.secrets.put_secret(scope="ai-keys", key="openai", string_value="sk-...")
External Models create a managed proxy endpoint that routes requests to third-party providers while applying Databricks governance.
%%{init: {"theme":"base","themeVariables":{"background":"#0B0E14","primaryTextColor":"#E0E6ED","lineColor":"#5D6470","darkMode":true,"primaryColor":"#2E4A4A","secondaryColor":"#374151","secondaryTextColor":"#E0E6ED","tertiaryColor":"#111827","tertiaryTextColor":"#E0E6ED","edgeLabelBackground":"#1f2937"}}}%%
flowchart LR
classDef source fill:#3F4B59,stroke:#9CA3AF,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef ingestion fill:#5A4B36,stroke:#C9A86B,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef processing fill:#535072,stroke:#8E82B4,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef storage fill:#2E4A4A,stroke:#5FAFA8,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef serving fill:#3D5550,stroke:#6BB7AA,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef governance fill:#5A3F52,stroke:#C28BB0,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
CODE[Application Code] -->|OpenAI API| EP[Databricks External Model Endpoint]
EP -->|Authenticate| SECRETS[Databricks Secrets]
EP -->|Route| OPENAI[OpenAI API]
EP -->|Route| ANTHROPIC[Anthropic API]
EP -->|Route| COHERE[Cohere API]
EP -->|Log| INF[Inference Table]
EP -->|Limit| RATE[Rate Limits]
CODE:::source
EP:::governance
SECRETS:::processing
OPENAI:::serving
ANTHROPIC:::serving
COHERE:::serving
INF:::storage
RATE:::ingestion
*External Models proxy: Databricks endpoints route to third-party providers with governance and logging.*
%%{init: {"theme":"base","themeVariables":{"background":"#0B0E14","primaryTextColor":"#E0E6ED","lineColor":"#5D6470","darkMode":true,"primaryColor":"#2E4A4A","secondaryColor":"#374151","secondaryTextColor":"#E0E6ED","tertiaryColor":"#111827","tertiaryTextColor":"#E0E6ED","edgeLabelBackground":"#1f2937"}}}%%
graph TD
classDef source fill:#3F4B59,stroke:#9CA3AF,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef ingestion fill:#5A4B36,stroke:#C9A86B,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef processing fill:#535072,stroke:#8E82B4,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef storage fill:#2E4A4A,stroke:#5FAFA8,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef serving fill:#3D5550,stroke:#6BB7AA,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef governance fill:#5A3F52,stroke:#C28BB0,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
PROVIDERS[Supported Providers] --> OPENAI_P[OpenAI]
PROVIDERS --> ANTHRO_P[Anthropic]
PROVIDERS --> COHERE_P[Cohere]
PROVIDERS --> AZURE_P[Azure OpenAI]
PROVIDERS --> GOOGLE_P[Google AI]
PROVIDERS --> CUSTOM_P[Custom Provider]
OPENAI_P --> GPT4[GPT-4o]
OPENAI_P --> GPT35[GPT-4o-mini]
ANTHRO_P --> CLAUDE_S[Claude 3.5 Sonnet]
ANTHRO_P --> CLAUDE_O[Claude 3 Opus]
PROVIDERS:::governance
OPENAI_P:::processing
ANTHRO_P:::storage
COHERE_P:::serving
AZURE_P:::ingestion
GOOGLE_P:::source
CUSTOM_P:::source
GPT4:::processing
GPT35:::processing
CLAUDE_S:::storage
CLAUDE_O:::storage
*Supported external model providers and popular models.*
Key Terms
Prerequisites and Setup
- Databricks workspace with Model Serving enabled.
- API keys for the external providers, stored in Databricks Secrets.
CAN_MANAGEpermission on serving endpoints (to create).
Step-by-Step Implementation
Configuration Reference
| Parameter | Default | Description |
|---|---|---|
provider | — | Provider name: openai, anthropic, cohere, google, etc. |
task | — | API task: llm/v1/chat, llm/v1/completions, llm/v1/embeddings |
openai_api_key | — | Secrets reference for OpenAI key |
anthropic_api_key | — | Secrets reference for Anthropic key |
openai_api_type | openai | Set to azure for Azure OpenAI |
openai_api_base | provider default | Custom API base URL |
openai_deployment_name | — | Azure OpenAI deployment name |