OpenAI SDK with Databricks-Hosted Models
Who this is for:
Architecture / Concept Overview: OpenAI SDK with Databricks-Hosted Models
%%{init: {"theme":"base","themeVariables":{"background":"#0B0E14","primaryTextColor":"#E0E6ED","lineColor":"#5D6470","darkMode":true,"primaryColor":"#2E4A4A","secondaryColor":"#374151","secondaryTextColor":"#E0E6ED","tertiaryColor":"#111827","tertiaryTextColor":"#E0E6ED","edgeLabelBackground":"#1f2937"}}}%%
flowchart LR
classDef source fill:#3F4B59,stroke:#9CA3AF,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef ingestion fill:#5A4B36,stroke:#C9A86B,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef processing fill:#535072,stroke:#8E82B4,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef storage fill:#2E4A4A,stroke:#5FAFA8,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef serving fill:#3D5550,stroke:#6BB7AA,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef governance fill:#5A3F52,stroke:#C28BB0,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
APP[Application Code]:::source
SDK[OpenAI Python SDK]:::ingestion
ENDPOINT[Databricks Serving Endpoint]:::processing
MODEL[Hosted LLM<br/>DBRX / Llama / Custom]:::storage
GPU[GPU Compute]:::serving
GOV[Data Governance Boundary]:::governance
APP --> SDK --> ENDPOINT --> MODEL --> GPU
ENDPOINT --> GOV
*The OpenAI SDK communicates with Databricks Model Serving endpoints using the same API format, routing inference to hosted models within your security perimeter.*
%%{init: {"theme":"base","themeVariables":{"background":"#0B0E14","primaryTextColor":"#E0E6ED","lineColor":"#5D6470","darkMode":true,"primaryColor":"#2E4A4A","secondaryColor":"#374151","secondaryTextColor":"#E0E6ED","tertiaryColor":"#111827","tertiaryTextColor":"#E0E6ED","edgeLabelBackground":"#1f2937"}}}%%
graph TD
classDef source fill:#3F4B59,stroke:#9CA3AF,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef ingestion fill:#5A4B36,stroke:#C9A86B,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef processing fill:#535072,stroke:#8E82B4,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef storage fill:#2E4A4A,stroke:#5FAFA8,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef serving fill:#3D5550,stroke:#6BB7AA,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef governance fill:#5A3F52,stroke:#C28BB0,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
COMPAT[OpenAI API Compatibility]:::source
CHAT[/v1/chat/completions]:::serving
EMBED[/v1/embeddings]:::processing
COMPLETIONS[/v1/completions]:::ingestion
COMPAT --> CHAT
COMPAT --> EMBED
COMPAT --> COMPLETIONS
CHAT --> STREAMING[Streaming Support]:::serving
CHAT --> FUNCTIONS[Function Calling]:::serving
EMBED --> BATCH[Batch Embedding]:::processing
COMPLETIONS --> LEGACY[Legacy Completions]:::ingestion
*Databricks serving endpoints support the major OpenAI API formats including chat completions, embeddings, and streaming.*
Key Terms
Prerequisites and Setup
- Databricks workspace with Model Serving enabled
- A serving endpoint deployed (Foundation Model API or custom)
- Python 3.8+ with the
openaipackage - Databricks personal access token or OAuth credentials
- Network access to the workspace serving endpoint
Step-by-Step Implementation
Configuration Reference
| Parameter | Description | Example |
|---|---|---|
api_key | Databricks PAT or OAuth token | dapi_abc123... |
base_url | Workspace serving endpoint URL | https://adb-*.azuredatabricks.net/serving-endpoints |
model | Serving endpoint name | databricks-meta-llama-3-1-70b-instruct |
temperature | Sampling temperature (0-2) | 0.1 for deterministic |
max_tokens | Maximum output tokens | 512, 1024, 4096 |
stream | Enable token streaming | True / False |
tools | Function calling definitions | JSON tool schema array |