Foundation Models: Accessing Meta Llama, Claude, and GPT via Secure APIs
Who this is for:
Architecture / Concept Overview: Foundation Models: Accessing Meta Llama, Claude, and GPT via Secure APIs
Foundation models are accessed through serving endpoints — either Databricks-hosted or external provider routes governed by AI Gateway.
%%{init: {"theme":"base","themeVariables":{"background":"#0B0E14","primaryTextColor":"#E0E6ED","lineColor":"#5D6470","darkMode":true,"primaryColor":"#2E4A4A","secondaryColor":"#374151","secondaryTextColor":"#E0E6ED","tertiaryColor":"#111827","tertiaryTextColor":"#E0E6ED","edgeLabelBackground":"#1f2937"}}}%%
flowchart LR
classDef source fill:#3F4B59,stroke:#9CA3AF,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef ingestion fill:#5A4B36,stroke:#C9A86B,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef processing fill:#535072,stroke:#8E82B4,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef storage fill:#2E4A4A,stroke:#5FAFA8,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef serving fill:#3D5550,stroke:#6BB7AA,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef governance fill:#5A3F52,stroke:#C28BB0,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
APP[Application Code] -->|OpenAI-Compatible API| EP[Serving Endpoint]
EP -->|Route| HOSTED[Databricks-Hosted Models]
EP -->|Route| EXT[External Models via AI Gateway]
HOSTED --> LLAMA[Meta Llama 3]
HOSTED --> DBRX[DBRX]
HOSTED --> MIXTRAL[Mixtral]
HOSTED --> BGE[BGE Embeddings]
EXT --> OPENAI[OpenAI GPT]
EXT --> ANTHROPIC[Anthropic Claude]
EXT --> COHERE[Cohere]
APP:::source
EP:::ingestion
HOSTED:::processing
EXT:::governance
LLAMA:::serving
DBRX:::serving
MIXTRAL:::serving
BGE:::storage
OPENAI:::source
ANTHROPIC:::source
COHERE:::source
*Foundation Model access paths: Databricks-hosted models run in-tenant; external models route through AI Gateway.*
%%{init: {"theme":"base","themeVariables":{"background":"#0B0E14","primaryTextColor":"#E0E6ED","lineColor":"#5D6470","darkMode":true,"primaryColor":"#2E4A4A","secondaryColor":"#374151","secondaryTextColor":"#E0E6ED","tertiaryColor":"#111827","tertiaryTextColor":"#E0E6ED","edgeLabelBackground":"#1f2937"}}}%%
graph TD
classDef source fill:#3F4B59,stroke:#9CA3AF,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef ingestion fill:#5A4B36,stroke:#C9A86B,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef processing fill:#535072,stroke:#8E82B4,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef storage fill:#2E4A4A,stroke:#5FAFA8,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef serving fill:#3D5550,stroke:#6BB7AA,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef governance fill:#5A3F52,stroke:#C28BB0,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
API[Foundation Model API] --> CHAT[Chat Completions]
API --> EMBED[Embeddings]
API --> COMPLETE[Text Completions]
CHAT --> SYSTEM[System Message]
CHAT --> USER_MSG[User Messages]
CHAT --> TOOLS_FC[Function Calling]
CHAT --> STREAM[Streaming]
EMBED --> SINGLE[Single Text]
EMBED --> BATCH[Batch Texts]
API:::governance
CHAT:::processing
EMBED:::storage
COMPLETE:::serving
SYSTEM:::ingestion
USER_MSG:::ingestion
TOOLS_FC:::source
STREAM:::source
SINGLE:::source
BATCH:::source
*Foundation Model API endpoints and supported interaction patterns.*
Key Terms
Prerequisites and Setup
- Databricks workspace with Foundation Model APIs enabled.
CAN_QUERYpermission on serving endpoints.- For external models: AI Gateway configured with provider API keys stored in Databricks Secrets.
Step-by-Step Implementation
Configuration Reference
| Parameter | Default | Description |
|---|---|---|
max_tokens | 256 | Maximum tokens in the response |
temperature | 1.0 | Sampling temperature |
top_p | 1.0 | Nucleus sampling threshold |
stop | [] | Stop sequences |
stream | false | Enable token-by-token streaming |
n | 1 | Number of completions to generate |
frequency_penalty | 0 | Penalty for repeated tokens |
presence_penalty | 0 | Penalty for discussing new topics |