Skip to content

AI Gateways

OpenRouter

  • It's not a provider, but behaves like one
  • It's a unified interface for LLMs. Offers a standard API format (OpenAI-compatible)
  • Adopts a single interface (e.g., for authentication, billing, error, etc)
  • https://openrouter.ai/

LiteLLM

from litellm import completion
import os

os.environ["OPENAI_API_KEY"] = "your-openai-key"
os.environ["ANTHROPIC_API_KEY"] = "your-anthropic-key"

# OpenAI
response = completion(model="openai/gpt-4o", messages=[{"role": "user", "content": "Hello!"}])

# Anthropic
response = completion(model="anthropic/claude-sonnet-4-20250514", messages=[{"role": "user", "content": "Hello!"}])

# ~/.zshrc
export LITELLM_API_KEY='...'
export LITELLM_BASE_URL='https://mylitellm.com'

# Anthropic
export ANTHROPIC_BASE_URL="$LITELLM_BASE_URL/anthropic" # proxy to anthropic
export ANTHROPIC_API_KEY="$LITELLM_API_KEY"
curl "$ANTHROPIC_BASE_URL/v1/models" -H "x-api-key: $ANTHROPIC_API_KEY" -H "anthropic-version: 2023-06-01"

# OpenAI
export OPENAI_BASE_URL="$LITELLM_BASE_URL/openai" # proxy to openai
export OPENAI_API_KEY="$LITELLM_API_KEY"
curl "$OPENAI_BASE_URL/v1/models" -H "Authorization: Bearer $OPENAI_API_KEY"

# if you use LITELLM_BASE_URL it will use the LITELLM API directly (which is OpenAI compatible)

Groq

  • https://groq.com/
  • It's a service that hosts opensource models
  • It's an AI inference company that using their own hardware with LPU (Language Processing Unit), instead of GPUs
  • They expose their hardware via a cloud API (OpenAI-compatible) so you can call models like:
  • Llama 3.3, Mixtral 8x, Gemma 2, Whisper
from groq import Groq

client = Groq()
response = client.chat.completions.create(
    model="llama-3.3-70b-versatile",
    messages=[{"role": "user", "content": "Explain RAG in one paragraph"}]
)