samples/litellm_1 — awesome-ai-stack

Example

LiteLLM — one interface, any provider

A real, runnable mini-project. Download it and run with Docker.

View samples/litellm_1 on GitHub ↗

June 28, 2026

LiteLLM — one interface, any provider

A tiny LiteLLM script: it sends one prompt through litellm.completion() and prints the reply. The same call works with Anthropic Claude, OpenAI, or Google AI Studio (Gemini) — change MODEL in .env, never the code.

Configure

cd samples/litellm_1
cp .env.sample .env
# edit .env: set MODEL and the matching provider key

MODEL picks the provider:

Provider	`MODEL` example	Key in `.env`
Anthropic Claude	`claude-opus-4-8`	`ANTHROPIC_API_KEY`
OpenAI	`gpt-4o`	`OPENAI_API_KEY`
Google AI Studio	`gemini/gemini-2.5-flash`	`GEMINI_API_KEY`
Ollama (local)	`ollama_chat/qwen3.5:9b`	`OLLAMA_API_BASE`

.env is gitignored — only .env.sample is committed.

Ollama (local models): first pull the model on the host — ollama pull qwen3.5:9b (or ollama run qwen3.5:9b). Then set MODEL=ollama_chat/qwen3.5:9b and point OLLAMA_API_BASE at the server — no API key needed. In a devcontainer with DooD the container reaches the host's Ollama at http://host.docker.internal:11434; running locally use http://localhost:11434.

Run with Docker

cd samples/litellm_1
docker build -t aas-litellm .
docker run --rm --env-file .env aas-litellm "Summarize litellm in one line."

Run with Docker (in a devcontainer with DooD)

In a dev container that talks to the host Docker daemon (Docker-outside-of-Docker), the foreground docker run above often prints nothing and exits 0 — but the run itself succeeds. The script runs to completion and Docker captures all of its output; only the live attached stream drops it over the VM boundary. You can confirm this: docker logs on the same container shows the full output, the container exits 0, and it is not an OOM. Run detached and follow the logs instead:

cd samples/litellm_1
docker build -t aas-litellm .
docker logs -f "$(docker run -d --env-file .env aas-litellm \
  "Summarize litellm in one line.")"

Run locally

cd samples/litellm_1
pip install -r requirements.txt
python app.py "Summarize litellm in one line."

python-dotenv loads .env automatically. Get keys from Anthropic, OpenAI, or Google AI Studio.

Example run

Output varies by model and run — LLMs are non-deterministic, so the exact wording (and an agent's steps) differ each time. Below is one run with claude-opus-4-8.

LiteLLM is an open-source library that provides a unified interface to call 100+ LLM APIs (OpenAI, Anthropic, Azure, Cohere, etc.) using the OpenAI format.

.env.sample

# Copy this file to `.env` and fill in the key for the provider you want.
#   cp .env.sample .env
#
# One codebase, many providers — LiteLLM routes the request based on MODEL.

# Pick the model. Examples:
#   claude-opus-4-8              Anthropic Claude   -> needs ANTHROPIC_API_KEY
#   gpt-4o                       OpenAI             -> needs OPENAI_API_KEY
#   gemini/gemini-2.5-flash      Google AI Studio   -> needs GEMINI_API_KEY
#   ollama_chat/qwen3.5:9b       Ollama (local)     -> needs OLLAMA_API_BASE, no key
MODEL=claude-opus-4-8

# Set only the key for the provider you chose; leave the rest blank.
ANTHROPIC_API_KEY=
OPENAI_API_KEY=
GEMINI_API_KEY=

# Ollama only — where the Ollama server runs (pull the model first, e.g.
# `ollama pull qwen3.5:9b`). Other providers ignore this line.
#   Local, no Docker:           http://localhost:11434
#   Docker / DooD on the host:  http://host.docker.internal:11434
OLLAMA_API_BASE=http://host.docker.internal:11434

app.py

"""LiteLLM: one provider-agnostic call for every model.

LiteLLM exposes 100+ providers behind a single OpenAI-style `completion()`
call. Pick the model with MODEL in `.env` (claude-opus-4-8, gpt-4o,
gemini/gemini-2.5-flash, …) and the calling code never changes — LiteLLM routes
the request to the right provider and normalizes the response back.

Run:
    docker build -t aas-litellm .
    docker run --rm --env-file .env aas-litellm "Summarize litellm in one line."
"""

import os
import sys

import litellm
from dotenv import load_dotenv

# Load .env when running locally; in Docker the keys come from --env-file.
load_dotenv()

# Drop params a given provider doesn't support, so one call works everywhere.
litellm.drop_params = True


DEFAULT_PROMPT = "Summarize what LiteLLM does in one sentence."


def main() -> None:
    prompt = " ".join(sys.argv[1:]) or DEFAULT_PROMPT

    # MODEL chooses the provider (claude-opus-4-8 / gpt-4o / gemini/gemini-2.5-flash).
    model = os.environ.get("MODEL", "claude-opus-4-8")

    resp = litellm.completion(
        model=model,
        messages=[{"role": "user", "content": prompt}],
    )
    print(resp.choices[0].message.content)


if __name__ == "__main__":
    main()

Dockerfile

# Minimal image for the LiteLLM sample.
FROM python:3.13-slim

# Stream stdout/stderr unbuffered so the output appears live.
ENV PYTHONUNBUFFERED=1

WORKDIR /app

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY app.py .

# Provide your config at run time (.env is NOT baked into the image):
#   docker run --rm --env-file .env aas-litellm "<prompt>"
ENTRYPOINT ["python", "app.py"]