samples/nemo-guardrails_1 — awesome-ai-stack

Example

NeMo Guardrails — implementation sample

A real, runnable mini-project. Download it and run with Docker.

View samples/nemo-guardrails_1 on GitHub ↗

June 28, 2026

NeMo Guardrails — implementation sample

A tiny NeMo Guardrails sample: it loads a Colang greeting rail and generates a reply, so the bot answers through a fixed, approved message instead of free-form output. The main and embedding models are both OpenAI, so no local model download is needed.

Configure

cd samples/nemo-guardrails_1
cp .env.sample .env
# edit .env: set OPENAI_API_KEY (used for the main + embedding models)

Run with Docker

cd samples/nemo-guardrails_1
docker build -t aas-nemo .
docker run --rm --env-file .env aas-nemo

Run with Docker (in a devcontainer with DooD)

The foreground docker run may print nothing under Docker-outside-of-Docker — run detached and follow the logs:

docker logs -f "$(docker run -d --env-file .env aas-nemo)"

Run locally

cd samples/nemo-guardrails_1
pip install -r requirements.txt
python app.py

Example run

The greeting rail maps the user's "hi there" to a fixed bot message, so the reply is the same every run:

bot: Hello! I'm a guardrailed assistant — I only respond through approved rails.

.env.sample

# Copy to `.env` and add your OpenAI API key (used for the main + embedding models).
#   cp .env.sample .env
OPENAI_API_KEY=

MODEL=gpt-4o-mini

app.py

"""NeMo Guardrails: put programmable rails around an LLM.

Loads a tiny rails config — a greeting flow written in Colang — and generates a
reply. The rail maps a user greeting to a fixed, safe bot message instead of
free-form output, showing how rails steer a conversation. Both the main and the
embedding model are OpenAI, so only OPENAI_API_KEY is needed (no local download).

Run:
    docker build -t aas-nemo .
    docker run --rm --env-file .env aas-nemo
"""

import os

from dotenv import load_dotenv
from nemoguardrails import LLMRails, RailsConfig

load_dotenv()

MODEL = os.environ.get("MODEL", "gpt-4o-mini")

YAML_CONFIG = f"""
models:
  - type: main
    engine: openai
    model: {MODEL}
  - type: embeddings
    engine: openai
    model: text-embedding-3-small
"""

COLANG_CONFIG = """
define user express greeting
  "hello"
  "hi there"
  "hey"

define bot express greeting
  "Hello! I'm a guardrailed assistant — I only respond through approved rails."

define flow greeting
  user express greeting
  bot express greeting
"""


def main() -> None:
    config = RailsConfig.from_content(
        yaml_content=YAML_CONFIG,
        colang_content=COLANG_CONFIG,
    )
    rails = LLMRails(config)

    response = rails.generate(messages=[{"role": "user", "content": "hi there"}])
    print("bot:", response["content"])


if __name__ == "__main__":
    main()

Dockerfile

FROM python:3.12-slim

ENV PYTHONUNBUFFERED=1

WORKDIR /app

# NeMo Guardrails pulls `annoy`, which ships no wheels and compiles from source.
RUN apt-get update \
    && apt-get install -y --no-install-recommends build-essential \
    && rm -rf /var/lib/apt/lists/*

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY app.py .

ENTRYPOINT ["python", "app.py"]