samples/firecrawl_1 — awesome-ai-stack

Example

Firecrawl web-scraping agent — a docs page to Markdown

A real, runnable mini-project. Download it and run with Docker.

View samples/firecrawl_1 on GitHub ↗

June 28, 2026

Firecrawl web-scraping agent — a docs page to Markdown

A ~45-line LangGraph ReAct agent with a single scrape tool backed by Firecrawl. Give it a task that needs a page's contents — "read this docs page and summarize it" — and it fetches the page as clean Markdown, reads it, and answers. The model is routed through LiteLLM, so the same code works with Anthropic Claude, OpenAI, or Google AI Studio (Gemini) — change MODEL in .env, never the code.

Configure

cd samples/firecrawl_1
cp .env.sample .env
# edit .env: set FIRECRAWL_API_KEY, MODEL, and the matching provider key

FIRECRAWL_API_KEY is always required — it's the scrape tool. Then MODEL picks the provider:

Provider	`MODEL` example	Key in `.env`
Anthropic Claude	`claude-opus-4-8`	`ANTHROPIC_API_KEY`
OpenAI	`gpt-4o`	`OPENAI_API_KEY`
Google AI Studio	`gemini/gemini-2.5-flash`	`GEMINI_API_KEY`

.env is gitignored — only .env.sample is committed.

Run with Docker

cd samples/firecrawl_1
docker build -t aas-firecrawl .
docker run --rm --env-file .env aas-firecrawl \
  "https://docs.firecrawl.dev 를 읽고 핵심 기능 3가지만 요약해줘"

Run with Docker (in a devcontainer with DooD)

In a dev container that talks to the host Docker daemon (Docker-outside-of-Docker), the foreground docker run above often prints nothing and exits 0 — but the run itself succeeds. The script runs to completion and Docker captures all of its output; only the live attached stream drops it over the VM boundary. You can confirm this: docker logs on the same container shows the full output, the container exits 0, and it is not an OOM. Run detached and follow the logs instead:

cd samples/firecrawl_1
docker build -t aas-firecrawl .
docker logs -f "$(docker run -d --env-file .env aas-firecrawl \
  "https://docs.firecrawl.dev 를 읽고 핵심 기능 3가지만 요약해줘")"

Run locally

cd samples/firecrawl_1
pip install -r requirements.txt
python app.py "https://docs.firecrawl.dev 를 읽고 핵심 기능 3가지만 요약해줘"

python-dotenv loads .env automatically. Get the Firecrawl key from Firecrawl, and model keys from Anthropic, OpenAI, or Google AI Studio.

How it works

Search finds where to look; scraping reads the page in. The loop is the spine, the tool is the reader.

The model reads the task and the scrape tool's schema.
It calls scrape with the URL it needs to read.
Firecrawl fetches the page and returns clean Markdown.
The model reads that Markdown and writes the answer.

Without the tool the model can't see past a link; with it, the page's real contents go straight into its reasoning.

Example run

Output varies by model and run — LLMs are non-deterministic, so the exact wording (and an agent's steps) differ each time. Below is one run with gemini/gemini-2.5-flash (the answer is Korean because the question is).

Firecrawl의 핵심 기능 3가지는 다음과 같습니다:

1. **웹 검색 (Web Search):** 웹에서 정보를 검색하는 기능입니다.
2. **스크래핑 (Scraping):** 웹 페이지에서 데이터를 추출하는 기능입니다.
3. **에이전트 (Agent):** 웹에서 데이터를 수집하는 데 사용되는 에이전트 기능입니다.

.env.sample

# Copy this file to `.env` and fill in the keys.
#   cp .env.sample .env

# Firecrawl API key — always required (it's the scrape tool).
# Get one at https://firecrawl.dev
FIRECRAWL_API_KEY=

# Pick the model — LiteLLM routes the request based on MODEL. Examples:
#   claude-opus-4-8              Anthropic Claude   -> needs ANTHROPIC_API_KEY
#   gpt-4o                       OpenAI             -> needs OPENAI_API_KEY
#   gemini/gemini-2.5-flash      Google AI Studio   -> needs GEMINI_API_KEY
MODEL=claude-opus-4-8

# Set only the key for the provider you chose; leave the rest blank.
ANTHROPIC_API_KEY=
OPENAI_API_KEY=
GEMINI_API_KEY=

app.py

"""Minimal web-scraping agent: a LangGraph ReAct loop with a Firecrawl scrape tool.

Given a task that needs the contents of a web page — e.g. "read this docs page
and summarize it" — the agent calls the `scrape` tool (Firecrawl), which fetches
the page and returns clean Markdown, then reads it and answers. The model is
routed through LiteLLM, so the same code runs on Anthropic Claude, OpenAI, or
Google AI Studio (Gemini); pick one via the MODEL env var (see .env.sample).

Run:
    docker build -t aas-firecrawl .
    docker run --rm --env-file .env aas-firecrawl \
        "https://docs.firecrawl.dev 를 읽고 핵심 기능 3가지만 요약해줘"
"""

import os
import sys

import litellm
from dotenv import load_dotenv
from langchain.agents import create_agent
from langchain_litellm import ChatLiteLLM
from langchain_core.tools import tool
from firecrawl import FirecrawlApp

# Load .env when running locally; in Docker the keys come from --env-file.
load_dotenv()

# Drop params a given provider doesn't support, so the same code runs across
# OpenAI / Claude / Gemini.
litellm.drop_params = True

_firecrawl = FirecrawlApp(api_key=os.environ.get("FIRECRAWL_API_KEY", ""))


@tool
def scrape(url: str) -> str:
    """Fetch a web page and return its content as clean Markdown.

    Use this to read what's actually behind a URL — docs, articles, anything the
    model can't see on its own. Pass a single absolute URL.
    """
    res = _firecrawl.scrape_url(url, formats=["markdown"])
    md = getattr(res, "markdown", None)
    if md is None and isinstance(res, dict):  # older SDKs return a dict
        md = res.get("markdown") or res.get("data", {}).get("markdown")
    # Cap the length so a long page doesn't blow up the prompt.
    return (md or "")[:6000] or "No content."


def message_text(content) -> str:
    """Flatten an assistant message's content to plain text (cloud models return a
    string; some local models return a list of blocks)."""
    if isinstance(content, list):
        return "".join(
            part.get("text", "")
            for part in content
            if isinstance(part, dict) and part.get("type") == "text"
        )
    return content


def main() -> None:
    question = " ".join(sys.argv[1:]) or "https://docs.firecrawl.dev 를 읽고 핵심 기능 3가지만 요약해줘"

    # MODEL chooses the provider (claude-opus-4-8 / gpt-4o / gemini/gemini-2.5-flash).
    model = ChatLiteLLM(model=os.environ.get("MODEL", "claude-opus-4-8"), temperature=0)
    agent = create_agent(model, tools=[scrape])

    result = agent.invoke({"messages": [{"role": "user", "content": question}]})
    print(message_text(result["messages"][-1].content))


if __name__ == "__main__":
    main()

Dockerfile

# Minimal image for the Firecrawl web-scraping sample agent.
FROM python:3.13-slim

# Stream stdout/stderr unbuffered so the agent's output appears live (and isn't
# lost if the container is killed mid-run, e.g. under low memory).
ENV PYTHONUNBUFFERED=1

WORKDIR /app

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY app.py .

# Provide your config at run time (.env is NOT baked into the image):
#   docker run --rm --env-file .env aas-firecrawl "<task>"
ENTRYPOINT ["python", "app.py"]