callbacks — Observer pattern for tracing, logging, streaming, metrics

Observer Tracing LangSmith

Architecture

flowchart LR
  subgraph Core["langchain_classic/callbacks/"]
    BASE["Base abstraction"]
    IMPLS["Concrete implementations"]
    BASE --> IMPLS
  end
  subgraph Up["Upstream callers"]
    U1["chains/"]
    U2["agents/"]
  end
  subgraph Down["Downstream"]
    D1["langchain_core"]
    D2["External providers"]
  end
  Up --> Core
  Core --> Down

Data Flow

sequenceDiagram
  participant Caller as Caller
  participant Callback as callbacks
  participant Core as langchain_core
  participant Ext as External
  Caller->>Callback: invoke(input)
  Note over Callback: Validate & route
  Callback->>Core: delegate primitive
  Core->>Ext: provider call
  Ext-->>Core: response
  Core-->>Callback: typed output
  Callback-->>Caller: result

Existence rationale

callbacks/ implements the observer pattern that lets every chain, agent, retriever, LLM call, and tool emit lifecycle events (on_*_start, on_*_end, on_*_error, on_*_new_token) to N subscribers without coupling. Without it, streaming UIs, distributed tracing (LangSmith), token cost accounting, and verbose=True debug printing would each need bespoke instrumentation in every component. By centralising on BaseCallbackHandler, langchain_classic lets a single CallbackManager fan an event out to a stdout printer, a file logger, a LangSmith tracer, and a streaming WebSocket — concurrently.

Design decisions

Algorithm deep-dives

1. Manager configuration & dispatch

Trace.

1. Caller calls manager = CallbackManager.configure(inheritable_callbacks, local_callbacks, verbose, inheritable_tags, …).
2. configure merges global handlers (langchain_classic.globals), inherited handlers from a parent run, and call-local handlers, dedup-by-id.
3. If verbose and no ConsoleCallbackHandler → adds one.
4. If env LANGCHAIN_TRACING_V2=true and no LangChainTracer → adds one.
5. Returned manager carries run_id, parent_run_id, tags, metadata.

On on_chain_start(...):

1. Generates a new run_id (UUID4).
2. For each handler with not handler.ignore_chain: try handler.on_chain_start(serialized, inputs, run_id, parent_run_id, tags, metadata).
3. Exceptions in handlers are caught + logged, never propagated — handler bugs must not crash the run.
4. Returns a CallbackManagerForChainRun whose parent_run_id = run_id so children form a tree.

Why per-call. Streaming UIs need a different handler set than batch processing; tracers need fresh run IDs per top-level call. A single global manager would either leak handlers across runs or require explicit teardown.

2. Streaming token dispatch

AsyncIteratorCallbackHandler is the bridge between LLM on_llm_new_token and an async generator the application consumes:

1. Maintains a asyncio.Queue.
2. on_llm_new_token(token) → queue.put_nowait(token).
3. on_llm_end → queue.put_nowait(SENTINEL).
4. The user's async loop iterates await queue.get() until SENTINEL.

This decouples LLM emission cadence from the consumer's read cadence — the LLM streams as fast as the network allows, the UI catches up on its own clock.

Error philosophy

Handler exceptions are quarantined. BaseCallbackManager._handle_event swallows handler exceptions and logs them — by design, a broken tracer must not break a working chain. Tools and chains, by contrast, propagate up — the application decides on retry.

Performance characteristics

• Hot-path cost: ~1 µs per handler per event (function call + ignore-flag check). With 3 handlers and 10 events per chain → ~30 µs of overhead on a multi-second LLM call. Negligible.
• Tracer cost: LangChainTracer posts to LangSmith asynchronously (fire-and-forget). Default queue is bounded; over capacity it drops events with a warning.
• Streaming: queue throughput is the bottleneck — tens of thousands of tokens/sec are easy.

Step-by-Step Walkthrough

What Could Go Wrong

Performance Profile

Directory Map

callbacks/
├── __init__.py
├── aim_callback.py
├── argilla_callback.py
├── arize_callback.py
├── arthur_callback.py
├── base.py
├── clearml_callback.py
├── comet_ml_callback.py
├── confident_callback.py
├── context_callback.py
├── file.py
├── flyte_callback.py
├── human.py
├── infino_callback.py
├── labelstudio_callback.py
├── llmonitor_callback.py
├── manager.py
├── mlflow_callback.py
├── openai_info.py
├── promptlayer_callback.py
├── sagemaker_callback.py
├── stdout.py
├── streaming_aiter.py
├── streaming_aiter_final_only.py
├── streaming_stdout.py
├── streaming_stdout_final_only.py
├── streamlit/
├── tracers/
├── trubrics_callback.py
├── utils.py
├── wandb_callback.py
└── whylabs_callback.py