output_parsers — Structured output extraction from LLM strings

Structured output extraction from LLM strings

Module langchain_classic/output_parsers/ — Structured output extraction from LLM strings. Part of the Models I/O group.

🎯 Why This Module Exists

This module is documented from the output_parsers_deep.md Phase 1B research artifact. See sections below for design decisions, algorithms, error philosophy, performance, and evolution clues.

Source files

Sub-directories

Models I/O

Architecture

flowchart LR
  subgraph Core["langchain_classic/output_parsers/"]
    BASE["Base abstraction"]
    IMPLS["Concrete implementations"]
    BASE --> IMPLS
  end
  subgraph Up["Upstream callers"]
    U1["chains/"]
    U2["agents/"]
  end
  subgraph Down["Downstream"]
    D1["langchain_core"]
    D2["External providers"]
  end
  Up --> Core
  Core --> Down

Data Flow

sequenceDiagram
  participant Caller as Caller
  participant Output_P as output_parsers
  participant Core as langchain_core
  participant Ext as External
  Caller->>Output_P: invoke(input)
  Note over Output_P: Validate & route
  Output_P->>Core: delegate primitive
  Core->>Ext: provider call
  Ext-->>Core: response
  Core-->>Output_P: typed output
  Output_P-->>Caller: result

Existence rationale

output_parsers/ turns raw LLM text into typed Python values: a Pydantic instance, a JSON dict, a list, an enum, a regex match group. The module exists because LLMs are unreliable serialisers — they emit JSON with markdown fences, forget closing braces, hallucinate fields, and occasionally refuse the format. The parsers wrap a forgiving extraction step in a Runnable so they slot into LCEL pipelines (prompt | llm | parser); two of them — OutputFixingParser and RetryOutputParser — go further, calling the LLM again with the failure to self-correct.

Design decisions

Decision	Choice	Alternatives	Rationale
Parser is a Runnable	`BaseOutputParser` is `Runnable[str, T]`	Plain function	Composes with LCEL; gets callbacks for free
`format_instructions` method	Each parser produces the prompt fragment that tells the LLM how to format its output	Manual prompt engineering	Keeps schema and instructions in sync (Pydantic-derived JSON-schema is the truth)
`parse_with_prompt` for self-correcting parsers	RetryOutputParser receives original prompt + LLM output to retry	Retry without context	Without the prompt, the corrector LLM doesn't know the target
Pydantic-based structure parser	`PydanticOutputParser[T]` validates against `T.schema()`	hand-written validators	Free validation + descriptive error messages
JSON markdown extraction	`parse_json_markdown` strips ```json fences before json.loads	strict json.loads	LLMs love markdown; tolerate it

Algorithm deep-dives

1. Retry vs Fix

OutputFixingParser:

parse(text) → on OutputParserException, call LLM with template "fix this output that failed to parse" + parser's format_instructions + the bad text.
Re-parse the fix.
Up to N retries.

RetryOutputParser:

Same flow, but the retry prompt also includes the original prompt — useful when failure was caused by misunderstanding the request, not just bad formatting.

The split matters: Fix is cheap (one extra LLM call with only the bad text + instructions), Retry is more expensive but recovers from semantic errors.

2. PydanticOutputParser

Trace.

format_instructions = boilerplate + json.dumps(model.schema()) + example.
parse(text):

Strip markdown fences.
json.loads.
Model(**parsed) — Pydantic v1 raises ValidationError.
On any exception → wrap in OutputParserException(message, llm_output=text).

The OutputParserException.send_to_llm = True flag tells OutputFixingParser whether the error is recoverable (default yes).

Error philosophy

Strict but recoverable. parse raises a typed exception with the offending text attached so a wrapping retry/fix parser can self-heal. Final raise is structured (OutputParserException) so chains can handle it specifically.

Step-by-Step Walkthrough

Follow a typical operation through the output_parsers/ module:

Caller invokes the public surface — usually .invoke(input) on a Runnable, or a legacy __call__(inputs_dict) on a Chain subclass.
Pydantic validation — every entry point is a BaseModel; constructor args, input dicts, and tool args_schema are validated before any expensive work runs. Fail-fast on type errors.
CallbackManager.configure — a fresh manager is built per call, fans on_*_start events to all subscribers (LangSmith tracer, stdout handler, custom logging), and stamps a UUID4 run_id for tree reconstruction.
output_parsers-specific logic — the module's core method runs: prompt rendering, retrieval, tool execution, memory load, document split, etc. See the deep-dive sections above.
Provider call — typically a single HTTP request to OpenAI/Anthropic/etc., or a local SDK call. Retries are handled by tenacity with exponential backoff and a configurable max-attempts knob.
Streaming fan-out — if the provider streams tokens, each chunk fires on_llm_new_token through the callback manager so downstream observers (UI, log, tracer) see partial output.
OutputParser pass — if a parser is attached, the raw provider output is parsed into a structured Python object; OutputParserException may trigger a retry chain (RetryWithErrorOutputParser).
Memory write-back — for chains/agents that have a BaseMemory attached, save_context(inputs, outputs) persists the turn before returning to the caller.
on_*_end fan-out — the callback manager closes the run span; tracers serialise the Run tree and ship it to LangSmith asynchronously.
Result returned — caller receives a typed object (AIMessage, dict, BaseModel, list[Document], ...) matching the module's declared output schema.

What Could Go Wrong

Failure Mode	Trigger	Detection	Recovery
Provider 5xx / rate-limit	OpenAI/Anthropic/etc. transient error	HTTP status from SDK	tenacity retries with exponential backoff up to `max_retries`; then surface to caller
Output parser failure	LLM returned malformed JSON / wrong schema	`OutputParserException` in parser	`RetryWithErrorOutputParser` re-prompts with the parser error included; or surface
Pydantic validation	Caller passes wrong-shaped dict	`ValidationError` at `__init__` / `invoke`	Surface immediately — no fallback is correct here
Callback handler exception	User-supplied tracer raises	Try/except around each fan-out	Logged and swallowed by default; `raise_error=True` to opt in to propagation
Provider auth missing	`OPENAI_API_KEY` unset	SDK `AuthenticationError`	Surface with a helpful message; no retry (would just fail again)

Performance Profile

Hot Paths

Provider HTTP round-trip dominates wall-clock time (100ms–30s depending on model and prompt length).
Pydantic validation is repeated on every invoke; cached schema parsing keeps it under 1ms for typical inputs.
Callback fan-out is synchronous by default — a slow handler will slow every call. Async tracers (LangSmith) push work to a background thread to avoid blocking.

Resource Usage

Memory: token-history buffers (in memory/) and document caches (in retrieval pipelines) are the largest steady-state allocations.
I/O: bounded by provider rate limits; vector store I/O for retrieval-flavored modules.
Concurrency: async support via a* methods; agents and parallel chains use asyncio.gather for concurrent tool calls.

Directory Map

Source tree for output_parsers

output_parsers/
├── __init__.py
├── boolean.py
├── combining.py
├── datetime.py
├── enum.py
├── ernie_functions.py
├── fix.py
├── format_instructions.py
├── json.py
├── list.py
├── loading.py
├── openai_functions.py
├── openai_tools.py
├── pandas_dataframe.py
├── prompts.py
├── pydantic.py
├── rail_parser.py
├── regex.py
├── regex_dict.py
├── retry.py
├── structured.py
├── xml.py
└── yaml.py