foundations — schema, _api, adapters, load, utils, graphs, chat_loaders

schema, _api, adapters, load, utils, graphs, chat_loaders

Module langchain_classic/foundations/ — schema, _api, adapters, load, utils, graphs, chat_loaders. Part of the Foundations group.

🎯 Why This Module Exists

This module is documented from the foundations_deep.md Phase 1B research artifact. See sections below for design decisions, algorithms, error philosophy, performance, and evolution clues.

Source files

Sub-directories

Foundations

Architecture

flowchart LR
  subgraph Core["langchain_classic/foundations/"]
    BASE["Base abstraction"]
    IMPLS["Concrete implementations"]
    BASE --> IMPLS
  end
  subgraph Up["Upstream callers"]
    U1["chains/"]
    U2["agents/"]
  end
  subgraph Down["Downstream"]
    D1["langchain_core"]
    D2["External providers"]
  end
  Up --> Core
  Core --> Down

Data Flow

sequenceDiagram
  participant Caller as Caller
  participant Foundati as foundations
  participant Core as langchain_core
  participant Ext as External
  Caller->>Foundati: invoke(input)
  Note over Foundati: Validate & route
  Foundati->>Core: delegate primitive
  Core->>Ext: provider call
  Ext-->>Core: response
  Core-->>Foundati: typed output
  Foundati-->>Caller: result

Existence rationale

These small modules are the plumbing: they don't hold the headline algorithms but they make the whole package interoperate.

load/ — Serializable mixin and dumps/loads so any chain/agent/tool can be JSON-round-tripped (used by LangSmith, hub, deployments).
_api/ — deprecation infrastructure (@deprecated, surface_langchain_deprecation_warnings, is_interactive_env) used package-wide.
schema/ — pure re-export of langchain_core types so legacy from langchain.schema import Document keeps working.
adapters/ — provider-shape adapters (adapters/openai.py translates between OpenAI-format messages and LangChain messages).
runnables/ — a few Runnables that pre-date or sit outside the chains/ hierarchy: OpenAIAssistantRunnable, create_openai_fn_runnable, create_structured_output_runnable.
utils/ — env helpers (get_from_dict_or_env), aiter helpers, format string helpers.
chat_loaders/ — converts platform-specific chat exports (Slack, Telegram, WhatsApp, Discord) into BaseMessage lists for fine-tuning datasets.
graphs/ — drivers for graph DB query (Neo4jGraph, NetworkxEntityGraph, KuzuGraph, NebulaGraph) used by chains/graph_qa/.
utilities/ — API-wrapper objects (SerpAPIWrapper, GoogleSearchAPIWrapper, WikipediaAPIWrapper, SQLDatabase) that tools wrap.
smith/ — LangSmith evaluation runner: run_on_dataset(dataset_name, llm_or_chain_factory, evaluators=[…]).

Design decisions

Module	Decision	Why
`load/`	`Serializable.is_lc_serializable() -> bool` + `__get_lc_namespace__()`	Lets serializer reconstruct the class from its declared module path even after refactors
`load/`	`secrets_map` indirection	Keep API keys out of dumps; load reinjects from env vars
`_api/`	`is_interactive_env()` suppresses warnings	Prevents `__getattr__` warnings polluting Jupyter / REPL outputs
`_api/`	`@deprecated(since="0.1.0", alternative="…")`	Standard, machine-readable deprecation metadata
`schema/`	Pure re-export	Backward compat for pre-0.1 import paths
`adapters/openai.py`	Translate dict messages ↔ `BaseMessage`	Lets users feed OpenAI-format JSON into LangChain pipelines
`runnables/openai_assistant.py`	Wraps the Assistants API as a single Runnable	The Assistant has its own server-side loop; we just adapt the surface
`chat_loaders/`	Each loader returns `Iterator[ChatSession]`	Streaming loads of multi-GB exports
`graphs/`	Each graph wrapper exposes `query(cypher) -> list[dict]` and `schema` property	Uniform interface so graph_qa chains are vendor-agnostic
`utilities/`	API wrappers separated from `tools/`	Reusable in non-tool contexts (e.g., directly inside a chain)
`smith/`	Dataset-driven evaluation	LangSmith's UI consumes dataset + run + feedback triples

Algorithm deep-dive — `Serializable` round-trip

Trace. dumps(chain):

Walk the instance recursively. For each Pydantic field:

If isinstance(v, Serializable) and v.is_lc_serializable(): recurse to _lc_kwargs.
Else: try json.dumps(v); if that fails, raise.

Build {"lc": 1, "type": "constructor", "id": [*ns, class_name], "kwargs": {...}}.

loads(s):

Parse JSON.
Look up (*ns, class_name) via import_class_from_namespace.
Reconstruct: Class(**rebuilt_kwargs) — recursing for Serializable subobjects.
Reinject secrets from environment if secrets_map declared.

Why a typed envelope. Forward compatibility — adding a field to a chain doesn't break old dumps; renaming a class can be supported via a registry remap.

Error philosophy

Foundation code prefers silent fallback to crashing user code:

Dump-time encountering an unknown type? Raise with a clear "register a serializer or implement __lc_kwargs__".
Deprecation warning? Suppressed in interactive envs to keep notebooks readable.
API wrapper missing optional dep? Lazy import; raise only when a method is called.

Step-by-Step Walkthrough

Follow a typical operation through the foundations/ module:

Caller invokes the public surface — usually .invoke(input) on a Runnable, or a legacy __call__(inputs_dict) on a Chain subclass.
Pydantic validation — every entry point is a BaseModel; constructor args, input dicts, and tool args_schema are validated before any expensive work runs. Fail-fast on type errors.
CallbackManager.configure — a fresh manager is built per call, fans on_*_start events to all subscribers (LangSmith tracer, stdout handler, custom logging), and stamps a UUID4 run_id for tree reconstruction.
foundations-specific logic — the module's core method runs: prompt rendering, retrieval, tool execution, memory load, document split, etc. See the deep-dive sections above.
Provider call — typically a single HTTP request to OpenAI/Anthropic/etc., or a local SDK call. Retries are handled by tenacity with exponential backoff and a configurable max-attempts knob.
Streaming fan-out — if the provider streams tokens, each chunk fires on_llm_new_token through the callback manager so downstream observers (UI, log, tracer) see partial output.
OutputParser pass — if a parser is attached, the raw provider output is parsed into a structured Python object; OutputParserException may trigger a retry chain (RetryWithErrorOutputParser).
Memory write-back — for chains/agents that have a BaseMemory attached, save_context(inputs, outputs) persists the turn before returning to the caller.
on_*_end fan-out — the callback manager closes the run span; tracers serialise the Run tree and ship it to LangSmith asynchronously.
Result returned — caller receives a typed object (AIMessage, dict, BaseModel, list[Document], ...) matching the module's declared output schema.

What Could Go Wrong

Failure Mode	Trigger	Detection	Recovery
Provider 5xx / rate-limit	OpenAI/Anthropic/etc. transient error	HTTP status from SDK	tenacity retries with exponential backoff up to `max_retries`; then surface to caller
Output parser failure	LLM returned malformed JSON / wrong schema	`OutputParserException` in parser	`RetryWithErrorOutputParser` re-prompts with the parser error included; or surface
Pydantic validation	Caller passes wrong-shaped dict	`ValidationError` at `__init__` / `invoke`	Surface immediately — no fallback is correct here
Callback handler exception	User-supplied tracer raises	Try/except around each fan-out	Logged and swallowed by default; `raise_error=True` to opt in to propagation
Provider auth missing	`OPENAI_API_KEY` unset	SDK `AuthenticationError`	Surface with a helpful message; no retry (would just fail again)

Performance Profile

Hot Paths

Provider HTTP round-trip dominates wall-clock time (100ms–30s depending on model and prompt length).
Pydantic validation is repeated on every invoke; cached schema parsing keeps it under 1ms for typical inputs.
Callback fan-out is synchronous by default — a slow handler will slow every call. Async tracers (LangSmith) push work to a background thread to avoid blocking.

Resource Usage

Memory: token-history buffers (in memory/) and document caches (in retrieval pipelines) are the largest steady-state allocations.
I/O: bounded by provider rate limits; vector store I/O for retrieval-flavored modules.
Concurrency: async support via a* methods; agents and parallel chains use asyncio.gather for concurrent tool calls.

Directory Map

Source tree for foundations

foundations/  (source listing unavailable)