tools — BaseTool + 64 first-party tool integrations
BaseTool + 64 first-party tool integrations
Module langchain_classic/tools/ — BaseTool + 64 first-party tool integrations. Part of the State & Tools group.
🎯 Why This Module Exists
This module is documented from the tools_deep.md Phase 1B research artifact. See sections below for design decisions, algorithms, error philosophy, performance, and evolution clues.
BaseTool Pydantic schema Async-aware 64 tools
Architecture
flowchart LR
subgraph Core["langchain_classic/tools/"]
BASE["Base abstraction"]
IMPLS["Concrete implementations"]
BASE --> IMPLS
end
subgraph Up["Upstream callers"]
U1["chains/"]
U2["agents/"]
end
subgraph Down["Downstream"]
D1["langchain_core"]
D2["External providers"]
end
Up --> Core
Core --> Down
Data Flow
sequenceDiagram participant Caller as Caller participant Tools as tools participant Core as langchain_core participant Ext as External Caller->>Tools: invoke(input) Note over Tools: Validate & route Tools->>Core: delegate primitive Core->>Ext: provider call Ext-->>Core: response Core-->>Tools: typed output Tools-->>Caller: result
Existence rationale
tools/ provides the **BaseTool ABC and 60+ integration sub-modules** that connect agents to the outside world: search engines, file systems, SQL DBs, REST APIs, browsers, email, calendars, code interpreters, and more. The existence of a separate tools/ module — rather than baking calls directly into agents — is what makes LangChain's "I gave my LLM a tool" model uniform: every tool exposes the same (name, description, args_schema, _run, _arun) shape, so the agent only needs to know how to format the call. Without tools/, agent flavours would each ship their own tool protocol, and integrators would need a separate adapter per agent type.
Design decisions visible in the code
| Decision | Choice made | Plausible alternatives | Inferred rationale | ||
|---|---|---|---|---|---|
| Tool definition shape | BaseTool ABC: name, description, args_schema (Pydantic), _run/_arun | Plain functions with introspection; Protocol typing | Pydantic schema → JSON-schema → OpenAI function spec without re-implementation; description used in prompts | ||
| Sync + async dual surface | Both _run and _arun; default _arun runs _run in a thread | Async-only | Many integrations are blocking (subprocess, requests); thread fallback prevents deadlocks | ||
| Decorator path | @tool decorator infers schema from type hints | Manual StructuredTool.from_function | Lowers the bar for one-off tools; still produces a real BaseTool | ||
| Toolkits | BaseToolkit.get_tools() returns related tools | Just import functions | Some integrations (Gmail, JIRA, SQL) come as a family — toolkit is the discovery surface | ||
| Render functions | render_text_description, render_text_description_and_args, convert_to_openai_function/tool | Inline string concat in agents | Lets each agent flavour render the same tool list to its preferred format | ||
| Tool exception handling | Tools may set `handle_tool_error: bool \ | str \ | Callable` | Always raise | Lets a tool decide whether its errors should kill the agent or be returned as observation |
| Lazy integration loading | tools/__init__.py uses __getattr__ to forward to langchain_community.tools | Eager imports | Prevents the heavy dependency tail (Selenium, boto3, GMail SDK) from being imported on import langchain_classic | ||
| Reserved root tools | A few tools live in the package root (requests.py, python.py, serpapi.py) | Move them under tools/ | Historical: they predate the tools/ package and are kept at root for stable imports |
Algorithm deep-dives
1. Tool argument validation
Trace.
BaseTool.run(tool_input)accepts either a string (single-arg tools) or a dict (multi-arg)._to_args_and_kwargs(tool_input)decides which and validates againstargs_schema.- On validation error: if
handle_tool_errorset → return formatted error string; else raise. _run(*args, **kwargs)is invoked with validated values.- Result is returned (or wrapped in
ToolMessagefor chat agents).
Why Pydantic. Three benefits at once: (a) descriptive error messages an LLM can self-correct from, (b) automatic JSON-schema generation for OpenAI/Anthropic tool calling, (c) field descriptions become documentation in the agent prompt.
2. convert_to_openai_function / convert_to_openai_tool
Problem. OpenAI's Chat Completions API needs {"name": ..., "description": ..., "parameters": <jsonschema>}. We have a BaseTool. Bridge.
Trace.
- Read
tool.name,tool.description,tool.args_schema. - Pydantic v1:
args_schema.schema()→ JSON-schema dict. - Strip
titleproperties anddefinitionsreferences that OpenAI rejects. - Wrap in
{"type": "function", "function": {...}}for the tools-API; leave bare for the legacy functions API.
Edge case. Tools without args_schema (single string input) get a synthesised {"type": "object", "properties": {"__arg1": {"type": "string"}}, "required": ["__arg1"]} — required because OpenAI's spec mandates an object schema.
3. render_text_description_and_args
Problem. ReAct/MRKL agents expect tool docs inline in the prompt, like "calculator: a calculator. Input: a math expression.". Generate this from a list of BaseTools.
Trace. Iterate tools → for each, format f"{tool.name}: {tool.description}", optionally followed by a JSON dump of args_schema. Returned string is fed into the prompt template at {tools} placeholder.
This is what makes a single tools=[…] argument work across flavours: each flavour calls a different render function but all consume the same BaseTool list.
Error philosophy
Tools are at the trust boundary — anything from a network search to executing Python code. The module's policy:
- Validation errors are user-facing by default (they indicate bad agent output) but can be downgraded to an observation via
handle_tool_error. - Runtime errors (HTTP 500, file not found) propagate by default — the agent operator must decide whether to retry or report.
- **
PythonREPLTool** (executing arbitrary code) emits a deprecation warning steering toward sandboxed alternatives likelangchain_experimental's sandboxed REPL. - Permission-sensitive tools (file_management, requests) accept allowlists / denylists or
root_dirconstraints.
Performance characteristics
- Lazy import wins big. Importing
langchain_classic.toolsis fast because individual integrations are loaded on first attribute access. - Async parallelism via
_arunlets multi-action agents fire 5+ HTTP/search tools concurrently. - Args validation is microseconds (Pydantic v1, single-shot model construction).
- The expensive bit is whatever the tool actually does — network, subprocess, search.
Evolution clues
- The dual presence of
convert_to_openai_function(function-calling era) andconvert_to_openai_tool(tools-API era) reflects OpenAI's API evolution. - Many sub-dirs are single-file integrations (
tools/jira/tool.py,tools/github/tool.py) — these are essentially thin wrappers overlangchain_communityplus a stable import path. tools/retriever.pyandtools/render.pyare the algorithmic core of the module; the rest is integration scaffolding.- The
BaseToolschema has accreted attributes (return_direct,verbose,callbacks,tags,metadata,handle_tool_error,handle_validation_error) — each addition usually maps to a feature an agent needed (e.g.,return_direct=Trueshort-circuits the loop after a known final tool).
Step-by-Step Walkthrough
Follow a typical operation through the tools/ module:
-
Caller invokes the public surface — usually
.invoke(input)on a Runnable, or a legacy__call__(inputs_dict)on aChainsubclass. - Pydantic validation — every entry point is a BaseModel; constructor args, input dicts, and tool args_schema are validated before any expensive work runs. Fail-fast on type errors.
-
CallbackManager.configure — a fresh manager is built per call, fans
on_*_startevents to all subscribers (LangSmith tracer, stdout handler, custom logging), and stamps a UUID4run_idfor tree reconstruction. - tools-specific logic — the module's core method runs: prompt rendering, retrieval, tool execution, memory load, document split, etc. See the deep-dive sections above.
- Provider call — typically a single HTTP request to OpenAI/Anthropic/etc., or a local SDK call. Retries are handled by tenacity with exponential backoff and a configurable max-attempts knob.
-
Streaming fan-out — if the provider streams tokens, each chunk fires
on_llm_new_tokenthrough the callback manager so downstream observers (UI, log, tracer) see partial output. -
OutputParser pass — if a parser is attached, the raw provider output is parsed into a structured Python object;
OutputParserExceptionmay trigger a retry chain (RetryWithErrorOutputParser). -
Memory write-back — for chains/agents that have a
BaseMemoryattached,save_context(inputs, outputs)persists the turn before returning to the caller. -
on_*_end fan-out — the callback manager closes the run span; tracers serialise the
Runtree and ship it to LangSmith asynchronously. - Result returned — caller receives a typed object (AIMessage, dict, BaseModel, list[Document], ...) matching the module's declared output schema.
What Could Go Wrong
| Failure Mode | Trigger | Detection | Recovery |
|---|---|---|---|
| Provider 5xx / rate-limit | OpenAI/Anthropic/etc. transient error | HTTP status from SDK | tenacity retries with exponential backoff up to max_retries; then surface to caller |
| Output parser failure | LLM returned malformed JSON / wrong schema |
OutputParserException in parser |
RetryWithErrorOutputParser re-prompts with the parser error included; or surface |
| Pydantic validation | Caller passes wrong-shaped dict |
ValidationError at __init__ / invoke
|
Surface immediately — no fallback is correct here |
| Callback handler exception | User-supplied tracer raises | Try/except around each fan-out | Logged and swallowed by default; raise_error=True to opt in to propagation |
| Provider auth missing |
OPENAI_API_KEY unset |
SDK AuthenticationError
|
Surface with a helpful message; no retry (would just fail again) |
Performance Profile
Hot Paths
- Provider HTTP round-trip dominates wall-clock time (100ms–30s depending on model and prompt length).
-
Pydantic validation is repeated on every
invoke; cached schema parsing keeps it under 1ms for typical inputs. - Callback fan-out is synchronous by default — a slow handler will slow every call. Async tracers (LangSmith) push work to a background thread to avoid blocking.
Resource Usage
-
Memory: token-history buffers (in
memory/) and document caches (in retrieval pipelines) are the largest steady-state allocations. - I/O: bounded by provider rate limits; vector store I/O for retrieval-flavored modules.
-
Concurrency: async support via
a*methods; agents and parallel chains useasyncio.gatherfor concurrent tool calls.
Directory Map
Source tree for tools
tools/
├── __init__.py
├── ainetwork/
├── amadeus/
├── arxiv/
├── azure_cognitive_services/
├── base.py
├── bearly/
├── bing_search/
├── brave_search/
├── clickup/
├── convert_to_openai.py
├── dataforseo_api_search/
├── ddg_search/
├── e2b_data_analysis/
├── edenai/
├── eleven_labs/
├── file_management/
├── github/
├── gitlab/
├── gmail/
├── golden_query/
├── google_cloud/
├── google_finance/
├── google_jobs/
├── google_lens/
├── google_places/
├── google_scholar/
├── google_search/
├── google_serper/
├── google_trends/
├── graphql/
├── human/
├── ifttt.py
├── interaction/
├── jira/
├── json/
├── memorize/
├── merriam_webster/
├── metaphor_search/
├── multion/
├── nasa/
├── nuclia/
├── office365/
├── openapi/
├── openweathermap/
├── playwright/
├── plugin.py
├── powerbi/
├── pubmed/
├── python/
├── reddit_search/
├── render.py
├── requests/
├── retriever.py
├── scenexplain/
├── searchapi/
├── searx_search/
├── shell/
├── slack/
├── sleep/
... (more files truncated)