backtesting/engine/ — Simulation Engine & Controller
Orchestrates day-by-day historical simulation with the full agent pipeline
The BacktestEngine orchestrates the full backtest run — date range iteration, agent invocation, trade execution, and metric aggregation. The AgentController normalizes agent I/O for each simulation step. The TradeExecutor translates Portfolio Tracking & Performance Metrics" href="../portfolio/index.html">portfolio-manager decisions (buy/sell/short/cover) into actual position changes. Together they create a realistic historical simulation with no look-ahead bias.
📚 What is this & why does it exist?
The engine is the core loop of the backtesting system. It iterates through each trading day in a historical range, running the complete LangGraph agent pipeline as if it were live — agents can only see data up to that day, preventing future-data leakage. The controller manages agent invocation and output normalization, the engine manages the day loop (fetch prices, call agents, execute trades, compute value), and the trader translates decisions into portfolio changes. This three-layer separation keeps each concern clean, testable, and independently replaceable.
💡 Real-world analogy
Think of a time machine for trading. The engine "travels" to each historical date, runs the AI agents as if they were making real-time decisions, then "travels" forward to the next business day. The BacktestEngine is the time machine itself — it transports you to each date, sets the scene (fetches prices, prepares the portfolio snapshot), and records what happened. The AgentController is the machine operator — it programs the destination parameters (which tickers, which analysts, which date range) and ensures the crew follows protocol. The TradeExecutor is the crew member who actually pulls the trading lever at each stop, converting abstract decisions ("buy 50 shares of AAPL") into concrete portfolio mutations.
🤔 Why Is It Designed This Way?
The engine follows a pipeline-of-components pattern where each class owns exactly one responsibility. The BacktestEngine could have been a single monolithic class that fetches data, calls agents, executes trades, and computes metrics all in one loop body. Instead, it delegates to specialized components — AgentController for agent I/O normalization, TradeExecutor for position mutations, PerformanceMetricsCalculator for analytics, and OutputBuilder for display. This decomposition was driven by testability: each component can be unit-tested in isolation with mock inputs. The trade-off is more indirection (you must trace across 4+ files to understand one day's flow), but this is acceptable because the flow is strictly linear — no callbacks, no branching pipelines.
| Design Choice | What Was Chosen | What Was Rejected | Why |
|---|---|---|---|
| Loop granularity | Day-by-day iteration with pd.date_range(freq="B") | Event-driven / tick-level | Agent pipeline is expensive (LLM calls); daily resolution matches typical hedge fund rebalancing frequency |
| Data prefetch | Bulk prefetch all tickers before loop | Lazy per-day fetching | API calls are cached; prefetching warms the cache so the loop body never blocks on network I/O |
| Agent normalization | Dedicated AgentController class | Inline normalization in engine loop | Agents can return inconsistent outputs (missing tickers, string quantities); a dedicated normalizer keeps the engine loop clean |
| Trade execution model | Synchronous, sequential per-ticker | Batch / atomic multi-ticker execution | Simulates real-world order-by-order execution; partial fills are handled naturally via cash checks |
Contents
Architecture
flowchart TD CLI["cli.py -- CLI Entry Point"] --> Engine["BacktestEngine -- Day Loop Orchestrator"] Engine --> Prefetch["_prefetch_data -- Warm API Cache"] Engine --> DateLoop["pd.date_range -- Business Day Iterator"] DateLoop --> PriceFetch["get_price_data -- Current Day Prices"] PriceFetch --> Controller["AgentController.run_agent -- Invoke and Normalize Agent"] Controller --> AgentFn["agent callable -- LangGraph Pipeline"] AgentFn --> Controller Controller --> Executor["TradeExecutor.execute_trade -- Position Mutations"] Executor --> Portfolio["Portfolio -- State Manager"] Portfolio --> Valuation["calculate_portfolio_value -- Daily NAV"] Valuation --> Metrics["PerformanceMetricsCalculator -- Sharpe, Drawdown"] Valuation --> Output["OutputBuilder -- Table Rows and Display"] Metrics --> Engine
Data Flow
sequenceDiagram
participant CLI as cli.py
participant Eng as BacktestEngine
participant AC as AgentController
participant Agent as LangGraph Agent
participant TE as TradeExecutor
participant PF as Portfolio
participant Val as Valuation
participant Perf as MetricsCalc
CLI->>Eng: run_backtest()
Eng->>Eng: _prefetch_data() for all tickers + SPY
loop Each business day
Eng->>Eng: Fetch current_prices via get_price_data
Eng->>AC: run_agent(agent, tickers, dates, portfolio)
AC->>Agent: agent(tickers, start, end, snapshot)
Agent-->>AC: AgentOutput (decisions + signals)
AC-->>Eng: Normalized AgentOutput
loop Each ticker
Eng->>TE: execute_trade(ticker, action, qty, price, portfolio)
TE->>PF: apply_long_buy / apply_long_sell / apply_short_open / apply_short_cover
PF-->>TE: executed quantity
end
Eng->>Val: calculate_portfolio_value(portfolio, prices)
Val-->>Eng: total_value
Eng->>Perf: compute_metrics(portfolio_values)
Perf-->>Eng: PerformanceMetrics
end
Key Classes & Functions
| Class / Function | File | Purpose |
|---|---|---|
BacktestEngine | engine.py | Main orchestrator: initializes components, runs the day-by-day loop, aggregates results |
BacktestEngine.__init__ | engine.py | Accepts agent callable, tickers, dates, capital; creates Portfolio, TradeExecutor, AgentController, MetricsCalculator, OutputBuilder, BenchmarkCalculator |
BacktestEngine.run_backtest | engine.py | Core loop: prefetches data, iterates business days, calls agent, executes trades, computes value and metrics |
BacktestEngine._prefetch_data | engine.py | Warms API cache for all tickers (prices, financials, insider trades, news) and SPY benchmark |
BacktestEngine.get_portfolio_values | engine.py | Returns the historical equity curve as list of PortfolioValuePoint |
AgentController | controller.py | Invokes the trading agent and normalizes its output into consistent AgentDecisions |
AgentController.run_agent | controller.py | Converts Portfolio to snapshot, calls agent, normalizes decisions (coerces action to Action enum, quantity to float, fills missing tickers with hold/0) |
TradeExecutor | trader.py | Stateless trade dispatcher — routes action to the correct Portfolio method |
TradeExecutor.execute_trade | trader.py | Dispatches buy/sell/short/cover to Portfolio; returns 0 for hold or invalid actions |
Action | types.py | Enum with values: buy, sell, short, cover, hold — used for input validation |
AgentOutput | types.py | TypedDict with decisions (per-ticker action+quantity) and analyst_signals |
PortfolioValuePoint | types.py | TypedDict for a single equity-curve data point (date, value, exposures) |
PerformanceMetrics | types.py | TypedDict for computed metrics: sharpe, sortino, max drawdown, exposures |
main | cli.py | CLI entry point: parses args, interactive prompts for analysts/model, constructs and runs BacktestEngine |
Data Structures
| Type | Name | Description |
|---|---|---|
| Enum | Action | 5-value string enum: buy, sell, short, cover, hold |
| Literal | ActionLiteral | Type alias for action string literals — backward-compatible with pre-enum code |
| TypedDict | AgentDecision | Per-ticker decision: {action: ActionLiteral, quantity: float} |
| Dict | AgentDecisions | Mapping of ticker to AgentDecision |
| TypedDict | AgentOutput | Complete agent response: {decisions, analyst_signals} |
| TypedDict | PortfolioValuePoint | Daily snapshot: Date, Portfolio Value, Long/Short/Gross/Net Exposure, L/S Ratio |
| TypedDict | PerformanceMetrics | Aggregated metrics: sharpe_ratio, sortino_ratio, max_drawdown, exposures |
| TypedDict | PortfolioSnapshot | Full portfolio state passed to agents: cash, margin, positions, realized gains |
Interactions
| Direction | Partner | Interface | Description |
|---|---|---|---|
| → | Portfolio | Portfolio.get_snapshot() | AgentController obtains a serializable snapshot to pass to the agent callable |
| → | Portfolio | Portfolio.apply_long_buy() etc. | TradeExecutor dispatches trade actions to mutate portfolio state |
| → | Valuation | calculate_portfolio_value() | Engine computes total NAV after each day's trades |
| → | Valuation | compute_exposures() | Engine records long/short/gross/net exposure for each day |
| → | MetricsCalculator | compute_metrics() | Engine updates performance metrics when 3+ data points exist |
| → | OutputBuilder | build_day_rows() | Engine generates display rows for each day's results |
| → | BenchmarkCalculator | get_return_pct() | Engine fetches SPY buy-and-hold return for comparison |
| → | API layer | get_prices(), get_price_data() | Engine and prefetch fetch historical price data via the tools API |
| ← | cli.py | BacktestEngine() constructor | CLI constructs engine with parsed args (tickers, dates, model, analysts) |
Configuration
| Knob / Parameter | Default | Description |
|---|---|---|
initial_capital | 100000 | Starting cash for the simulated portfolio |
margin_requirement | 0.0 | Margin ratio for short positions (0 = no margin required) |
start_date | 1 month ago | First date of simulation (YYYY-MM-DD) |
end_date | today | Last date of simulation (YYYY-MM-DD) |
tickers | (required) | Comma-separated list of stock symbols to trade |
selected_analysts | interactive | Which analyst agents to include in the pipeline |
model_name / model_provider | interactive | LLM model for agent inference (supports OpenAI, Anthropic, Ollama) |
freq="B" | Business days | Date range frequency; hardcoded to business days (no weekends/holidays) |
Directory Map
Source tree for engine
backtesting/
├── engine.py ← BacktestEngine: day-loop orchestrator
├── controller.py ← AgentController: agent invocation + output normalization
├── trader.py ← TradeExecutor: action dispatch to Portfolio
├── types.py ← Action enum, TypedDicts (AgentOutput, PerformanceMetrics, etc.)
└── cli.py ← CLI entry point: arg parsing, interactive prompts, engine launch
📑 Annotated Code Walkthrough
The most important code in this sub-module — the core simulation loop inside BacktestEngine.run_backtest():
BacktestEngine.run_backtest() — The Day Loop
def run_backtest(self) -> PerformanceMetrics:
self._prefetch_data() # [1]
dates = pd.date_range(self._start_date, self._end_date, freq="B") # [2]
if len(dates) > 0:
self._portfolio_values = [
{"Date": dates[0], "Portfolio Value": self._initial_capital} # [3]
]
for current_date in dates: # [4]
lookback_start = (current_date - relativedelta(months=1)).strftime("%Y-%m-%d")
current_date_str = current_date.strftime("%Y-%m-%d")
previous_date_str = (current_date - relativedelta(days=1)).strftime("%Y-%m-%d")
# Fetch current prices for all tickers
current_prices: Dict[str, float] = {}
missing_data = False
for ticker in self._tickers:
try:
price_data = get_price_data(ticker, previous_date_str, current_date_str)
if price_data.empty:
missing_data = True; break
current_prices[ticker] = float(price_data.iloc[-1]["close"]) # [5]
except Exception:
missing_data = True; break
if missing_data:
continue # [6]
agent_output = self._agent_controller.run_agent( # [7]
self._agent, tickers=self._tickers,
start_date=lookback_start, end_date=current_date_str,
portfolio=self._portfolio, ...)
for ticker in self._tickers: # [8]
d = agent_output["decisions"].get(ticker, {"action": "hold", "quantity": 0})
executed_qty = self._executor.execute_trade(
ticker, d["action"], d["quantity"], current_prices[ticker], self._portfolio)
total_value = calculate_portfolio_value(self._portfolio, current_prices) # [9]
if len(self._portfolio_values) > 3: # [10]
computed = self._perf.compute_metrics(self._portfolio_values)
if computed:
self._performance_metrics.update(computed)
return self._performance_metrics
Line-by-line:
- [1] Prefetch: Warms the API cache for all tickers and SPY. Without this, every day in the loop would trigger network requests, making the backtest orders of magnitude slower.
- [2] Business-day range:
freq="B"skips weekends and generates only trading days. This ensures the simulation never tries to trade on non-market days. - [3] Initial equity point: Seeds the equity curve with starting capital at day 0. Required so that return calculations have a baseline reference point.
- [4] Day loop: Each iteration represents one simulated trading day. The lookback window is 1 month — agents see only this window of historical data to avoid look-ahead bias.
- [5] Close price: Uses
iloc[-1]["close"]to get the most recent closing price. The query range is[previous_day, current_day]to handle weekends/holidays gracefully. - [6] Skip missing data: If any ticker lacks price data for this day, the entire day is skipped rather than trading with partial information. This prevents mismatched position valuations.
- [7] Agent invocation: The controller normalizes agent output — coercing actions to the
Actionenum, quantities to floats, and filling missing tickers with hold/0. - [8] Trade execution: Each ticker is executed sequentially. The executor dispatches to the correct Portfolio method (buy/sell/short/cover) and returns the actual executed quantity (may be less than requested if insufficient cash).
- [9] Valuation: After all trades, the portfolio's total value is computed: cash + long positions - short positions. This is appended to the equity curve.
- [10] Metrics gate: Performance metrics are only computed when 3+ data points exist. Computing Sharpe or Sortino with fewer points would produce meaningless results.
⚠️ Edge Cases & Boundaries
| Edge Case | What Happens | Where in Code | Risk Level |
|---|---|---|---|
| Missing price data for a ticker | Entire day is skipped (continue); no trades executed | engine.py:run_backtest L116-130 | 🟡 Medium — could skip many days for illiquid stocks |
| Agent returns unknown action string | AgentController coerces to Action.HOLD; TradeExecutor returns 0 | controller.py:run_agent L56-57 | 🟢 Low |
| Agent returns negative or None quantity | Coerced to 0.0; no trade executed | controller.py:run_agent L50-53, trader.py:execute_trade L18-19 | 🟢 Low |
| Insufficient cash for buy order | Portfolio buys max affordable quantity (partial fill) | portfolio.py:apply_long_buy L99-111 | 🟡 Medium — silent partial fills may surprise users |
| Empty date range (start == end) | Loop body never executes; returns default empty metrics | engine.py:run_backtest L99-105 | 🟢 Low |
| lookback_start equals current_date | Day is skipped to avoid zero-length lookback window | engine.py:run_backtest L111-112 | 🟢 Low |
Code Examples
Example: Launching a Backtest from CLI
# Construct and run the engine programmatically
from src.backtesting.engine import BacktestEngine
from src.main import run_hedge_fund
engine = BacktestEngine(
agent=run_hedge_fund,
tickers=["AAPL", "GOOGL", "MSFT"],
start_date="2025-01-01",
end_date="2025-06-01",
initial_capital=100000,
model_name="gpt-4o",
model_provider="OpenAI",
selected_analysts=["technical_analyst", "fundamentals_analyst"],
initial_margin_requirement=0.5,
)
metrics = engine.run_backtest()
print(f"Sharpe: {metrics.get('sharpe_ratio', 'N/A')}")
print(f"Max DD: {metrics.get('max_drawdown', 'N/A')}%")
# Access the full equity curve
values = engine.get_portfolio_values()
for point in values[-3:]:
print(f"{point['Date']}: ${point['Portfolio Value']:,.2f}")
Example: TradeExecutor Dispatch
from src.backtesting.trader import TradeExecutor
from src.backtesting.portfolio import Portfolio
portfolio = Portfolio(tickers=["AAPL"], initial_cash=50000, margin_requirement=0.5)
executor = TradeExecutor()
# Buy 100 shares at $150
bought = executor.execute_trade("AAPL", "buy", 100, 150.0, portfolio)
# bought = 100 (cash: 50000 - 15000 = 35000)
# Try to short 200 shares at $150 — margin limits apply
shorted = executor.execute_trade("AAPL", "short", 200, 150.0, portfolio)
# margin_required = 200 * 150 * 0.5 = 15000
# available_cash = 35000 - 0 = 35000 (enough for full order)
# Hold action returns 0 with no side effects
held = executor.execute_trade("AAPL", "hold", 50, 150.0, portfolio)
# held = 0
Example: AgentController Normalization
# The controller ensures every ticker has a valid decision,
# even if the agent omits some or returns malformed data
controller = AgentController()
raw_output = agent(tickers=["AAPL", "TSLA"], ...)
# raw_output might be: {"decisions": {"AAPL": {"action": "buy", "quantity": "50"}}}
# Note: TSLA is missing, quantity is a string
normalized = controller.run_agent(agent, tickers=["AAPL", "TSLA"], ...)
# normalized["decisions"]["AAPL"] = {"action": "buy", "quantity": 50.0}
# normalized["decisions"]["TSLA"] = {"action": "hold", "quantity": 0.0}