Memory Management

Overview

Fixed-Window Rate Limiter

The FixedWindowRateLimiter divides time into fixed windows (e.g., 60 seconds). Each window allows a set number of requests. The consume() method returns whether the request is allowed, how many tokens remain, and — if denied — how long the caller should wait before retrying.

interface RateLimitResult {
  allowed: boolean;
  retryAfterMs: number;
  remaining: number;
}

class FixedWindowRateLimiter {
  private windowStartMs: number = 0;
  private consumed: number = 0;

  constructor(
    private readonly windowMs: number,
    private readonly maxPerWindow: number
  ) {}

  consume(): RateLimitResult {
    const now = Date.now();

    // Reset window if expired
    if (now - this.windowStartMs >= this.windowMs) {
      this.windowStartMs = now;
      this.consumed = 0;
    }

    if (this.consumed < this.maxPerWindow) {
      this.consumed++;
      return {
        allowed: true,
        retryAfterMs: 0,
        remaining: this.maxPerWindow - this.consumed,
      };
    }

    const windowEnd = this.windowStartMs + this.windowMs;
    return {
      allowed: false,
      retryAfterMs: Math.max(0, windowEnd - now),
      remaining: 0,
    };
  }

  reset(): void {
    this.windowStartMs = 0;
    this.consumed = 0;
  }
}

const apiLimiter = new FixedWindowRateLimiter(60_000, 100);

async function makeApiCall(payload: unknown): Promise<Response> {
  const result = apiLimiter.consume();
  if (!result.allowed) {
    await sleepWithAbort(result.retryAfterMs);
    return makeApiCall(payload); // retry after wait
  }
  return fetch("/api/endpoint", {
    method: "POST",
    body: JSON.stringify(payload),
  });
}

Bounded Collections

Every in-memory collection in the caching layer has an explicit size limit. These bounds prevent any single subsystem from consuming unbounded memory.

Rate Limit Retry Runners

When external APIs rate-limit requests, the retry runner handles backoff automatically. Two factory functions create pre-configured runners: createRateLimitRetryRunner for general APIs and createTelegramRetryRunner for Telegram-specific limits.

interface RetryRunnerParams {
  maxAttempts: number;
  backoff: BackoffPolicy;
  isTransient: (error: unknown) => boolean;
  onRetry?: (attempt: number, error: unknown) => void;
}

const TRANSIENT_PATTERN = /429|timeout|connect|reset|closed|unavailable|temporarily/i;

function createTelegramRetryRunner(): RetryRunner {
  return createRateLimitRetryRunner({
    maxAttempts: 3,
    backoff: {
      initialMs: 400,
      maxMs: 30_000,
      factor: 2,
      jitter: 0.1,
    },
    isTransient: (err) => {
      const msg = err instanceof Error ? err.message : String(err);
      return TRANSIENT_PATTERN.test(msg);
    },
    onRetry: (attempt, err) => {
      console.warn(`Telegram retry #${attempt}:`, err);
    },
  });
}

function createRateLimitRetryRunner(
  params: RetryRunnerParams
): RetryRunner {
  return {
    async run<T>(fn: () => Promise<T>): Promise<T> {
      let lastError: unknown;
      for (let attempt = 1; attempt <= params.maxAttempts; attempt++) {
        try {
          return await fn();
        } catch (err) {
          lastError = err;
          if (!params.isTransient(err) || attempt === params.maxAttempts) {
            throw err;
          }
          const delayMs = computeBackoff(params.backoff, attempt);
          params.onRetry?.(attempt, err);
          await sleepWithAbort(delayMs);
        }
      }
      throw lastError;
    },
  };
}

Backoff Policy

The BackoffPolicy type defines exponential backoff parameters. The computeBackoff function calculates the delay for a given attempt using the formula: base × factor^(attempt-1) + random jitter. The result is clamped to maxMs.

interface BackoffPolicy {
  initialMs: number;
  maxMs: number;
  factor: number;
  jitter: number; // 0..1 — proportion of base delay added as random noise
}

function computeBackoff(policy: BackoffPolicy, attempt: number): number {
  const base = policy.initialMs * Math.pow(policy.factor, attempt - 1);
  const jitterAmount = base * policy.jitter * Math.random();
  return Math.min(base + jitterAmount, policy.maxMs);
}

function sleepWithAbort(
  ms: number,
  signal?: AbortSignal
): Promise<void> {
  return new Promise((resolve) => {
    if (signal?.aborted) {
      resolve();
      return;
    }
    const timer = setTimeout(resolve, ms);
    signal?.addEventListener("abort", () => {
      clearTimeout(timer);
      resolve();
    }, { once: true });
  });
}

Memory Safety & Fatal Errors

The unhandled-rejections.ts module classifies unhandled promise rejections into three categories: fatal (immediate exit), transient (log + retry), and unknown (log + continue). ERR_OUT_OF_MEMORY is always fatal.

type ErrorClass = "fatal" | "transient" | "unknown";

function classifyError(err: unknown): ErrorClass {
  const msg = err instanceof Error ? err.message : String(err);

  // Fatal — no recovery possible
  if (/ERR_OUT_OF_MEMORY/i.test(msg)) return "fatal";
  if (/heap out of memory/i.test(msg)) return "fatal";

  // Transient — safe to retry
  if (/SQLITE_BUSY/i.test(msg)) return "transient";
  if (/SQLITE_LOCKED/i.test(msg)) return "transient";
  if (/ECONNRESET/i.test(msg)) return "transient";
  if (/ETIMEDOUT/i.test(msg)) return "transient";

  return "unknown";
}

process.on("unhandledRejection", (reason) => {
  const cls = classifyError(reason);
  console.error(`Unhandled rejection [${cls}]:`, reason);

  if (cls === "fatal") {
    process.exit(137);
  }
  // transient and unknown: logged, not crashed
});

Session Disk Budget

Each session has a disk budget enforced by periodic sweeps. The sweep scans the session directory, sums file sizes, and prunes the oldest entries until total usage falls below the configured quota.

interface SessionDiskBudgetSweepResult {
  prunedEntries: number;
  freedBytes: number;
  scanDurationMs: number;
  budgetExceeded: boolean;
}

async function sweepSessionDisk(
  sessionDir: string,
  budgetBytes: number
): Promise<SessionDiskBudgetSweepResult> {
  const start = Date.now();
  const entries = await readdir(sessionDir, { withFileTypes: true });

  // Sort by mtime ascending (oldest first)
  const files = await Promise.all(
    entries
      .filter(e => e.isFile())
      .map(async (e) => {
        const p = join(sessionDir, e.name);
        const stat = await fsStat(p);
        return { path: p, size: stat.size, mtimeMs: stat.mtimeMs };
      })
  );
  files.sort((a, b) => a.mtimeMs - b.mtimeMs);

  let totalBytes = files.reduce((s, f) => s + f.size, 0);
  let prunedEntries = 0;
  let freedBytes = 0;

  while (totalBytes > budgetBytes && files.length > 0) {
    const oldest = files.shift()!;
    await unlink(oldest.path);
    totalBytes -= oldest.size;
    freedBytes += oldest.size;
    prunedEntries++;
  }

  return {
    prunedEntries,
    freedBytes,
    scanDurationMs: Date.now() - start,
    budgetExceeded: totalBytes > budgetBytes,
  };
}

Architecture Diagrams

Rate Limiter Decision Flow

flowchart TD
    A[Incoming Request] --> B{Window expired?}
    B -- Yes --> C[Reset window + counter]
    C --> D{Consumed < max?}
    B -- No --> D
    D -- Yes --> E[Increment counter]
    E --> F[Return allowed=true, remaining=N]
    D -- No --> G[Calculate retryAfterMs]
    G --> H[Return allowed=false]
    H --> I{Caller retries?}
    I -- Yes --> J[sleepWithAbort retryAfterMs]
    J --> A
    I -- No --> K[Request dropped]
    

Memory Pressure Response States

stateDiagram-v2
    [*] --> Normal : process start
    Normal --> Warning : heap usage > 70%
    Warning --> Normal : GC reclaims memory
    Warning --> Critical : heap usage > 85%
    Critical --> Warning : aggressive GC + eviction
    Critical --> OOM : allocation fails
    OOM --> Fatal : ERR_OUT_OF_MEMORY
    Fatal --> [*] : process.exit(137)

    Normal --> Normal : bounded collections enforce caps
    Warning --> Warning : rate limiter slows intake
    Critical --> Critical : disk budget sweep runs