plugins — Plugin System & Memory Backends

🎯 Why This Module Exists

Without the plugins module, every memory backend would need to be hard-coded into the agent’s core initialization path. Adding a new memory provider—say, a knowledge-graph backend or a cloud-hosted vector store—would require modifying the agent entry point, importing heavyweight SDKs unconditionally, and redeploying the entire system. The plugins module decouples what backends exist from which backend is active. It provides a filesystem-based discovery mechanism that scans for plugin.yaml manifests at startup, presents available options to the CLI without importing anything, and only loads the one backend selected in config.yaml. This lazy-loading architecture means the agent starts quickly even when 8 backends are installed, and users can swap memory strategies—from Holographic’s local embeddings to Honcho’s dialectic modeling to Mem0’s external service—with a single config change and zero code modifications. The plugin system also isolates dependency failures: if Honcho’s SDK is missing, that provider shows as “unavailable” but every other backend continues to work. This is essential for a tool that runs across vastly different environments, from GPU-equipped research servers to lightweight CI containers.

🚶 Step-by-Step Narrated Walkthrough

Follow a single memory save-and-recall operation as it flows through the entire plugins pipeline:

1. Agent Startup — AIAgent.__init__() reads config.yaml and finds memory.provider: holographic. It calls load_memory_provider("holographic").
2. Discovery Scan — discover_memory_providers() walks plugins/memory/, finding 8 subdirectories. For each, it reads plugin.yaml to extract the name, description, and availability flag—without importing any Python code.
3. Availability Check — The holographic backend’s plugin.yaml declares no external dependencies, so it passes the availability check. Backends like honcho or mem0 would check for API keys in environment variables.
4. Dynamic Import — importlib.util.spec_from_file_location() creates a module spec for plugins/memory/holographic/__init__.py. The loader registers parent packages in sys.modules first, enabling relative imports within the plugin.
5. Submodule Registration — Before executing the main module, the loader pre-registers all .py files in the holographic directory (store.py, retrieval.py, holographic.py) so that from .store import MemoryStore works correctly.
6. Provider Extraction — The loader first tries mod.register(collector)—if the module uses the register pattern, the _ProviderCollector captures the instance. If that fails, it falls back to scanning the module for a MemoryProvider subclass and instantiating it directly.
7. Instance Ready — The HolographicMemory instance is returned to the agent. It holds a local vector store, an embedding model reference, and retrieval configuration.
8. Memory Save — During a conversation, the agent calls provider.store("user prefers dark mode", embedding). The holographic backend computes an embedding vector, adds it to the local FAISS-like index, and persists the mapping to disk.
9. Memory Recall — Later, the agent calls provider.recall("what theme does the user like?"). The holographic backend embeds the query, performs cosine similarity search against the stored vectors, and returns the top-k matches ranked by relevance.
10. CLI Discovery — Meanwhile, discover_plugin_cli_commands() checks only the active provider’s cli.py. If holographic has CLI commands (e.g., hermes memory list), they’re registered into the main argparse tree. Unused providers’ CLI modules are never imported.

Holographic Vector Similarity Retrieval

The Problem: When the agent needs to recall past interactions, it must find the most semantically relevant memories from potentially thousands of stored entries. Simple keyword matching fails because natural language queries rarely use the same words as stored memories (“what theme does the user like?” vs. “user prefers dark mode”).

The Approach: Holographic converts both stored memories and recall queries into dense embedding vectors using a local embedding model. At recall time, it computes the cosine similarity between the query vector and every stored vector, then returns the top-k results above a configurable similarity threshold. The local vector index is organized for batch retrieval, allowing the agent to fetch related memories in a single pass.

Why This Algorithm: Cosine similarity on dense embeddings captures semantic meaning that keyword search misses. Alternatives like BM25 (term frequency) would fail on paraphrased content. Full-text search with stemming would miss conceptual similarity. The trade-off is memory: each embedding vector requires storage proportional to the model’s hidden dimension (typically 384–768 floats). For a local-first backend, this is acceptable—even 100K memories at 768 dimensions is only ~300 MB.

graph TD
    A["Query: 'what theme?'"] --> B["Embed query
→ vector q"]
    B --> C{"Index size
> threshold?"}
    C -->|"Small (< 10K)"| D["Brute-force cosine
similarity scan"]
    C -->|"Large (10K+)"| E["Approximate NN
with HNSW index"]
    D --> F["Rank by similarity"]
    E --> F
    F --> G{"Score >
min_threshold?"}
    G -->|Yes| H["Return top-k
memories"]
    G -->|No| I["Return empty
(no relevant memories)"]

Hot Paths

The most performance-sensitive code paths in this module:

• Plugin Discovery: Filesystem walk + YAML parse — O(n) in number of backend directories. Typically < 10ms for 8 backends. This runs once at startup and is intentionally cheap.
• Dynamic Import: importlib.util.spec_from_file_location() + exec_module() — 50–200ms depending on backend complexity and SDK import time. Only runs for the active provider.
• Holographic Recall: Embedding computation + vector similarity search — O(d×n) for brute-force where d = embedding dimension, n = stored memories. Dominant cost for large memory stores.
• CLI Command Discovery: Imports only cli.py (not the full backend SDK) — keeps CLI startup under 100ms even for backends with heavy dependencies.

Resource Usage

• Memory: Only the active backend’s modules are loaded into memory. Holographic additionally allocates a vector index proportional to stored memories (~300 bytes per memory entry at 768 dimensions). Cloud backends (supermemory, mem0) have minimal local memory footprint.
• I/O: Discovery reads plugin.yaml files from disk (one per backend, ~100 bytes each). Holographic reads/writes its vector index file on store/recall. Cloud backends make HTTP requests per operation. config.yaml is read once at startup to determine the active provider.