Plugin Dispatch System Deep-Dive Focus Deep-Dive

Architecture Overview

The plugin system follows the observer pattern. The Context object owns a list of Plugin* instances and broadcasts execution events to every registered plugin through notify* methods. Plugins override only the callbacks they care about.

flowchart TD
  CTX[Context] -->|owns| PL[PluginList — std::list of pairs]
  PL --> P1[Logger — core]
  PL --> P2[MemCheck — core]
  PL --> P3[RaceDetector — optional]
  PL --> P4[InteractiveDebugger — optional]
  PL --> P5[DynamicPlugin — from env var]

  MEM[Memory::store] -->|notifyMemoryStore| CTX
  WI[WorkItem::execute] -->|notifyInstructionExecuted| CTX
  CTX -->|iterates list| P1
  CTX -->|iterates list| P2
  CTX -->|iterates list| P3

Core Data Structures

// Context.h
typedef std::pair<Plugin*, bool>  PluginEntry;   // (plugin, owned?)
typedef std::list<PluginEntry>    PluginList;

class Context {
private:
    PluginList              m_plugins;         // all registered plugins
    std::list<void*>        m_pluginLibraries; // dlopen handles for cleanup
    Memory*                 m_globalMemory;
    llvm::LLVMContext*      m_llvmContext;
    mutable const KernelInvocation* m_kernelInvocation;
};Copy

The bool in each PluginEntry indicates ownership. Core plugins are created by Context itself (true) and deleted during teardown. Externally registered plugins (false) are only unlinked, not destroyed.

Plugin Loading

Core plugins (always loaded)

The Context constructor calls loadPlugins(), which unconditionally instantiates the mandatory plugins and conditionally loads optional ones based on environment variables:

Dynamic plugins via OCLGRIND_PLUGINS

// Context.cpp — loadPlugins(), dynamic section
const char* dynamicPlugins = getenv("OCLGRIND_PLUGINS");
if (dynamicPlugins) {
    std::istringstream ss(dynamicPlugins);
    std::string libpath;
    while (std::getline(ss, libpath, ':')) {    // colon-delimited
        // Unix: dlopen + dlsym
        void* library = dlopen(libpath.c_str(), RTLD_NOW);
        void* init    = dlsym(library, "initializePlugins");
        // Windows: LoadLibraryA + GetProcAddress (analogous)

        // Call the plugin's initializer — it registers via Context*
        ((void (*)(Context*))init)(this);
        m_pluginLibraries.push_back(library);
    }
}Copy

Registration & unregistration

void Context::registerPlugin(Plugin* plugin) {
    m_plugins.push_back({plugin, false});  // external ownership
}

void Context::unregisterPlugin(Plugin* plugin) {
    m_plugins.remove_if([&](const PluginEntry& e){ return e.first == plugin; });
}Copy

The NOTIFY Dispatch Macro

All notify* methods share the same broadcast pattern, factored into a preprocessor macro:

#define NOTIFY(function, ...)                       \
{                                                   \
    PluginList::const_iterator itr;                 \
    for (itr = m_plugins.begin();                   \
         itr != m_plugins.end(); itr++)             \
    {                                               \
        itr->first->function(__VA_ARGS__);          \
    }                                               \
}Copy

sequenceDiagram
  participant Mem as Memory::store
  participant Ctx as Context
  participant Log as Logger
  participant MC as MemCheck
  participant RD as RaceDetector

  Mem->>Ctx: notifyMemoryStore(mem, addr, sz, data)
  Note over Ctx: check m_kernelInvocation

  alt has current WorkItem
    Ctx->>Log: memoryStore(mem, workItem, addr, sz, data)
    Ctx->>MC: memoryStore(mem, workItem, addr, sz, data)
    Ctx->>RD: memoryStore(mem, workItem, addr, sz, data)
  else has current WorkGroup only
    Ctx->>Log: memoryStore(mem, workGroup, addr, sz, data)
    Ctx->>MC: memoryStore(mem, workGroup, addr, sz, data)
  else host context
    Ctx->>Log: hostMemoryStore(mem, addr, sz, data)
    Ctx->>MC: hostMemoryStore(mem, addr, sz, data)
  end

Context-aware dispatch

Several notify methods inspect m_kernelInvocation to determine which overload to call on plugins. Memory callbacks have three tiers:

1. WorkItem-level — when a specific work-item is active (e.g., during instruction execution)
2. WorkGroup-level — when only a work-group context exists (e.g., during async copies)
3. Host-level — when no kernel is running (e.g., host clEnqueueReadBuffer)

void Context::notifyMemoryStore(const Memory* memory, size_t address,
                                size_t size, const uint8_t* storeData) const {
    if (m_kernelInvocation) {
        if (m_kernelInvocation->getCurrentWorkItem()) {
            NOTIFY(memoryStore, memory,
                   m_kernelInvocation->getCurrentWorkItem(),
                   address, size, storeData);
        } else if (m_kernelInvocation->getCurrentWorkGroup()) {
            NOTIFY(memoryStore, memory,
                   m_kernelInvocation->getCurrentWorkGroup(),
                   address, size, storeData);
        }
    } else {
        NOTIFY(hostMemoryStore, memory, address, size, storeData);
    }
}Copy

Plugin Base Class

All plugins inherit from Plugin (defined in Plugin.h) and override the virtual callbacks they need. Every method has an empty default body, so plugins are opt-in:

class Plugin {
public:
    Plugin(const Context* context) : m_context(context) {}
    virtual ~Plugin() {}

    // Override to return true if thread-safe
    virtual bool isThreadSafe() const;

    // Kernel lifecycle
    virtual void kernelBegin(const KernelInvocation*) {}
    virtual void kernelEnd(const KernelInvocation*) {}

    // Work-group lifecycle
    virtual void workGroupBegin(const WorkGroup*) {}
    virtual void workGroupComplete(const WorkGroup*) {}
    virtual void workGroupBarrier(const WorkGroup*, uint32_t flags) {}

    // Work-item lifecycle
    virtual void workItemBegin(const WorkItem*) {}
    virtual void workItemComplete(const WorkItem*) {}

    // Instruction observation
    virtual void instructionExecuted(const WorkItem*,
                  const llvm::Instruction*, const TypedValue&) {}

    // Memory callbacks (multiple overloads)
    virtual void memoryLoad(const Memory*, const WorkItem*,
                            size_t addr, size_t sz) {}
    virtual void memoryStore(const Memory*, const WorkItem*,
                             size_t addr, size_t sz, const uint8_t*) {}
    virtual void memoryAtomicLoad(const Memory*, const WorkItem*,
                                  AtomicOp, size_t, size_t) {}
    virtual void memoryAtomicStore(const Memory*, const WorkItem*,
                                   AtomicOp, size_t, size_t) {}
    virtual void hostMemoryLoad(const Memory*, size_t, size_t) {}
    virtual void hostMemoryStore(const Memory*, size_t, size_t, const uint8_t*) {}
    // ... memoryAllocated, memoryDeallocated, memoryMap, memoryUnmap

    virtual void log(MessageType type, const char* message) {}

protected:
    const Context* m_context;
};Copy

Complete Callback Catalogue

Thread Safety Considerations

Work-groups may execute in parallel on separate host threads. The Context queries every registered plugin before enabling parallelism:

bool Context::isThreadSafe() const {
    for (const PluginEntry& p : m_plugins) {
        if (!p.first->isThreadSafe())
            return false;   // one unsafe plugin forces serial execution
    }
    return true;
}Copy

Additional synchronization points:

• LLVM instruction printing — a static std::mutex in Context.cpp protects LLVM's non-thread-safe Instruction::print().
• Atomic memory operations — the Memory class uses its own 64-mutex array for global address space atomics (see Memory Model).
• Plugin list is immutable during execution — plugins are registered only during initialization; no locking is needed for the dispatch loop.

Writing a Custom Plugin

A minimal plugin as a dynamic shared library:

// MyPlugin.cpp
#include "core/Plugin.h"
#include "core/Context.h"

class MyPlugin : public oclgrind::Plugin {
public:
    MyPlugin(const oclgrind::Context* ctx) : Plugin(ctx) {}

    void instructionExecuted(const oclgrind::WorkItem* wi,
                             const llvm::Instruction* inst,
                             const oclgrind::TypedValue& res) override {
        m_count++;
    }
    void kernelEnd(const oclgrind::KernelInvocation*) override {
        std::cout << "Instructions executed: " << m_count << "\n";
    }
    bool isThreadSafe() const override { return false; }
private:
    size_t m_count = 0;
};

static MyPlugin* instance = nullptr;

extern "C" void initializePlugins(oclgrind::Context* ctx) {
    instance = new MyPlugin(ctx);
    ctx->registerPlugin(instance);
}
extern "C" void releasePlugins(oclgrind::Context* ctx) {
    ctx->unregisterPlugin(instance);
    delete instance;
}Copy

Load with: OCLGRIND_PLUGINS=/path/to/libMyPlugin.so oclgrind ./my_app

Plugin Lifecycle

stateDiagram-v2
  [*] --> Constructed : Context constructor
  Constructed --> CoreLoaded : loadPlugins — core plugins
  CoreLoaded --> DynamicLoaded : loadPlugins — OCLGRIND_PLUGINS dlopen
  DynamicLoaded --> Ready : initializePlugins callback

  state Ready {
    [*] --> Idle
    Idle --> Dispatching : notify* called
    Dispatching --> Idle : iteration complete
  }

  Ready --> Teardown : Context destructor
  Teardown --> Unloaded : delete owned plugins
  Unloaded --> LibsClosed : dlclose libraries
  LibsClosed --> [*]