Memory Model Deep-Dive Focus Deep-Dive

Address Encoding Scheme

Every simulated pointer is a single size_t whose bits are split into two fields:

graph LR
  A[size_t address] --> B[buffer index
high m_numBitsBuffer bits]
  A --> C[byte offset
low m_numBitsAddress bits]
  B -->|shift right| D[extractBuffer]
  C -->|mask| E[extractOffset]

The constructor receives bufferBits (the number of bits reserved for the buffer index). The remaining bits hold the offset within each buffer:

// Memory.cpp – constructor
Memory::Memory(unsigned addrSpace, unsigned bufferBits, const Context* context)
    : m_context(context), m_addressSpace(addrSpace), m_totalAllocated(0),
      m_numBitsBuffer(bufferBits)
{
    m_numBitsAddress = (sizeof(size_t) * 8) - m_numBitsBuffer;
    m_maxNumBuffers  = ((size_t)1 << m_numBitsBuffer) - 1; // index 0 = NULL
    m_maxBufferSize  = ((size_t)1 << m_numBitsAddress);
    m_memory.push_back(NULL); // slot 0 reserved for NULL
}Copy

Encoding and decoding are trivial bit operations:

// Encode – allocateBuffer returns this
size_t address = ((size_t)b) << m_numBitsAddress;

// Decode
size_t Memory::extractBuffer(size_t address) const {
    return (address >> m_numBitsAddress);
}
size_t Memory::extractOffset(size_t address) const {
    return (address & (((size_t)-1) >> m_numBitsBuffer));
}Copy

OpenCL Address Spaces

Each Memory instance is bound to exactly one address space. The enum is defined in the common header:

The Context owns the global memory (AddrSpaceGlobal). Each WorkItem owns a private memory instance, and each WorkGroup owns a local memory instance.

Buffer Management

flowchart TD
  A[allocateBuffer called] --> B{size <= maxBufferSize?}
  B -- No --> Z[return 0 — failure]
  B -- Yes --> C[getNextBuffer]
  C --> D{m_freeBuffers empty?}
  D -- Yes --> E[index = m_memory.size — extend vector]
  D -- No --> F[index = m_freeBuffers.front — pop]
  E --> G[Create Buffer struct]
  F --> G
  G --> H[new unsigned char size]
  H --> I[Insert into m_memory at index]
  I --> J[memcpy initData or memset 0]
  J --> K[m_totalAllocated += size]
  K --> L[notifyMemoryAllocated]
  L --> M[return index shifted left m_numBitsAddress]

The Buffer struct

struct Buffer {
    size_t          size;   // allocated byte count
    cl_mem_flags    flags;  // CL_MEM_READ_ONLY, CL_MEM_USE_HOST_PTR, …
    unsigned char*  data;   // raw storage
};Copy

Key data members

Deallocation pushes the freed index onto m_freeBuffers and sets the vector slot to NULL. If the buffer was not created with CL_MEM_USE_HOST_PTR, delete[] is called on data.

Load & Store

Both operations follow the same pattern: notify → validate → memcpy.

bool Memory::load(unsigned char* dest, size_t address, size_t size) const {
    m_context->notifyMemoryLoad(this, address, size);  // plugins observe
    if (!isAddressValid(address, size)) return false;

    size_t offset = extractOffset(address);
    Buffer* buffer = m_memory[extractBuffer(address)];
    memcpy(dest, buffer->data + offset, size);
    return true;
}

bool Memory::store(const unsigned char* src, size_t address, size_t size) {
    m_context->notifyMemoryStore(this, address, size, src);
    if (!isAddressValid(address, size)) return false;

    size_t offset = extractOffset(address);
    Buffer* buffer = m_memory[extractBuffer(address)];
    memcpy(buffer->data + offset, src, size);
    return true;
}Copy

Address Validation

bool Memory::isAddressValid(size_t address, size_t size) const {
    size_t buffer = extractBuffer(address);
    size_t offset = extractOffset(address);
    if (buffer == 0 ||                        // NULL pointer
        buffer >= m_memory.size() ||          // out-of-range index
        !m_memory[buffer] ||                   // freed slot
        offset + size > m_memory[buffer]->size) // bounds overflow
        return false;
    return true;
}Copy

Atomic Operations

Atomic ops use a pool of 64 mutexes (for global memory only). The mutex index is selected by (offset >> 2) & 63, providing 4-byte-granularity striping:

#define NUM_ATOMIC_MUTEXES 64
mutex atomicMutex[NUM_ATOMIC_MUTEXES];
#define ATOMIC_MUTEX(offset) atomicMutex[((offset) >> 2) & (NUM_ATOMIC_MUTEXES - 1)]

template <typename T>
T Memory::atomic(AtomicOp op, size_t address, T value) {
    m_context->notifyMemoryAtomicLoad(this, op, address, sizeof(T));
    m_context->notifyMemoryAtomicStore(this, op, address, sizeof(T));
    if (!isAddressValid(address, sizeof(T))) return 0;

    size_t offset = extractOffset(address);
    T* ptr = (T*)(m_memory[extractBuffer(address)]->data + offset);

    if (m_addressSpace == AddrSpaceGlobal)
        ATOMIC_MUTEX(offset).lock();

    T old = *ptr;
    switch (op) {
        case AtomicAdd:  *ptr = old + value;  break;
        case AtomicAnd:  *ptr = old & value;  break;
        case AtomicOr:   *ptr = old | value;  break;
        case AtomicXor:  *ptr = old ^ value;  break;
        case AtomicXchg: *ptr = value;         break;
        case AtomicInc:  *ptr = old + 1;       break;
        case AtomicDec:  *ptr = old - 1;       break;
        case AtomicMax:  *ptr = old > value ? old : value; break;
        case AtomicMin:  *ptr = old < value ? old : value; break;
        case AtomicSub:  *ptr = old - value;  break;
        default: break;
    }

    if (m_addressSpace == AddrSpaceGlobal)
        ATOMIC_MUTEX(offset).unlock();
    return old;  // always returns the PREVIOUS value
}Copy

Compare-and-Swap

atomicCmpxchg only notifies the store if the comparison succeeds:

template <typename T>
T Memory::atomicCmpxchg(size_t address, T cmp, T value) {
    m_context->notifyMemoryAtomicLoad(this, AtomicCmpXchg, address, sizeof(T));
    if (!isAddressValid(address, sizeof(T))) return 0;

    T* ptr = (T*)(m_memory[extractBuffer(address)]->data
                  + extractOffset(address));
    if (m_addressSpace == AddrSpaceGlobal) ATOMIC_MUTEX(extractOffset(address)).lock();

    T old = *ptr;
    if (old == cmp) {
        *ptr = value;
        m_context->notifyMemoryAtomicStore(this, AtomicCmpXchg, address, sizeof(T));
    }

    if (m_addressSpace == AddrSpaceGlobal) ATOMIC_MUTEX(extractOffset(address)).unlock();
    return old;
}Copy

Supported Atomic Operations

Memory Object Lifecycle

stateDiagram-v2
  [*] --> Allocated : allocateBuffer
  Allocated --> InUse : load / store / atomic
  InUse --> InUse : load / store / atomic
  InUse --> Mapped : mapBuffer
  Mapped --> InUse : unmapBuffer
  InUse --> Freed : deallocateBuffer
  Freed --> Recycled : getNextBuffer reuses index
  Recycled --> Allocated : allocateBuffer