gpu-compute

Architecture Overview

The gpu-compute module implements a cycle-level model of AMD GCN-class GPU compute units, including wavefront scheduling, SIMD execution, and HSA-compatible kernel dispatch. The module models the full pipeline from AQL packet submission through GPUCommandProcessor, dispatch via GPUDispatcher, and execution within ComputeUnits containing multiple SIMD lanes.

flowchart TD
    HSA["HSA Runtime / AQL Packets"]
    CmdProc["GPUCommandProcessor"]
    Shader["Shader (GPU Top-Level)"]
    Disp["GPUDispatcher"]
    CU1["ComputeUnit 0"]
    CU2["ComputeUnit 1"]
    CUN["ComputeUnit N"]
    FS["FetchStage"]
    SB["ScoreboardCheckStage"]
    SS["ScheduleStage"]
    ES["ExecStage"]
    GMP["GlobalMemPipeline"]
    LMP["LocalMemPipeline"]
    SMP["ScalarMemPipeline"]
    VRF["VectorRegisterFile"]
    SRF["ScalarRegisterFile"]

    HSA --> CmdProc
    CmdProc --> Shader
    Shader --> Disp
    Disp --> CU1
    Disp --> CU2
    Disp --> CUN
    CU1 --> FS
    FS --> SB
    SB --> SS
    SS --> ES
    ES --> GMP
    ES --> LMP
    ES --> SMP
    SS --> VRF
    SS --> SRF

The Shader object owns all compute units and manages GPU-wide state like aperture registers and active CU tracking. The GPUCommandProcessor receives AQL dispatch packets from the HSA packet processor, reads kernel metadata via DMA, and hands tasks to the dispatcher for workgroup-level scheduling across compute units.

Pipeline Stages

Each ComputeUnit contains a five-stage pipeline that processes wavefronts every cycle:

flowchart LR
    F["FetchStage\n(I-Cache Access)"] --> SC["ScoreboardCheckStage\n(Hazard Detection)"]
    SC --> S["ScheduleStage\n(Resource Arbitration)"]
    S --> E["ExecStage\n(ALU / Mem Issue)"]
    E --> M["Memory Pipelines\n(Global / Local / Scalar)"]

Kernel Dispatch Flow

sequenceDiagram
    participant App as HSA Application
    participant CP as GPUCommandProcessor
    participant SH as Shader
    participant DI as GPUDispatcher
    participant CU as ComputeUnit
    participant WF as Wavefront

    App->>CP: Submit AQL Dispatch Packet
    CP->>CP: DMA read kernel metadata (MQD)
    CP->>SH: dispatchKernelObject()
    SH->>DI: dispatch(HSAQueueEntry)
    DI->>DI: scheduleDispatch()
    loop For each workgroup
        DI->>CU: Allocate WG resources
        CU->>WF: Create wavefronts (64 work-items each)
    end
    loop Per cycle
        WF->>CU: FetchStage → ScoreboardCheck → Schedule → Execute
        CU->>CU: Memory pipeline access
    end
    WF->>DI: notifyWgCompl()
    DI->>CP: sendCompletionSignal()
    CP->>App: Update HSA signal

The dispatch path begins when the HSA runtime enqueues an AQL packet. The GPUCommandProcessor reads kernel metadata via DMA, then the GPUDispatcher maps workgroups to compute units with available resources. Each workgroup is split into wavefronts of 64 work-items that execute in SIMD lockstep through the pipeline stages.

Key Classes

Deep Dive: Wavefront Lifecycle

Wavefront States

A wavefront transitions through several states during its lifetime, managed by the status field:

stateDiagram-v2
    [*] --> S_STOPPED: Created
    S_STOPPED --> S_RUNNING: Resources allocated
    S_RUNNING --> S_STALLED: Structural hazard
    S_STALLED --> S_RUNNING: Hazard cleared
    S_RUNNING --> S_WAITCNT: waitcnt instruction
    S_WAITCNT --> S_RUNNING: Counters reach zero
    S_RUNNING --> S_BARRIER_WAIT: Barrier instruction
    S_BARRIER_WAIT --> S_RUNNING: All WFs arrive
    S_RUNNING --> S_RETURNING: Last instruction
    S_RETURNING --> [*]: Resources freed

Instruction Buffer

Each wavefront maintains an instruction buffer (deque<GPUDynInstPtr>) that is populated by the FetchStage. The scoreboard checks the oldest instruction for data hazards before allowing it to proceed to scheduling. Key classification methods:

• isGmInstruction() — global memory operation → routed to GlobalMemPipeline
• isLmInstruction() — local memory (LDS) operation → routed to LocalMemPipeline
• isOldestInstWaitcnt() — wait counter instruction, stalls until pending operations complete

Deep Dive: Register Management

The GPU register system has two distinct file types, managed by a pluggable RegisterManagerPolicy:

The RegisterManagerPolicy interface provides virtual methods for allocation checking (canAllocateVgprs, canAllocateSgprs), mapping (mapVgpr, mapSgpr), and freeing. The concrete StaticRegisterManagerPolicy partitions the physical register files statically among wavefront slots.

Each register file supports operand readiness tracking — the ScheduleStage queries operandsReady() before issuing an instruction, and calls scheduleWriteOperands() to reserve write-back slots after execution.

Deep Dive: Memory Pipelines

Global Memory Pipeline

Handles off-chip memory requests through the GPU TLB and cache hierarchy. Maintains an ordered response buffer (gmOrderedRespBuffer) keyed by sequence number to ensure in-order retirement. Tracks inflight loads and stores separately, with configurable queue depths.

Local Memory Pipeline

Provides access to the Local Data Store (LDS), a per-workgroup scratchpad memory. The LdsChunk class represents the allocated memory region with templated read<T>/write<T> access. The pipeline tracks bank conflict cycles for performance analysis.

flowchart LR
    EX["ExecStage"] --> G["GlobalMemPipeline"]
    EX --> L["LocalMemPipeline"]
    EX --> S["ScalarMemPipeline"]
    G --> TLB["GPU TLB"]
    TLB --> Cache["L1 Data Cache"]
    Cache --> L2["L2 Cache / Memory"]
    L --> LDS["LDS (Shared Memory)"]
    S --> STLB["Scalar TLB"]
    STLB --> SC["Scalar Cache"]

Cross-Module Interactions

Directory Map

Configuration Parameters

Code Examples

Kernel Dispatch Sequence

// GPUCommandProcessor receives AQL packet and dispatches kernel
void GPUCommandProcessor::submitDispatchPkt(void *raw_pkt,
                                             uint32_t queue_id,
                                             Addr host_pkt_addr)
{
    auto *task = new HSAQueueEntry(aql_pkt, queue_id, host_pkt_addr);
    // DMA read kernel object to get code and metadata
    dispatchKernelObject(task);
}

// GPUDispatcher maps workgroups to compute units
void GPUDispatcher::dispatch(HSAQueueEntry *task)
{
    hsaQueueEntries[task->dispatchId()] = task;
    scheduleDispatch();
}

// Each ComputeUnit creates wavefronts for assigned workgroups
void ComputeUnit::startWavefront(Wavefront *wf, int wg_id,
                                  HSAQueueEntry *task)
{
    wf->kernId = task->dispatchId();
    wf->setStatus(Wavefront::S_RUNNING);
    wf->resizeRegFiles(task->numVgprs(), task->numSgprs());
}

Pipeline Stage Execution

// Each cycle: stages execute in reverse order (exec → sched → sb → fetch)
void ComputeUnit::exec()
{
    // Execute stage — issue to ALUs or memory pipelines
    execStage.exec();

    // Schedule stage — arbitrate resources, read operands
    scheduleStage.exec();

    // Scoreboard — check hazards, populate ready list
    scoreboardCheckStage.exec();

    // Fetch — select wavefronts, issue I-cache requests
    fetchStage.exec();

    // Memory pipelines
    globalMemoryPipe.exec();
    localMemoryPipe.exec();
}

Barrier Synchronization

// WFBarrier tracks workgroup barrier participation
class WFBarrier {
    int _numAtBarrier = 0;
    int _maxBarrierCnt = 0;
public:
    bool allAtBarrier() const { return _numAtBarrier == _maxBarrierCnt; }
    void incNumAtBarrier() { ++_numAtBarrier; }
    void setMaxBarrierCnt(int cnt) { _maxBarrierCnt = cnt; }
    void reset() { _numAtBarrier = 0; _maxBarrierCnt = 0; }
};