gpgpu-sim — GPU Timing Simulation Engine

GPU Timing Simulation Engine

The performance-simulation core of GPGPU-Sim. Models a complete NVIDIA-style GPU micro-architecture at the cycle level — shader-core pipelines, L1/L2 cache hierarchy, on-chip interconnect, and DDR DRAM controllers — to produce detailed performance and power statistics for architectural research.

Files

25.5k

Lines of Code

18+

Classes

C++

Language

cycle-level shader cores cache hierarchy DRAM modeling

What Is This?

🎭 Analogy: Imagine a theme-park simulation game. Each ride is a shader core that processes visitors (threads) in groups (warps). The park’s road network is the interconnect shuttling people between attractions. Gift shops along the way are caches — if the souvenir you want is in stock (cache hit), you grab it instantly; otherwise an order goes to the central warehouse (DRAM). The park manager (gpgpu_sim::cycle()) advances every ride, road, and shop by one tick of the clock each cycle, collecting visitor-throughput statistics to optimize the park layout.

In reality, src/gpgpu-sim/ is the cycle-accurate timing model for every major GPU hardware block. It accepts CUDA kernel launches, distributes Cooperative Thread Arrays (CTAs) to shader cores, and simulates the full memory hierarchy from L1 through DRAM, producing the performance counters that drive GPU architecture research.

Architecture

flowchart LR
  subgraph Shader["Shader Pipeline"]
    FETCH["Fetch\n(L1I Cache)"]
    DECODE["Decode\n(ibuffer)"]
    SCHED["Warp Scheduler\n(GTO/LRR)"]
    SCOREB["Scoreboard\n(RAW check)"]
    OPCOLL["Operand Collector\n(reg read)"]
    EXEC["Execute\n(SP/SFU/MEM)"]
    WB["Writeback"]
  end
  subgraph Memory["Memory Hierarchy"]
    LDST["Load/Store Unit"]
    L1D["L1 Data Cache"]
    L1T["L1 Texture Cache"]
    ICNT["Interconnect\n(xbar / BookSim)"]
    L2["L2 Cache\n(sub-partitions)"]
  end
  subgraph DRAM_Sub["DRAM Subsystem"]
    ROP["ROP Delay"]
    DRAM_CTRL["DRAM Controller\n(dram_t)"]
    FRFCFS["FR-FCFS Scheduler"]
    BANKS["Bank Array\n(row buffers)"]
  end

  FETCH --> DECODE --> SCHED
  SCHED --> SCOREB --> OPCOLL --> EXEC --> WB
  EXEC --> LDST
  LDST --> L1D
  LDST --> L1T
  L1D --> ICNT
  L1T --> ICNT
  ICNT --> L2
  L2 --> ROP
  ROP --> DRAM_CTRL
  DRAM_CTRL --> FRFCFS --> BANKS
  BANKS --> DRAM_CTRL
  DRAM_CTRL --> L2
  L2 --> ICNT
  ICNT --> LDST

Data Flow — Memory Request Lifecycle

sequenceDiagram
    participant SC as Shader Core
    participant LDST as ldst_unit
    participant L1 as L1D Cache
    participant ICNT as Interconnect
    participant L2 as L2 Cache
    participant ROP as ROP Delay
    participant DC as DRAM Controller
    participant BK as DRAM Banks

    SC->>LDST: Execute memory instruction
    LDST->>L1: cache_access(addr)
    alt L1 HIT
        L1-->>LDST: Data (1-cycle)
    else L1 MISS
        L1->>ICNT: mem_fetch injected (icnt_push)
        ICNT->>L2: Route to partition
        alt L2 HIT
            L2-->>ICNT: Reply (L2_TO_ICNT queue)
        else L2 MISS
            L2->>ROP: ROP delay queue
            ROP->>DC: push(mem_fetch)
            DC->>BK: ACTIVATE + READ/WRITE
            BK-->>DC: Data after tCAS+burst
            DC-->>L2: Fill (DRAM_TO_L2 queue)
            L2-->>ICNT: Reply with filled data
        end
        ICNT-->>LDST: Response (icnt_pop)
    end
    LDST-->>SC: Writeback to register file

Step-by-Step Walkthrough

🚶 Tracing a Memory Load Instruction

Follow a single LDG (global load) instruction from fetch to data delivery:

Fetch — The fetch stage round-robins across warps. When warp W has an empty instruction buffer slot and its PC maps to a valid L1I line, the instruction bytes are fetched into the ibuffer (2-entry FIFO).
Decode — The instruction is decoded into a warp_inst_t identifying it as a memory operation targeting the ldst_unit functional unit.
Scoreboard Check — The warp scheduler (e.g., GTO) calls Scoreboard::checkCollision(wid, inst). If the destination register has a pending write from an earlier instruction, the warp stalls.
Issue & Register Reserve — The scheduler issues the instruction into the ldst_unit pipeline register. Scoreboard::reserveRegisters() marks the destination register as pending.
Operand Collection — The opndcoll_rfu_t reads the base-address register from the banked register file via its crossbar arbiter. Once all source operands are ready, the instruction moves to execution.
Address Generation & Coalescing — The ldst_unit computes per-thread effective addresses and coalesces them into one or more mem_access_t objects (128-byte sectors). Each becomes a mem_fetch envelope.
L1D Probe — data_cache::access() probes the tag array. On a HIT, data returns in one cycle to the writeback stage. On a MISS, an MSHR entry is allocated and the mem_fetch is queued for the interconnect.
Interconnect Transit — icnt_push() injects the request. The on-chip network (BookSim or local crossbar) routes it to the target memory partition based on the address decoder’s sub-partition mapping.
L2 Probe — The memory_sub_partition receives the request from the ICNT-to-L2 queue. The L2 tag_array is probed. On a HIT, the reply is pushed to the L2-to-ICNT queue. On a MISS, the request passes through the ROP delay and enters the DRAM-to-L2 pipeline.
DRAM Scheduling — The dram_t controller receives the request. The FR-FCFS scheduler bins it by (bank, row). If the target row is already open (row-buffer hit), a column command issues immediately; otherwise an ACTIVATE is needed first.
DRAM Service — After the bank timing constraints (tRCD, CL, burst length) are satisfied, data is placed in the returnq. The controller pops it and pushes a fill response back to the L2 sub-partition.
L2 Fill & Reply — The L2 cache allocates a line (RESERVED→VALID), writes the data, and pushes the reply into the L2-to-ICNT queue. The MSHR entry is released.
Response Delivery — The interconnect routes the reply back to the originating shader cluster. icnt_pop() delivers it to the ldst_unit response FIFO.
Writeback — The ldst_unit processes the response, writes data to the register file, and calls Scoreboard::releaseRegisters(). The destination register is no longer pending — dependent warps can now issue.

Key Components

Class	File	Purpose
`gpgpu_sim`	gpu-sim.h/cc	Top-level simulator; owns clusters, memory partitions, and the main `cycle()` loop
`shader_core_ctx`	shader.h/cc	Single shader/SM core: fetch → decode → issue → execute → writeback pipeline
`simt_core_cluster`	shader.h/cc	Cluster of shader cores sharing an interconnect injection port
`shd_warp_t`	shader.h	Per-warp state: PC, active-thread mask, instruction buffer, barriers
`scheduler_unit`	shader.h	Abstract warp scheduler with GTO, LRR, RRR, Two-Level sub-classes
`ldst_unit`	shader.h/cc	Load/store functional unit managing L1D/L1C/L1T caches and memory requests
`Scoreboard`	scoreboard.h/cc	Per-warp register-dependency tracker preventing RAW hazards at issue
`opndcoll_rfu_t`	shader.h	Operand-collector / register-file-unit with crossbar arbiter
`baseline_cache`	gpu-cache.h/cc	Base configurable set-associative cache (tag array + MSHR + bandwidth)
`data_cache` / `l1_cache`	gpu-cache.h/cc	L1 data/instruction/texture/constant cache specializations
`l2_cache`	gpu-cache.h/cc	L2 cache slice within a memory sub-partition
`tag_array`	gpu-cache.h/cc	Set-associative tag store with LRU/FIFO replacement and sector support
`mshr_table`	gpu-cache.h/cc	Miss-Status Holding Registers — merges concurrent misses to the same block
`memory_partition_unit`	l2cache.h/cc	One per DRAM channel; houses sub-partitions (L2 slices) and a DRAM controller
`memory_sub_partition`	l2cache.h/cc	Single L2 slice with four FIFO queues (icnt↔L2, L2↔DRAM)
`dram_t`	dram.h/cc	DDR DRAM controller: banks, bank-groups, timing constraints, command scheduling
`frfcfs_scheduler`	dram_sched.h/cc	First-Ready First-Come-First-Served DRAM request scheduler with R/W modes
`mem_fetch`	mem_fetch.h/cc	Memory-request envelope carrying address, type, timestamps through the hierarchy
`linear_to_raw_address_translation`	addrdec.h/cc	Address decoder mapping linear addresses to (chip, bank, row, col, sub-partition)
`LocalInterconnect`	local_interconnect.h/cc	Crossbar interconnect with Round-Robin or iSLIP arbitration

Deep-Dive Topics

📦 Cache Hierarchy

L1D/L1I/L1T/L1C and L2 caches with configurable associativity, sector support, and MSHR merging.

Tag probe with LRU/FIFO replacement
Sector-mask validity checking
RESERVATION_FAIL deadlock prevention
Write-back & write-through policies

⚡ DRAM Scheduling

FR-FCFS scheduler with row-buffer hit prioritization and read/write mode switching via watermarks.

Per-bank request binning by row
ACTIVATE / PRECHARGE / READ / WRITE commands
DDR timing: tRCD, tCAS, tRP, tRAS, tCCDL
Separate write queue with high/low watermarks

🔧 Shader Core Pipeline

5-stage pipeline executed in reverse order: writeback → execute → issue → decode → fetch.

Multi-scheduler support (1–4 per core)
Scoreboard-based RAW hazard detection
Operand collector with banked register file
SP, SFU, MEM functional units

🎲 Warp Scheduling

Pluggable scheduling policies that determine which warp issues each cycle.

GTO: Greedy-Then-Oldest (favors MLP)
LRR: Loose Round-Robin
Two-Level: active/pending warp pools
SWL: Static Warp Limiting

📌 Memory Coalescing & Address Decoding

Per-thread addresses coalesced into 128-byte sector accesses; configurable address-to-partition mapping.

Coalescing in ldst_unit
LINEAR / XOR / IPOLY partition indexing
Bit-field mask configuration
Sweep test for alias validation

Directory Map

Source tree for src/gpgpu-sim/

src/gpgpu-sim/
├── gpu-sim.h / gpu-sim.cc          ← Top-level gpgpu_sim class & cycle() loop
├── shader.h / shader.cc            ← shader_core_ctx, simt_core_cluster, scheduler_unit, ldst_unit
├── gpu-cache.h / gpu-cache.cc      ← Cache hierarchy: baseline_cache, data_cache, tag_array, mshr_table
├── l2cache.h / l2cache.cc          ← memory_partition_unit, memory_sub_partition
├── dram.h / dram.cc                ← dram_t controller, bank state machine, timing
├── dram_sched.h / dram_sched.cc    ← frfcfs_scheduler, FIFO scheduler
├── mem_fetch.h / mem_fetch.cc      ← mem_fetch envelope, status FSM
├── addrdec.h / addrdec.cc          ← Address decoder (linear → chip/bank/row/col)
├── scoreboard.h / scoreboard.cc    ← Per-warp register dependency tracking
├── local_interconnect.h / .cc      ← xbar_router, LocalInterconnect
├── shader_trace.h                  ← Shader-core trace/visualization hooks
├── l2cache_trace.h                 ← L2 cache trace hooks
├── mem_latency_stat.h / .cc        ← Memory latency statistics collection
├── power_stat.h / power_stat.cc    ← Power/energy statistics (AccelWattch)
├── stat-tool.h / stat-tool.cc      ← General statistics utilities
├── gpu-misc.h / gpu-misc.cc        ← Miscellaneous GPU helper functions
├── hashing.h / hashing.cc          ← IPOLY & XOR hash functions for address mapping
├── mem_fetch_status.tup            ← mem_fetch status enum definitions (26 states)
├── traffic_breakdown.h / .cc       ← Memory traffic classification & breakdown
├── gpu-sim_power.cc                ← Power model integration (McPAT / AccelWattch)
├── icnt_wrapper.h / icnt_wrapper.cc← Interconnect abstraction (BookSim / local xbar)
├── visualizer.h                    ← Visualization interface
└── Makefile                        ← Module build rules

Configuration Knobs

Parameter	Default	Description
`gpgpu_n_clusters`	10	Number of SIMT core clusters
`gpgpu_n_cores_per_cluster`	3	Shader cores per cluster
`gpgpu_shader_core_pipeline`	`"1024:32"`	Threads-per-core : warp-size
`gpgpu_shader_registers`	8192	Architectural registers per core
`gpgpu_shmem_size`	16384	Shared memory bytes per core
`gpgpu_scheduler`	`"gto"`	Warp scheduler type (lrr / gto / rrr / two_level / warp_limiting)
`gpgpu_num_sched_per_core`	1	Scheduler units per core
`gpgpu_cache:dl2`	`"64:128:8,..."`	L2 config — sets:line_sz:assoc, replacement, MSHR
`gpgpu_dram_scheduler`	1	0 = FIFO, 1 = FR-FCFS
`gpgpu_dram_timing_opt`	(timing string)	DDR timing: nbk:tCCD:tRRD:tRCD:tRAS:tRP:tRC:CL:WL:tWR…
`gpgpu_dram_buswidth`	4	DRAM bus width in bytes
`gpgpu_dram_burst_length`	4	Burst length in cycles
`network_mode`	1	1 = BookSim (INTERSIM), 2 = Local crossbar
`gpu_deadlock_detect`	1	Enable deadlock detection (every 50K cycles)
`gpgpu_flush_l1_cache`	0	Flush L1 data cache after each kernel
`gpgpu_flush_l2_cache`	0	Flush L2 cache after each kernel

Code Examples

Top-Level Simulator Class

class gpgpu_sim : public gpgpu_t {
 public:
  gpgpu_sim(const gpgpu_sim_config &config, gpgpu_context *ctx);
  void launch(kernel_info_t *kinfo);
  bool can_start_kernel();
  unsigned finished_kernel();
  void set_kernel_done(kernel_info_t *kernel);
  void init();
  void cycle();          // advance every component by one clock tick
  bool active();
  bool cycle_insn_cta_max_hit();
  void deadlock_check();
  // ...
};

Scoreboard — Register Dependency Tracking

class Scoreboard {
 public:
  Scoreboard(unsigned sid, unsigned n_warps, class gpgpu_t *gpu);
  void reserveRegisters(const warp_inst_t *inst);
  void releaseRegisters(const warp_inst_t *inst);
  void releaseRegister(unsigned wid, unsigned regnum);
  bool checkCollision(unsigned wid, const inst_t *inst) const;
  bool pendingWrites(unsigned wid) const;
 private:
  std::vector<std::set<unsigned>> reg_table;   // warp → pending-write registers
  std::vector<std::set<unsigned>> longopregs;  // long-latency (mem) registers
};

DRAM Controller Interface

class dram_t {
 public:
  dram_t(unsigned int partition_id, const memory_config *config,
         class memory_stats_t *stats, class memory_partition_unit *mp,
         class gpgpu_sim *gpu);
  bool full(bool is_write) const;
  void push(class mem_fetch *data);
  void cycle();
  class mem_fetch *return_queue_pop();
 private:
  void scheduler_fifo();
  void scheduler_frfcfs();
  bool issue_col_command(int j);
  bool issue_row_command(int j);
  bank_t **bk;
  frfcfs_scheduler *m_frfcfs_scheduler;
  fifo_pipeline<dram_req_t> *rwq, *mrqq;
  fifo_pipeline<mem_fetch>  *returnq;
};

Clock Domains

The main gpgpu_sim::cycle() loop drives four independent clock domains. The fastest-period domain fires first each tick:

Domain	Default Freq	What Runs
CORE	500 MHz	Shader clusters, CTA issue, power model
ICNT	2 GHz	Interconnect transfer, shader ↔ memory response routing
L2	2 GHz	L2 cache cycles, icnt → mem pushes
DRAM	2 GHz	DRAM controller scheduling & DDR timing

Glossary

Term	Meaning
CTA	Cooperative Thread Array (thread block) — the unit of work distributed to a core
Warp	Group of 32 threads executing in SIMT lock-step
SIMT Stack	Stack tracking thread divergence/reconvergence at branch points
MSHR	Miss-Status Holding Register — tracks pending cache misses and merges requests
FR-FCFS	First-Ready First-Come-First-Served — DRAM scheduling that prioritises row-buffer hits
Row Buffer	Sense-amp row in a DRAM bank; an open row allows column commands without re-activation
mem_fetch	Envelope object tracking a single memory request through all hierarchy stages (26 states)
Operand Collector	Hardware unit that reads source operands from banked register files before execution
Sector Cache	Cache where a line is divided into sectors; individual sectors can be valid/invalid independently
ROP	Raster Operations Pipeline — fixed delay for memory requests before L2

Module Interactions

Peer Module	Direction	Mechanism
`abstract_hardware_model`	← kernels, threads	`kernel_info_t`, `warp_inst_t`, `mem_access_t`
`intersim2` / BookSim	↔ packets	Function-pointer table `icnt_push/icnt_pop`
`cuda-sim`	← functional exec	`func_exec_inst()` called from `issue_warp()`
Power model (AccelWattch)	→ stats	`mcpat_cycle()` called every CORE tick
`stream_manager`	← kernel dispatch	`launch()` API receiving `kernel_info_t`
SST / Balar co-simulation	↔ memory	`sst_gpgpu_sim`, `sst_memory_interface`