cache โ Classic Cache Hierarchy
High-performance, configurable cache hierarchy with coherence, MSHR tracking, pluggable replacement policies, hardware prefetchers, and compressed cache support.
Table of Contents
Overview
📚 What is this & why does it exist?
Imagine a multi-level library system. Your desk (L1 cache) has the few books you're currently reading โ instant access. The room's bookshelf (L2 cache) holds more books but takes a short walk. The building's main library (L3 / memory) has everything but requires a trip downstairs. The replacement policy decides which book to remove from your desk when it's full. The prefetcher is a helpful librarian who notices your reading pattern and pre-fetches the next book before you ask. The MSHR is the checkout slip tracking which books you've requested but haven't arrived yet.
The cache/ directory implements gem5's Classic memory system cache hierarchy. It provides:
- BaseCache โ abstract base managing tags, MSHRs, write queues, and port handling
- Cache โ coherent cache with snooping and flexible topologies
- NoncoherentCache โ simplified non-snooping cache
- MSHR / WriteQueue โ miss tracking and write coalescing
- Replacement Policies โ 12+ algorithms (LRU, FIFO, BIP, BRRIP, SHiP, TreePLRU, โฆ)
- Prefetchers โ 14+ engines (Stride, BOP, AMPM, SPP, DCPT, ISB, โฆ)
- Tags โ set-associative, fully-associative LRU, sector, compressed
- Compressors โ BDI, C-Pack, FPC, FPCD, FrequentValues
Class Hierarchy
classDiagram
class ClockedObject {
+clockPeriod()
+curCycle()
}
class BaseCache {
-MSHRQueue mshrQueue
-WriteQueue writeBuffer
-BaseTags* tags
-BasePrefetcher* prefetcher
+recvTimingReq()
+recvTimingResp()
+handleFill()
+satisfyRequest()
}
class Cache {
-doFastWrites: bool
-outstandingSnoop: set
+serviceMSHRTargets()
+handleSnoop()
+recvTimingSnoopReq()
}
class NoncoherentCache {
+handleTimingReqMiss()
+serviceMSHRTargets()
}
class MSHR {
-targets: TargetList
-deferredTargets: TargetList
+allocate()
+deallocate()
+getTarget()
}
class WriteQueueEntry {
+isFull(): bool
}
class BaseTags {
+accessBlock()
+findBlock()
+insertBlock()
}
class BaseSetAssoc
class FALRU
class SectorTags
class CompressedTags
class ReplacementPolicy_Base {
+getVictim()
+reset()
+touch()
}
class LRURP
class FIFORP
class BIPRP
class BRRIPRP
class SHiPRP
class TreePLRURP
class BasePrefetcher {
+notify()
+getPacket()
}
class Queued
class Stride
class BOP
class AMPM
ClockedObject <|-- BaseCache
BaseCache <|-- Cache
BaseCache <|-- NoncoherentCache
BaseCache *-- MSHR
BaseCache *-- WriteQueueEntry
BaseCache *-- BaseTags
BaseCache *-- BasePrefetcher
BaseTags <|-- BaseSetAssoc
BaseTags <|-- FALRU
BaseTags <|-- SectorTags
SectorTags <|-- CompressedTags
ReplacementPolicy_Base <|-- LRURP
ReplacementPolicy_Base <|-- FIFORP
ReplacementPolicy_Base <|-- BIPRP
BIPRP <|-- BRRIPRP
ReplacementPolicy_Base <|-- SHiPRP
ReplacementPolicy_Base <|-- TreePLRURP
BasePrefetcher <|-- Queued
Queued <|-- Stride
Queued <|-- BOP
Queued <|-- AMPM
Cache Request Processing Flow
flowchart TD
REQ["recvTimingReq()"]
LOOKUP["Tag Lookup"]
HIT{"Hit?"}
RESP["Send Response"]
MSHR_CHK{"MSHR exists?"}
COALESCE["Coalesce into MSHR"]
ALLOC["Allocate MSHR"]
SEND["sendTimingReq() downstream"]
FILL["handleFill()"]
VICTIM["Select victim (RP)"]
WB{"Dirty victim?"}
WBACK["Writeback victim"]
INSERT["Insert new block"]
DEALLOC["Deallocate MSHR"]
REQ --> LOOKUP
LOOKUP --> HIT
HIT -->|Yes| RESP
HIT -->|No| MSHR_CHK
MSHR_CHK -->|Yes| COALESCE
MSHR_CHK -->|No| ALLOC
ALLOC --> SEND
SEND -->|Response arrives| FILL
FILL --> VICTIM
VICTIM --> WB
WB -->|Yes| WBACK
WB -->|No| INSERT
WBACK --> INSERT
INSERT --> DEALLOC
DEALLOC --> RESP
Key Classes Reference
| Class | File | Description |
|---|---|---|
BaseCache | base.hh | Abstract base for all classic caches. Manages tag arrays, MSHR queues, write buffers, prefetcher integration, and both CPU-side and memory-side ports. |
Cache | cache.hh | Coherent cache with full snooping support, snoop filtering, and fast-write optimization. |
NoncoherentCache | noncoherent_cache.hh | Cache without coherence protocol support โ suitable for instruction caches or accelerator scratchpads. |
MSHR | mshr.hh | Tracks outstanding cache misses. Supports target coalescing so multiple requests to the same block share one downstream fetch. |
MSHRQueue | mshr_queue.hh | Manages the pool of MSHR entries, blocking the cache when exhausted. |
WriteQueueEntry | write_queue_entry.hh | Tracks pending writebacks and write-cleans to lower memory. |
CacheBlk | cache_blk.hh | Represents a single cache block with valid/dirty bits, tag, data, coherence state. |
BaseTags | tags/base.hh | Abstract tag-store interface. Subclasses implement different organizations. |
BaseSetAssoc | tags/base_set_assoc.hh | Standard set-associative tag store using configurable indexing policies. |
FALRU | tags/fa_lru.hh | Fully-associative cache with LRU stack โ useful for small caches and ideal-cache studies. |
SectorTags | tags/sector_tags.hh | Sector cache: groups of blocks share a single tag, reducing tag overhead. |
CompressedTags | tags/compressed_tags.hh | Supports variable-size compressed blocks within a sector-like structure. |
Base (RP) | replacement_policies/base.hh | Abstract replacement policy interface. All policies implement getVictim(), reset(), touch(). |
Base (PF) | prefetch/base.hh | Abstract prefetcher base. Notified on cache accesses; generates prefetch requests. |
Queued | prefetch/queued.hh | Prefetcher with a FIFO queue of pending prefetch requests, throttled by degree and distance. |
Replacement Policies
All replacement policies live in cache/replacement_policies/ and extend a common base class. The cache invokes getVictim() to select an eviction candidate and touch() / reset() on access and insertion.
| Policy | File | Algorithm |
|---|---|---|
LRURP | lru_rp.hh | Least Recently Used โ evicts the block with the oldest last-access timestamp. |
FIFORP | fifo_rp.hh | First-In First-Out โ evicts the block inserted earliest, ignoring reuse. |
LFURP | lfu_rp.hh | Least Frequently Used โ evicts the block with the fewest accesses. |
MRURP | mru_rp.hh | Most Recently Used โ evicts the most recently accessed block (thrash-resistant). |
RandomRP | random_rp.hh | Random โ evicts a randomly chosen block. |
BIPRP | bip_rp.hh | Bimodal Insertion โ inserts most blocks in LRU position; a small fraction in MRU (thrash protection). |
BRRIPRP | brrip_rp.hh | Bimodal Re-Reference Interval Prediction โ extends BIP with multi-bit RRPV counters for finer granularity. |
SHiPRP | ship_rp.hh | Signature-based Hit Predictor โ uses PC-based signatures to predict reuse and set insertion priority. |
TreePLRURP | tree_plru_rp.hh | Tree Pseudo-LRU โ binary tree of 1-bit pointers; hardware-efficient LRU approximation. |
SecondChanceRP | second_chance_rp.hh | Second Chance (Clock) โ FIFO with a use bit; gives accessed blocks a second chance. |
DuelingRP | dueling_rp.hh | Set Dueling โ dynamically selects between two constituent policies using tournament sampling. |
Prefetch Engines
Prefetchers in cache/prefetch/ observe cache access patterns and issue speculative fetches. The Queued base maintains a FIFO of generated addresses, throttled by configurable degree.
| Prefetcher | File | Strategy |
|---|---|---|
StridePrefetcher | stride.hh | Detects constant-stride access patterns per PC; Atomic and Timing Simple CPUs โ gem5 Wiki" href="../../cpu_doc/simple/index.html">simple and effective for array traversals. |
BOPPrefetcher | bop.hh | Best Offset Prefetching โ evaluates many candidate offsets and picks the best-performing one. |
AMPMPrefetcher | access_map_pattern_matching.hh | Access Map Pattern Matching โ tracks access bitmaps per page zone to detect complex patterns. |
SignaturePathPrefetcher | signature_path.hh | SPP โ builds a signature table from delta history; follows lookahead paths for deep prefetching. |
SignaturePathPrefetcherV2 | signature_path_v2.hh | SPP v2 with enhanced confidence and global accuracy feedback. |
DCPTPrefetcher | delta_correlating_prediction_tables.hh | Delta Correlating Prediction Tables โ correlates deltas between consecutive addresses. |
IndirectMemoryPrefetcher | indirect_memory.hh | Handles pointer-chasing patterns by tracking indirect memory accesses. |
IrregularStreamBuffer | irregular_stream_buffer.hh | ISB โ maps structural addresses to temporal streams for irregular workloads. |
SBOOEPrefetcher | sbooe.hh | Sandbox-Based Optimal Offset Estimation โ lightweight offset evaluation using sandbox. |
PIFPrefetcher | pif.hh | Proactive Instruction Fetch โ uses temporal address correlation for instruction streams. |
STeMSPrefetcher | spatio_temporal_memory_streaming.hh | Spatio-Temporal Memory Streaming โ combines spatial and temporal patterns. |
SlimAMPMPrefetcher | slim_ampm.hh | Lightweight AMPM variant with reduced storage. |
TaggedPrefetcher | tagged.hh | Simple next-line prefetcher triggered on every cache access. |
MultiPrefetcher | multi.hh | Wrapper that combines multiple prefetcher instances. |
MSHR & Write Buffer Handling
MSHRs are the backbone of non-blocking cache operation. Each MSHR tracks one outstanding cache line fetch and coalesces all requests targeting the same block.
MSHR Lifecycle
MSHRQueue::allocate() creates an MSHR if one is available. If all MSHRs are busy, the cache blocks.sendTimingReq() to the next cache level or memory controller.BaseCache::recvTimingResp() calls handleFill() to insert the block and service all coalesced targets.Write Buffer
The WriteQueue holds pending writebacks and write-cleans. When a dirty block is evicted, a WriteQueueEntry is allocated. The write buffer has its own flow control and can cause the cache to block independently of MSHRs.
// Simplified MSHR allocation in BaseCache::access()
MSHR *mshr = mshrQueue.findMatch(blk_addr);
if (mshr) {
// Coalesce: add this request as a target
mshr->allocateTarget(pkt, pkt->headerDelay, order++);
} else {
// New miss: allocate MSHR and send downstream
mshr = mshrQueue.allocate(blk_addr, blk_size, pkt, curTick(), order++);
memSidePort.sendTimingReq(mshr->getTarget()->pkt);
}
Tag Organization
Tag stores determine how cache blocks are indexed and looked up. gem5 provides several implementations:
| Tag Type | File | Organization |
|---|---|---|
BaseSetAssoc | tags/base_set_assoc.hh | Standard set-associative with configurable indexing (modular, skewed, etc.). |
FALRU | tags/fa_lru.hh | Fully-associative with LRU tracking. Useful for small filter caches or way-profiling studies. |
SectorTags | tags/sector_tags.hh | Groups of consecutive blocks share one tag. Reduces tag area for large caches. |
CompressedTags | tags/compressed_tags.hh | Extends sector tags: multiple compressed blocks can share a data entry if they fit. |
Component Deep Dives
Replacement Policies
12 algorithms from simple LRU/FIFO to advanced BRRIP and SHiP. Extends ReplacementPolicy::Base with getVictim(), reset(), touch(), and invalidate() hooks.
Prefetch Algorithms
14+ prefetchers from simple tagged/stride to sophisticated SPP, AMPM, and ISB. All hook into cache access notifications via BasePrefetcher::notify().
Compression
Data compressors for compressed caches: BDI (Base-Delta-Immediate), C-Pack, FPC (Frequent Pattern Compression), FPCD, and FrequentValues. Compress blocks to fit more data in same area.
MSHR & Write Queue
Non-blocking cache operation with configurable MSHR count, target coalescing, deferred targets for snoop handling, and separate write buffer for writebacks.
Indexing Policies
Set selection strategies: modular (traditional), skewed (XOR-based), and custom policies in tags/indexing_policies/.
Partitioning Policies
Cache partitioning for QoS: way partitioning and capacity allocation across requestors, managed by PartitionManager.
Configuration Examples
Classic cache hierarchy is configured through Python SimObjects. Key parameters:
// Python configuration example (gem5 style)
// L1 Data Cache
system.cpu.dcache = Cache(
size = '64kB',
assoc = 8,
tag_latency = 2,
data_latency = 2,
response_latency = 2,
mshrs = 4,
tgts_per_mshr = 20,
write_buffers = 8,
replacement_policy = LRURP(),
prefetcher = StridePrefetcher(degree=4)
)
// L2 Shared Cache
system.l2cache = Cache(
size = '2MB',
assoc = 16,
tag_latency = 10,
data_latency = 10,
response_latency = 10,
mshrs = 20,
tgts_per_mshr = 12,
write_buffers = 16,
clusivity = 'mostly_incl',
replacement_policy = TreePLRURP(),
prefetcher = BOPPrefetcher()
)
| Parameter | Type | Description | Typical L1 | Typical L2 |
|---|---|---|---|---|
size | MemorySize | Total cache capacity | 32-64 kB | 256 kB - 4 MB |
assoc | int | Ways per set | 4-8 | 8-16 |
tag_latency | Cycles | Tag array lookup latency | 1-2 | 8-20 |
data_latency | Cycles | Data array read latency | 1-2 | 8-20 |
mshrs | int | Number of MSHRs (outstanding misses) | 4-8 | 16-32 |
write_buffers | int | Write buffer entries | 8 | 16-32 |
clusivity | enum | Inclusion policy | โ | mostly_incl |
replacement_policy | SimObject | Replacement policy instance | LRURP | TreePLRURP |
prefetcher | SimObject | Prefetcher instance | Stride | BOP/SPP |