cache โ€” Classic Cache Hierarchy

High-performance, configurable cache hierarchy with coherence, MSHR tracking, pluggable replacement policies, hardware prefetchers, and compressed cache support.

26+
Source Files
4
Sub-directories
12
Replacement Policies
14+
Prefetchers
8+
Compressors
Table of Contents

    Overview

    📚 What is this & why does it exist?

    Analogy โ€” Library Shelf

    Imagine a multi-level library system. Your desk (L1 cache) has the few books you're currently reading โ€” instant access. The room's bookshelf (L2 cache) holds more books but takes a short walk. The building's main library (L3 / memory) has everything but requires a trip downstairs. The replacement policy decides which book to remove from your desk when it's full. The prefetcher is a helpful librarian who notices your reading pattern and pre-fetches the next book before you ask. The MSHR is the checkout slip tracking which books you've requested but haven't arrived yet.

    The cache/ directory implements gem5's Classic memory system cache hierarchy. It provides:

    • BaseCache โ€” abstract base managing tags, MSHRs, write queues, and port handling
    • Cache โ€” coherent cache with snooping and flexible topologies
    • NoncoherentCache โ€” simplified non-snooping cache
    • MSHR / WriteQueue โ€” miss tracking and write coalescing
    • Replacement Policies โ€” 12+ algorithms (LRU, FIFO, BIP, BRRIP, SHiP, TreePLRU, โ€ฆ)
    • Prefetchers โ€” 14+ engines (Stride, BOP, AMPM, SPP, DCPT, ISB, โ€ฆ)
    • Tags โ€” set-associative, fully-associative LRU, sector, compressed
    • Compressors โ€” BDI, C-Pack, FPC, FPCD, FrequentValues

    Class Hierarchy

    Click to expand
    classDiagram
        class ClockedObject {
            +clockPeriod()
            +curCycle()
        }
        class BaseCache {
            -MSHRQueue mshrQueue
            -WriteQueue writeBuffer
            -BaseTags* tags
            -BasePrefetcher* prefetcher
            +recvTimingReq()
            +recvTimingResp()
            +handleFill()
            +satisfyRequest()
        }
        class Cache {
            -doFastWrites: bool
            -outstandingSnoop: set
            +serviceMSHRTargets()
            +handleSnoop()
            +recvTimingSnoopReq()
        }
        class NoncoherentCache {
            +handleTimingReqMiss()
            +serviceMSHRTargets()
        }
        class MSHR {
            -targets: TargetList
            -deferredTargets: TargetList
            +allocate()
            +deallocate()
            +getTarget()
        }
        class WriteQueueEntry {
            +isFull(): bool
        }
        class BaseTags {
            +accessBlock()
            +findBlock()
            +insertBlock()
        }
        class BaseSetAssoc
        class FALRU
        class SectorTags
        class CompressedTags
        class ReplacementPolicy_Base {
            +getVictim()
            +reset()
            +touch()
        }
        class LRURP
        class FIFORP
        class BIPRP
        class BRRIPRP
        class SHiPRP
        class TreePLRURP
        class BasePrefetcher {
            +notify()
            +getPacket()
        }
        class Queued
        class Stride
        class BOP
        class AMPM
    
        ClockedObject <|-- BaseCache
        BaseCache <|-- Cache
        BaseCache <|-- NoncoherentCache
        BaseCache *-- MSHR
        BaseCache *-- WriteQueueEntry
        BaseCache *-- BaseTags
        BaseCache *-- BasePrefetcher
        BaseTags <|-- BaseSetAssoc
        BaseTags <|-- FALRU
        BaseTags <|-- SectorTags
        SectorTags <|-- CompressedTags
        ReplacementPolicy_Base <|-- LRURP
        ReplacementPolicy_Base <|-- FIFORP
        ReplacementPolicy_Base <|-- BIPRP
        BIPRP <|-- BRRIPRP
        ReplacementPolicy_Base <|-- SHiPRP
        ReplacementPolicy_Base <|-- TreePLRURP
        BasePrefetcher <|-- Queued
        Queued <|-- Stride
        Queued <|-- BOP
        Queued <|-- AMPM
      

    Cache Request Processing Flow

    Click to expand
    flowchart TD
        REQ["recvTimingReq()"]
        LOOKUP["Tag Lookup"]
        HIT{"Hit?"}
        RESP["Send Response"]
        MSHR_CHK{"MSHR exists?"}
        COALESCE["Coalesce into MSHR"]
        ALLOC["Allocate MSHR"]
        SEND["sendTimingReq() downstream"]
        FILL["handleFill()"]
        VICTIM["Select victim (RP)"]
        WB{"Dirty victim?"}
        WBACK["Writeback victim"]
        INSERT["Insert new block"]
        DEALLOC["Deallocate MSHR"]
    
        REQ --> LOOKUP
        LOOKUP --> HIT
        HIT -->|Yes| RESP
        HIT -->|No| MSHR_CHK
        MSHR_CHK -->|Yes| COALESCE
        MSHR_CHK -->|No| ALLOC
        ALLOC --> SEND
        SEND -->|Response arrives| FILL
        FILL --> VICTIM
        VICTIM --> WB
        WB -->|Yes| WBACK
        WB -->|No| INSERT
        WBACK --> INSERT
        INSERT --> DEALLOC
        DEALLOC --> RESP
      

    Key Classes Reference

    ClassFileDescription
    BaseCachebase.hhAbstract base for all classic caches. Manages tag arrays, MSHR queues, write buffers, prefetcher integration, and both CPU-side and memory-side ports.
    Cachecache.hhCoherent cache with full snooping support, snoop filtering, and fast-write optimization.
    NoncoherentCachenoncoherent_cache.hhCache without coherence protocol support โ€” suitable for instruction caches or accelerator scratchpads.
    MSHRmshr.hhTracks outstanding cache misses. Supports target coalescing so multiple requests to the same block share one downstream fetch.
    MSHRQueuemshr_queue.hhManages the pool of MSHR entries, blocking the cache when exhausted.
    WriteQueueEntrywrite_queue_entry.hhTracks pending writebacks and write-cleans to lower memory.
    CacheBlkcache_blk.hhRepresents a single cache block with valid/dirty bits, tag, data, coherence state.
    BaseTagstags/base.hhAbstract tag-store interface. Subclasses implement different organizations.
    BaseSetAssoctags/base_set_assoc.hhStandard set-associative tag store using configurable indexing policies.
    FALRUtags/fa_lru.hhFully-associative cache with LRU stack โ€” useful for small caches and ideal-cache studies.
    SectorTagstags/sector_tags.hhSector cache: groups of blocks share a single tag, reducing tag overhead.
    CompressedTagstags/compressed_tags.hhSupports variable-size compressed blocks within a sector-like structure.
    Base (RP)replacement_policies/base.hhAbstract replacement policy interface. All policies implement getVictim(), reset(), touch().
    Base (PF)prefetch/base.hhAbstract prefetcher base. Notified on cache accesses; generates prefetch requests.
    Queuedprefetch/queued.hhPrefetcher with a FIFO queue of pending prefetch requests, throttled by degree and distance.

    Replacement Policies

    All replacement policies live in cache/replacement_policies/ and extend a common base class. The cache invokes getVictim() to select an eviction candidate and touch() / reset() on access and insertion.

    PolicyFileAlgorithm
    LRURPlru_rp.hhLeast Recently Used โ€” evicts the block with the oldest last-access timestamp.
    FIFORPfifo_rp.hhFirst-In First-Out โ€” evicts the block inserted earliest, ignoring reuse.
    LFURPlfu_rp.hhLeast Frequently Used โ€” evicts the block with the fewest accesses.
    MRURPmru_rp.hhMost Recently Used โ€” evicts the most recently accessed block (thrash-resistant).
    RandomRPrandom_rp.hhRandom โ€” evicts a randomly chosen block.
    BIPRPbip_rp.hhBimodal Insertion โ€” inserts most blocks in LRU position; a small fraction in MRU (thrash protection).
    BRRIPRPbrrip_rp.hhBimodal Re-Reference Interval Prediction โ€” extends BIP with multi-bit RRPV counters for finer granularity.
    SHiPRPship_rp.hhSignature-based Hit Predictor โ€” uses PC-based signatures to predict reuse and set insertion priority.
    TreePLRURPtree_plru_rp.hhTree Pseudo-LRU โ€” binary tree of 1-bit pointers; hardware-efficient LRU approximation.
    SecondChanceRPsecond_chance_rp.hhSecond Chance (Clock) โ€” FIFO with a use bit; gives accessed blocks a second chance.
    DuelingRPdueling_rp.hhSet Dueling โ€” dynamically selects between two constituent policies using tournament sampling.

    Prefetch Engines

    Prefetchers in cache/prefetch/ observe cache access patterns and issue speculative fetches. The Queued base maintains a FIFO of generated addresses, throttled by configurable degree.

    PrefetcherFileStrategy
    StridePrefetcherstride.hhDetects constant-stride access patterns per PC; Atomic and Timing Simple CPUs โ€” gem5 Wiki" href="../../cpu_doc/simple/index.html">simple and effective for array traversals.
    BOPPrefetcherbop.hhBest Offset Prefetching โ€” evaluates many candidate offsets and picks the best-performing one.
    AMPMPrefetcheraccess_map_pattern_matching.hhAccess Map Pattern Matching โ€” tracks access bitmaps per page zone to detect complex patterns.
    SignaturePathPrefetchersignature_path.hhSPP โ€” builds a signature table from delta history; follows lookahead paths for deep prefetching.
    SignaturePathPrefetcherV2signature_path_v2.hhSPP v2 with enhanced confidence and global accuracy feedback.
    DCPTPrefetcherdelta_correlating_prediction_tables.hhDelta Correlating Prediction Tables โ€” correlates deltas between consecutive addresses.
    IndirectMemoryPrefetcherindirect_memory.hhHandles pointer-chasing patterns by tracking indirect memory accesses.
    IrregularStreamBufferirregular_stream_buffer.hhISB โ€” maps structural addresses to temporal streams for irregular workloads.
    SBOOEPrefetchersbooe.hhSandbox-Based Optimal Offset Estimation โ€” lightweight offset evaluation using sandbox.
    PIFPrefetcherpif.hhProactive Instruction Fetch โ€” uses temporal address correlation for instruction streams.
    STeMSPrefetcherspatio_temporal_memory_streaming.hhSpatio-Temporal Memory Streaming โ€” combines spatial and temporal patterns.
    SlimAMPMPrefetcherslim_ampm.hhLightweight AMPM variant with reduced storage.
    TaggedPrefetchertagged.hhSimple next-line prefetcher triggered on every cache access.
    MultiPrefetchermulti.hhWrapper that combines multiple prefetcher instances.

    MSHR & Write Buffer Handling

    MSHRs are the backbone of non-blocking cache operation. Each MSHR tracks one outstanding cache line fetch and coalesces all requests targeting the same block.

    MSHR Lifecycle

    Allocation โ€” On a cache miss, MSHRQueue::allocate() creates an MSHR if one is available. If all MSHRs are busy, the cache blocks.
    Target coalescing โ€” Subsequent misses to the same block are added as targets to the existing MSHR instead of issuing new downstream requests.
    Downstream fetch โ€” The MSHR triggers a sendTimingReq() to the next cache level or memory controller.
    Response handling โ€” When data returns, BaseCache::recvTimingResp() calls handleFill() to insert the block and service all coalesced targets.
    Deallocation โ€” The MSHR is freed and the cache unblocks, allowing new misses to proceed.

    Write Buffer

    The WriteQueue holds pending writebacks and write-cleans. When a dirty block is evicted, a WriteQueueEntry is allocated. The write buffer has its own flow control and can cause the cache to block independently of MSHRs.

    // Simplified MSHR allocation in BaseCache::access()
    MSHR *mshr = mshrQueue.findMatch(blk_addr);
    if (mshr) {
        // Coalesce: add this request as a target
        mshr->allocateTarget(pkt, pkt->headerDelay, order++);
    } else {
        // New miss: allocate MSHR and send downstream
        mshr = mshrQueue.allocate(blk_addr, blk_size, pkt, curTick(), order++);
        memSidePort.sendTimingReq(mshr->getTarget()->pkt);
    }

    Tag Organization

    Tag stores determine how cache blocks are indexed and looked up. gem5 provides several implementations:

    Tag TypeFileOrganization
    BaseSetAssoctags/base_set_assoc.hhStandard set-associative with configurable indexing (modular, skewed, etc.).
    FALRUtags/fa_lru.hhFully-associative with LRU tracking. Useful for small filter caches or way-profiling studies.
    SectorTagstags/sector_tags.hhGroups of consecutive blocks share one tag. Reduces tag area for large caches.
    CompressedTagstags/compressed_tags.hhExtends sector tags: multiple compressed blocks can share a data entry if they fit.

    Component Deep Dives

    Replacement Policies

    12 algorithms from simple LRU/FIFO to advanced BRRIP and SHiP. Extends ReplacementPolicy::Base with getVictim(), reset(), touch(), and invalidate() hooks.

    replacement_policies/ ยท 12 policies ยท All extend Base

    Prefetch Algorithms

    14+ prefetchers from simple tagged/stride to sophisticated SPP, AMPM, and ISB. All hook into cache access notifications via BasePrefetcher::notify().

    prefetch/ ยท 14+ engines ยท Queued base class

    Compression

    Data compressors for compressed caches: BDI (Base-Delta-Immediate), C-Pack, FPC (Frequent Pattern Compression), FPCD, and FrequentValues. Compress blocks to fit more data in same area.

    compressors/ ยท 8+ algorithms ยท CompressedTags integration

    MSHR & Write Queue

    Non-blocking cache operation with configurable MSHR count, target coalescing, deferred targets for snoop handling, and separate write buffer for writebacks.

    mshr.hh, write_queue.hh ยท Core to cache performance

    Indexing Policies

    Set selection strategies: modular (traditional), skewed (XOR-based), and custom policies in tags/indexing_policies/.

    tags/indexing_policies/ ยท Pluggable indexing

    Partitioning Policies

    Cache partitioning for QoS: way partitioning and capacity allocation across requestors, managed by PartitionManager.

    tags/partitioning_policies/ ยท QoS integration

    Configuration Examples

    Classic cache hierarchy is configured through Python SimObjects. Key parameters:

    // Python configuration example (gem5 style)
    // L1 Data Cache
    system.cpu.dcache = Cache(
        size = '64kB',
        assoc = 8,
        tag_latency = 2,
        data_latency = 2,
        response_latency = 2,
        mshrs = 4,
        tgts_per_mshr = 20,
        write_buffers = 8,
        replacement_policy = LRURP(),
        prefetcher = StridePrefetcher(degree=4)
    )
    
    // L2 Shared Cache
    system.l2cache = Cache(
        size = '2MB',
        assoc = 16,
        tag_latency = 10,
        data_latency = 10,
        response_latency = 10,
        mshrs = 20,
        tgts_per_mshr = 12,
        write_buffers = 16,
        clusivity = 'mostly_incl',
        replacement_policy = TreePLRURP(),
        prefetcher = BOPPrefetcher()
    )
    ParameterTypeDescriptionTypical L1Typical L2
    sizeMemorySizeTotal cache capacity32-64 kB256 kB - 4 MB
    associntWays per set4-88-16
    tag_latencyCyclesTag array lookup latency1-28-20
    data_latencyCyclesData array read latency1-28-20
    mshrsintNumber of MSHRs (outstanding misses)4-816-32
    write_buffersintWrite buffer entries816-32
    clusivityenumInclusion policyโ€”mostly_incl
    replacement_policySimObjectReplacement policy instanceLRURPTreePLRURP
    prefetcherSimObjectPrefetcher instanceStrideBOP/SPP

    Directory Map

    src/mem/cache/ โ”œโ”€โ”€ base.hh / base.cc # BaseCache โ€” abstract cache base โ”œโ”€โ”€ cache.hh / cache.cc # Cache โ€” coherent cache โ”œโ”€โ”€ noncoherent_cache.hh / .cc # NoncoherentCache โ”œโ”€โ”€ cache_blk.hh / .cc # CacheBlk โ€” block representation โ”œโ”€โ”€ mshr.hh / mshr.cc # MSHR โ€” miss tracking โ”œโ”€โ”€ mshr_queue.hh / .cc # MSHRQueue โ€” MSHR pool โ”œโ”€โ”€ write_queue.hh / .cc # WriteQueue โ€” writeback buffer โ”œโ”€โ”€ write_queue_entry.hh / .cc # WriteQueueEntry โ”œโ”€โ”€ queue.hh / queue_entry.hh # Generic queue infrastructure โ”œโ”€โ”€ cache_probe_arg.hh # Probe argument types โ”œโ”€โ”€ Cache.py # Python SimObject definitions โ”œโ”€โ”€ replacement_policies/ โ”‚ โ”œโ”€โ”€ base.hh # ReplacementPolicy::Base โ”‚ โ”œโ”€โ”€ lru_rp.hh / .cc # LRU โ”‚ โ”œโ”€โ”€ fifo_rp.hh / .cc # FIFO โ”‚ โ”œโ”€โ”€ lfu_rp.hh / .cc # LFU โ”‚ โ”œโ”€โ”€ mru_rp.hh / .cc # MRU โ”‚ โ”œโ”€โ”€ random_rp.hh / .cc # Random โ”‚ โ”œโ”€โ”€ bip_rp.hh / .cc # BIP โ”‚ โ”œโ”€โ”€ brrip_rp.hh / .cc # BRRIP โ”‚ โ”œโ”€โ”€ ship_rp.hh / .cc # SHiP โ”‚ โ”œโ”€โ”€ tree_plru_rp.hh / .cc # Tree Pseudo-LRU โ”‚ โ”œโ”€โ”€ second_chance_rp.hh / .cc # Second Chance โ”‚ โ””โ”€โ”€ dueling_rp.hh / .cc # Set Dueling โ”œโ”€โ”€ prefetch/ โ”‚ โ”œโ”€โ”€ base.hh / .cc # BasePrefetcher โ”‚ โ”œโ”€โ”€ queued.hh / .cc # Queued (common queue logic) โ”‚ โ”œโ”€โ”€ stride.hh / .cc # Stride prefetcher โ”‚ โ”œโ”€โ”€ bop.hh / .cc # Best Offset Prefetching โ”‚ โ”œโ”€โ”€ access_map_pattern_matching.hh # AMPM โ”‚ โ”œโ”€โ”€ signature_path.hh / .cc # SPP โ”‚ โ”œโ”€โ”€ delta_correlating_prediction_tables.hh # DCPT โ”‚ โ”œโ”€โ”€ indirect_memory.hh / .cc # Indirect memory โ”‚ โ””โ”€โ”€ ... (14+ engines) โ”œโ”€โ”€ tags/ โ”‚ โ”œโ”€โ”€ base.hh / .cc # BaseTags โ”‚ โ”œโ”€โ”€ base_set_assoc.hh / .cc # Set-associative โ”‚ โ”œโ”€โ”€ fa_lru.hh / .cc # Fully-associative LRU โ”‚ โ”œโ”€โ”€ sector_tags.hh / .cc # Sector cache โ”‚ โ”œโ”€โ”€ compressed_tags.hh / .cc # Compressed cache โ”‚ โ”œโ”€โ”€ indexing_policies/ # Set index functions โ”‚ โ””โ”€โ”€ partitioning_policies/ # Way partitioning โ””โ”€โ”€ compressors/ โ”œโ”€โ”€ base.hh / .cc # Base compressor โ”œโ”€โ”€ base_delta.hh / .cc # BDI compressor โ”œโ”€โ”€ cpack.hh / .cc # C-Pack โ”œโ”€โ”€ fpc.hh / .cc # FPC โ”œโ”€โ”€ fpcd.hh / .cc # FPCD โ””โ”€โ”€ frequent_values.hh / .cc # FrequentValues