python

Configuration System and Standard Library โ€” the control plane of gem5

๐Ÿ“ 236 files ๐Ÿ“ 40K LOC ๐Ÿ“š stdlib ๐Ÿ”— PyBind11 ๐Ÿ Python
SimObject Standard Library Components Configuration Resource Management
~10 min read
๐ŸŽ›๏ธ Analogy: The Python module is like the control panel of an elaborate model railway. You don't build the engines or tracks in Python โ€” those exist in C++. Instead, you use Python to wire together boards, processors, caches, and memory into a complete system, set every knob and dial, then press "run." The m5 package provides the raw wiring, while the gem5 standard library gives you pre-assembled modules you can snap together.
Table of Contents

Architecture Overview

The Python module is divided into two major packages: the low-level m5 infrastructure that provides PyBind11 bindings to C++ simulator objects, and the high-level gem5 standard library that offers composable board/processor/memory abstractions. Together they enable full system configuration from Python scripts.

flowchart TD
    User["User Config Script"]
    Sim["gem5.simulate.Simulator"]
    Board["gem5.components.boards"]
    Proc["gem5.components.processors"]
    Mem["gem5.components.memory"]
    Cache["gem5.components.cachehierarchies"]
    Res["gem5.resources"]
    M5["m5.SimObject"]
    Params["m5.params"]
    Simulate["m5.simulate"]
    Bind["_m5 (PyBind11 C++)"]
    CPP["C++ Simulator Core"]

    User --> Sim
    Sim --> Board
    Board --> Proc
    Board --> Mem
    Board --> Cache
    Sim --> Res
    Board --> M5
    M5 --> Params
    M5 --> Simulate
    Simulate --> Bind
    Bind --> CPP

The architecture follows a layered design: user scripts interact with Simulator, which orchestrates AbstractBoard subclasses. Boards compose processors, caches, and memory modules. At the bottom, m5.SimObject and its metaclass system create C++ objects via the _m5 PyBind11 bridge.

m5 Package โ€” Core Bindings

The m5 package provides the foundational infrastructure that bridges Python configuration to C++ simulation objects. Key submodules:

ModulePurpose
m5.SimObjectBase class for all configurable objects; metaclass-driven parameter system
m5.simulateSimulation execution: instantiate(), simulate(), drain(), checkpoint()
m5.paramsParameter descriptors: ParamDesc, VectorParamDesc, Port
m5.proxyDynamic parameter resolution: Self, Parent proxies
m5.eventEvent scheduling interface to C++ EventQueue
m5.statsPipeline โ€” gem5 Wiki" class="xref">Statistics collection and output formatting
m5.coreTick management, debugging, build info
m5.ticksTime-to-tick conversion utilities
m5.traceDebug trace flag management
m5.objectsAuto-generated SimObject subclass registry

MetaSimObject Metaclass

The MetaSimObject metaclass intercepts class creation for every SimObject subclass. It collects parameter declarations, builds parameter descriptors, resolves inheritance, and registers the class in the global SimObject type registry. This allows the declarative Python style to automatically generate C++ parameter structs.

gem5 Standard Library

The modern gem5 standard library provides a component-based API for building simulation systems without deep knowledge of m5 internals.

๐Ÿ“‹ boards

AbstractBoard connects processor, Cache Hierarchy | gem5 Wiki" href="../mem_doc/cache/index.html">cache hierarchy, and memory into a complete system. Implementations: SimpleBoard, X86Board, ArmBoard, RiscvBoard.

โš™๏ธ processors

AbstractProcessor wraps CPU cores. Implementations: SimpleProcessor (timing/atomic/O3), SwitchableProcessor for CPU switching.

๐Ÿ’พ memory

AbstractMemorySystem for DRAM configuration. SingleChannelDDR4, DualChannelDDR4, HBM2Stack.

๐Ÿ”„ cachehierarchies

Cache system abstraction. Classic (PrivateL1/SharedL2), Ruby (MESI, MOESI, CHI protocols).

๐Ÿ“ฆ resources

WorkloadResource, BinaryResource, DiskImageResource โ€” automatic download and caching from gem5-resources repository.

๐Ÿ—๏ธ prebuilt

Ready-to-use configurations: demo boards (X86/ARM/RISC-V), Viper GPU board, RiscvMatched SiFive board.

Key Classes

ClassPackageParentDescription
MetaSimObjectm5typeMetaclass that intercepts SimObject class creation; collects params
SimObjectm5โ€”Base for all configurable objects; parameter declaration and C++ binding
SimObjectCliWrapperm5โ€”CLI parameter override interface for SimObjects
Simulatorgem5.simulateโ€”Top-level simulation runner; manages board, exit events, run loop
AbstractBoardgem5.componentsSubSystemConnects processor + cache + memory; handles clocking and ranges
AbstractProcessorgem5.componentsSubSystemCPU abstraction; exposes cores, ISA, switching interface
AbstractMemorySystemgem5.componentsSubSystemMemory controller configuration and range management
AbstractCacheHierarchygem5.componentsSubSystemCache topology abstraction; classic and Ruby implementations
WorkloadResourcegem5.resourcesโ€”Downloadable workload binaries with metadata
obtain_resourcegem5.resourcesfunctionFetches and caches resources from gem5-resources repository
SimPointgem5.utilsโ€”SimPoint analysis utility for sampling-based simulation
requiresgem5.utilsfunctionRuntime ISA/protocol compatibility checks

Deep Dive: SimObject System

Parameter Lifecycle

sequenceDiagram
    participant U as User Script
    participant Meta as MetaSimObject
    participant SO as SimObject
    participant Inst as m5.simulate
    participant CPP as C++ Object

    U->>Meta: class MyObj(SimObject): type='MyObj' ...
    Meta->>Meta: Collect params, build descriptors
    U->>SO: obj = MyObj(param1=val)
    SO->>SO: Store params in _values dict
    U->>Inst: instantiate(root)
    Inst->>SO: adoptOrphanParams()
    Inst->>SO: unproxyParams() (resolve Self/Parent)
    Inst->>CPP: _create_cpp_objects()
    Inst->>CPP: connectPorts()
    Inst->>CPP: init() / startup()

The instantiation sequence resolves all proxy parameters (Self.attr, Parent.attr), creates the C++ objects in dependency order, connects ports, and calls init() and startup() on each object. The configuration can be dumped in INI, JSON, or DOT format for debugging.

Parameter Types

DescriptorUsageExample
Param.IntSingle integercache_size = Param.MemorySize('64kB')
VectorParamList of valuesport = VectorRequestPort('mem ports')
DictParamDescKey-value mappingDictionary-style parameters
Proxy.SelfLate bindingclk_domain = Self.clk_domain
PortConnection pointmem_side = RequestPort('to memory')

Deep Dive: Component Model

flowchart TD
    AB["AbstractBoard"]
    AP["AbstractProcessor"]
    AM["AbstractMemorySystem"]
    AC["AbstractCacheHierarchy"]

    AB -->|"has-a"| AP
    AB -->|"has-a"| AM
    AB -->|"has-a"| AC

    AP --> SP["SimpleProcessor"]
    AP --> SWP["SwitchableProcessor"]
    AM --> SC["SingleChannelDDR4"]
    AM --> DC["DualChannelDDR4"]
    AC --> CL["ClassicCacheHierarchy"]
    AC --> RB["RubyCacheHierarchy"]

    AB --> SB["SimpleBoard"]
    AB --> X86["X86Board"]
    AB --> ARM["ArmBoard"]

The component model follows an abstract factory pattern. Each board implements _pre_instantiate() to wire components and get_memory_ranges() to define the address map. The connect_things() method links processor ports to cache hierarchy ports and cache to memory controller ports.

Simulation Flow

flowchart LR
    A["Create Board"] --> B["Set Workload"]
    B --> C["Create Simulator"]
    C --> D["simulator.run()"]
    D --> E["instantiate()"]
    E --> F["simulate()"]
    F --> G{Exit Event?}
    G -->|"handler"| H["On-exit handler"]
    H --> F
    G -->|"done"| I["Stats dump & cleanup"]

Runtime Operations

The simulation engine supports runtime control operations through the m5.simulate module:

Cross-Module Interactions

DependencyModulePurpose
C++ bindings_m5 (PyBind11)Core simulation engine, object creation, event scheduling
Parameter structsall src/ modulesEvery C++ SimObject has auto-generated Python params
gem5-resourcesexternalWorkload binaries, disk images, checkpoints
ISA definitionsarch/*ISA-specific board and processor configurations
Memory systemmemSystem | gem5 Wiki" class="xref">MemCtrl, DRAM interface parameter wrappers

Directory Map

src/python/ โ”œโ”€โ”€ m5/ โ€” Core infrastructure package โ”‚ โ”œโ”€โ”€ SimObject.py โ€” MetaSimObject + SimObject base class โ”‚ โ”œโ”€โ”€ simulate.py โ€” instantiate(), simulate(), drain() โ”‚ โ”œโ”€โ”€ params.py โ€” Parameter descriptor system โ”‚ โ”œโ”€โ”€ proxy.py โ€” Self/Parent proxy resolution โ”‚ โ”œโ”€โ”€ core.py โ€” Tick management, build info โ”‚ โ”œโ”€โ”€ event.py โ€” EventQueue Python interface โ”‚ โ”œโ”€โ”€ debug.py โ€” Debug flag management โ”‚ โ”œโ”€โ”€ trace.py โ€” Trace output control โ”‚ โ”œโ”€โ”€ ticks.py โ€” Time conversion utilities โ”‚ โ”œโ”€โ”€ params/ โ€” Auto-generated param classes โ”‚ โ”œโ”€โ”€ objects/ โ€” Auto-generated SimObject wrappers โ”‚ โ”œโ”€โ”€ stats/ โ€” Statistics output formatters โ”‚ โ””โ”€โ”€ ext/ โ€” Third-party extensions โ”œโ”€โ”€ gem5/ โ€” Standard library โ”‚ โ”œโ”€โ”€ simulate/ โ”‚ โ”‚ โ””โ”€โ”€ simulator.py โ€” Simulator orchestrator class โ”‚ โ”œโ”€โ”€ components/ โ”‚ โ”‚ โ”œโ”€โ”€ boards/ โ€” AbstractBoard + implementations โ”‚ โ”‚ โ”œโ”€โ”€ processors/ โ€” AbstractProcessor + implementations โ”‚ โ”‚ โ”œโ”€โ”€ memory/ โ€” AbstractMemorySystem + DDR configs โ”‚ โ”‚ โ”œโ”€โ”€ cachehierarchies/ โ€” Classic + Ruby cache hierarchies โ”‚ โ”‚ โ””โ”€โ”€ devices/ โ€” I/O and peripheral components โ”‚ โ”œโ”€โ”€ resources/ โ€” Resource download and caching โ”‚ โ”œโ”€โ”€ utils/ โ”‚ โ”‚ โ”œโ”€โ”€ requires.py โ€” ISA/protocol compatibility checks โ”‚ โ”‚ โ”œโ”€โ”€ simpoint.py โ€” SimPoint analysis utility โ”‚ โ”‚ โ”œโ”€โ”€ override.py โ€” Method override decorator โ”‚ โ”‚ โ””โ”€โ”€ multisim/ โ€” Multi-simulation orchestration โ”‚ โ””โ”€โ”€ prebuilt/ โ”‚ โ”œโ”€โ”€ demo/ โ€” Demo boards (x86, AArch32, AArch64, TrustZone security extensions, SVE/SVE2 vector processing, PMU, and hardware transactional memory." href="../arch_doc/arm/index.html">ARM, RISC-V) โ”‚ โ”œโ”€โ”€ viper/ โ€” AMD Viper GPU board โ”‚ โ””โ”€โ”€ riscvmatched/ โ€” SiFive Unmatched board โ””โ”€โ”€ importer.py โ€” Custom Python import machinery

Configuration Patterns

PatternAPIUse Case
Atomic and Timing Simple CPUs โ€” gem5 Wiki" href="../cpu_doc/simple/index.html">Simple SE modeSimpleBoard + SimpleProcessor + SingleChannelDDR4Quick syscall emulation experiments
Full SystemX86Board + SwitchableProcessor + DualChannelDDR4Full OS boot with CPU switching
GPU computeprebuilt.viper.BoardAMD GPU + CPU heterogeneous system
CustomSubclass AbstractBoard + override connect_things()Bespoke system topologies

Code Examples

Simple SE-mode Simulation

from gem5.components.boards.simple_board import SimpleBoard
from gem5.components.processors.simple_processor import SimpleProcessor
from gem5.components.memory.single_channel import SingleChannelDDR4_2400
from gem5.components.cachehierarchies.classic.private_l1_shared_l2 import (
    PrivateL1SharedL2CacheHierarchy,
)
from gem5.isas import ISA
from gem5.resources.resource import obtain_resource
from gem5.simulate.simulator import Simulator

# Build system components
processor = SimpleProcessor(cpu_type="timing", isa=ISA.X86, num_cores=1)
cache = PrivateL1SharedL2CacheHierarchy(l1d_size="32kB", l1i_size="32kB",
                                          l2_size="256kB")
memory = SingleChannelDDR4_2400(size="2GiB")

# Assemble board
board = SimpleBoard(processor=processor, cache_hierarchy=cache,
                     memory=memory, clk_freq="3GHz")
board.set_se_binary_workload(obtain_resource("x86-hello64-static"))

# Run simulation
sim = Simulator(board=board)
sim.run()

Custom SimObject Declaration

from m5.SimObject import SimObject
from m5.params import Param, VectorParam

class MyAccelerator(SimObject):
    """Custom accelerator SimObject with configurable parameters."""
    type = 'MyAccelerator'
    cxx_header = 'my_module/accelerator.hh'
    cxx_class = 'gem5::MyAccelerator'

    num_engines = Param.Int(4, "Number of compute engines")
    queue_depth = Param.Int(16, "Command queue depth")
    clock_freq = Param.Clock('1GHz', "Operating frequency")
    mem_ports = VectorRequestPort("Memory-side ports")

CPU Switching at Runtime

from gem5.components.processors.switchable_processor import SwitchableProcessor

processor = SwitchableProcessor(
    starting_core_type="atomic",
    switch_core_type="o3",
    isa=ISA.ARM,
    num_cores=4,
)

# In exit event handler:
def switch_handler():
    processor.switch()  # Drains and swaps CPU models
    yield False         # Continue simulation