pci — PCI Bus and Device Framework

Class Architecture

The PCI subsystem is built on a layered class hierarchy. PciDevice inherits from DmaDevice (which provides both PIO and DMA ports), and specializes into Type 0 endpoints and Type 1 bridges:

classDiagram
    class ClockedObject {
        +clockPeriod()
        +curCycle()
    }
    class PioDevice {
        #PioPort pioPort
        +read(pkt)* Tick
        +write(pkt)* Tick
        +getAddrRanges()* AddrRangeList
    }
    class DmaDevice {
        #DmaPort dmaPort
        +dmaRead(addr, size, event, data)
        +dmaWrite(addr, size, event, data)
        +dmaPending() bool
    }
    class PciDevice {
        #PCIConfig _config
        #vector~PciBar*~ BARs
        #PciUpstream::DeviceInterface upstreamInterface
        +read(pkt) Tick
        +write(pkt) Tick
        +readConfig(pkt)* Tick
        +writeConfig(pkt)* Tick
        +readDevice(pkt)* Tick
        +writeDevice(pkt)* Tick
        +intrPost()
        +intrClear()
        +pciToDma(pci_addr) Addr
        +getAddrRanges() AddrRangeList
    }
    class PciEndpoint {
        #PCIConfigType0 config
        +writeConfig(pkt) Tick
    }
    class PciType1Device {
        #PCIConfigType1 config
        +writeConfig(pkt) Tick
    }
    class PciBar {
        #Addr _addr
        #Addr _size
        +write(interface, val)* uint32_t
        +range() AddrRange
        +isMem()* bool
        +isIo()* bool
    }
    class PciBarNone {
        +write() uint32_t
    }
    class PciIoBar {
        +write() uint32_t
        +isIo() bool
    }
    class PciMemBar {
        -bool _wide
        -uint64_t _lower
        -uint64_t _upper
        +write() uint32_t
        +isMem() bool
        +wide() bool
    }
    class PciMemUpperBar {
        -PciMemBar* _lower
        +write() uint32_t
    }
    class PciHost {
        +getConfigAddrRange()
    }
    class GenericPciHost {
        #Platform platform
        #Addr confBase
        #Addr pciPioBase
        #Addr pciMemBase
        +mapPciInterrupt()
    }
    class PciBus {
        #PortID configErrorPortID
        #AddrRange configRange
        +findPort() PortID
    }

    ClockedObject <|-- PioDevice
    PioDevice <|-- DmaDevice
    DmaDevice <|-- PciDevice
    PciDevice <|-- PciEndpoint
    PciDevice <|-- PciType1Device
    PciDevice o-- PciBar
    PciBar <|-- PciBarNone
    PciBar <|-- PciIoBar
    PciBar <|-- PciMemBar
    PciBar <|-- PciMemUpperBar
    PciIoBar <|-- PciLegacyIoBar
    PciHost <|-- GenericPciHost
    GenericPciHost --> PciBus : manages
    PciDevice --> PciHost : registers with

PCI Configuration Space

Every PCI device has a 256-byte configuration header. The first 64 bytes are standardized (Type 0 for endpoints, Type 1 for bridges). gem5 implements this via PCIConfig, PCIConfigType0, and PCIConfigType1 unions in pcireg.h.

Configuration Access Flow

sequenceDiagram
    participant OS as Guest OS
    participant HOST as GenericPciHost
    participant BUS as PciBus
    participant DEV as PciDevice

    Note over OS,DEV: Config Read — OS enumerates device
    OS->>HOST: MMIO read to ECAM region (confBase + BDF offset)
    HOST->>HOST: Decode bus/device/function from address
    HOST->>BUS: Forward to config address range
    BUS->>DEV: Route to device's config port
    DEV->>DEV: readConfig(pkt) — return from _config
    DEV-->>BUS: Config data response
    BUS-->>HOST: Forward response
    HOST-->>OS: Return vendor ID, device ID, etc.

    Note over OS,DEV: BAR Programming — OS assigns addresses
    OS->>HOST: MMIO write to BAR register offset
    HOST->>BUS: Forward write
    BUS->>DEV: writeConfig(pkt)
    DEV->>DEV: PciBar::write() — update address mapping
    DEV-->>OS: BAR programmed, device responds at new address

BAR Types

PCI Host Bridge

GenericPciHost is the configurable root complex that bridges between the system memory bus and the PCI subsystem. It provides three key mappings:

Interrupt Routing

The host bridge maps PCI interrupt pins (INTA#–INTD#) to platform-level interrupt numbers. The default mapPciInterrupt() uses the device's interrupt line register, but subclasses can override this for platform-specific swizzling (e.g., AArch32, AArch64, TrustZone security extensions, SVE/SVE2 vector processing, PMU, and hardware transactional memory." href="../../arch_doc/arm/index.html">ARM GIC SPI mapping).

// GenericPciHost interrupt delivery
void GenericPciHost::interfacePostInt(const PciDevAddr &addr, PciIntPin pin)
{
    platform.postPciInt(mapPciInterrupt(addr, pin));
}

void GenericPciHost::interfaceClearInt(const PciDevAddr &addr, PciIntPin pin)
{
    platform.clearPciInt(mapPciInterrupt(addr, pin));
}

PCI Bus

PciBus extends NoncoherentXBar to implement PCI bus topology. It adds a special config error port that catches configuration accesses to non-existent devices (returning 0xFFFFFFFF, the standard PCI "no device" response).

// PciBus — a non-coherent crossbar with config error handling
class PciBus : public NoncoherentXBar
{
  public:
    PARAMS(PciBus);
    PciBus(const Params &p);

    Port &getPort(const std::string &if_name,
                  PortID idx = InvalidPortID) override;

  protected:
    // Override to handle config space routing
    PortID findPort(AddrRange addr_range, PacketPtr pkt = nullptr) override;

    PortID configErrorPortID;
    AddrRange configRange;
};

Key Classes Reference

Device Lifecycle

Registration and Discovery

When a PciDevice is constructed, it receives a PciUpstream::DeviceInterface from the PCI host. This interface provides methods for address translation and interrupt delivery:

// PciUpstream::DeviceInterface methods used by PciDevice
Addr pioAddr(Addr pci_addr);   // translate PCI I/O addr → physical
Addr memAddr(Addr pci_addr);   // translate PCI mem addr → physical
Addr dmaAddr(Addr pci_addr);   // translate PCI bus addr → DMA addr
void postInt();                 // post interrupt via host bridge
void clearInt();                // clear interrupt via host bridge

Read/Write Dispatch

PciDevice::read() and write() are final — they dispatch to config space or device space based on the address:

flowchart TD
    A["Incoming Packet"] --> B{Address in config range?}
    B -->|Yes| C["readConfig() / writeConfig()"]
    B -->|No| D{Address matches a BAR?}
    D -->|Yes| E["readDevice() / writeDevice()"]
    D -->|No| F["Error response"]
    C --> G["Return config register data"]
    E --> H["Device-specific logic"]

MSI and MSI-X Support

gem5's PCI implementation supports both MSI (Message Signaled Interrupts) and MSI-X. The capability structures are stored in the device's config space:

MSI-X Table Structure

// MSI-X table entry in PciDevice
struct MSIXTable {
    uint32_t addr_lo;   // Lower 32 bits of message address
    uint32_t addr_hi;   // Upper 32 bits of message address
    uint32_t msg_data;  // Message data value
    uint32_t vec_ctrl;  // Vector control (bit 0 = mask)
};

// MSI-X PBA (Pending Bit Array)
struct MSIXPbaEntry {
    uint64_t bits;      // 64 pending bits per entry
};

Copy Engine — A PCI Device Example

The CopyEngine in pci/copy_engine.hh is a concrete example of a PCI endpoint device. It models the Intel I/O Acceleration Technology (I/O AT) DMA copy engine, which the guest OS can use to offload memory copies:

// Simplified usage in gem5 Python config
from m5.objects import CopyEngine, PciMemBar

copy_engine = CopyEngine(
    pci_bus=0, pci_dev=4, pci_func=0,
    InterruptLine=0x20,
    BAR0=PciMemBar(size='4KiB')  # Control registers
)
system.pci_host.attach(copy_engine)

Configuration Reference

File Map

src/dev/pci/
├── device.hh / device.cc       ← PciDevice, PciEndpoint, PciType1Device, all PciBar variants
├── host.hh / host.cc           ← PciHost, GenericPciHost (root complex)
├── bus.hh / bus.cc             ← PciBus (NoncoherentXBar with config routing)
├── upstream.hh / upstream.cc   ← PciUpstream interface and DeviceInterface
├── up_down_bridge.hh/cc        ← PciUpDownBridge (bus-to-bus bridge)
├── types.hh                    ← PciDevAddr, PciBusNum, PciIntPin types
├── pcireg.h                    ← PCI config register layouts (PCIConfigType0/1)
├── copy_engine.hh / copy_engine.cc     ← Intel I/O AT copy engine device
├── copy_engine_defs.hh         ← Copy engine register definitions
├── PciDevice.py                ← Python config for PciDevice, PciBar, PciEndpoint
├── PciHost.py                  ← Python config for PciHost, GenericPciHost
├── CopyEngine.py               ← Python config for CopyEngine
├── PciUpstream.py              ← Python config for PciUpstream
└── SConscript                  ← Build definitions

Creating a New PCI Device

To implement a custom PCI endpoint device in gem5, inherit from PciEndpoint and override readDevice() and writeDevice():

// my_pci_device.hh
#include "dev/pci/device.hh"
#include "params/MyPciDevice.hh"

namespace gem5 {

class MyPciDevice : public PciEndpoint
{
  public:
    PARAMS(MyPciDevice);
    MyPciDevice(const Params &p);

  protected:
    // Handle reads to BAR-mapped addresses
    Tick readDevice(PacketPtr pkt) override
    {
        int bar_num;
        Addr offset;
        getBAR(pkt->getAddr(), bar_num, offset);

        // Decode register at offset within BAR
        switch (offset) {
          case 0x00: // STATUS register
            pkt->setLE<uint32_t>(statusReg);
            break;
          case 0x04: // DATA register
            pkt->setLE<uint32_t>(dataReg);
            break;
          default:
            pkt->setLE<uint32_t>(0);
        }
        pkt->makeAtomicResponse();
        return pioDelay;
    }

    // Handle writes to BAR-mapped addresses
    Tick writeDevice(PacketPtr pkt) override
    {
        int bar_num;
        Addr offset;
        getBAR(pkt->getAddr(), bar_num, offset);

        uint32_t val = pkt->getLE<uint32_t>();
        switch (offset) {
          case 0x08: // COMMAND register
            processCommand(val);
            break;
        }
        pkt->makeAtomicResponse();
        return pioDelay;
    }

  private:
    uint32_t statusReg = 0;
    uint32_t dataReg = 0;
    void processCommand(uint32_t cmd);
};

} // namespace gem5

Glossary