PCIe

Configuration Mohammed Rabil

Space and
BAR
So Far…
Configuration Space

 The first PCs required users to set switches and jumpers to assign resources
for each card installed and this frequently resulted in conflicting memory, IO
and interrupt settings.
 The subsequent IO architectures, Extended ISA (EISA) and the IBM PS2
systems, were the first to implemented plug and play architectures.
 In these architectures configuration files were shipped with each plug‐in card
that allowed system software to assign basic resources.
 PCI extended this capability by implementing standardized configuration
registers that permit generic shrink‐wrapped OSs to manage virtually all
system resources
 Having a standard way to enable error reporting, interrupt delivery, address
mapping and more, allows one entity, the configuration software, to allocate
and configure the system resources which virtually eliminates resource
conflicts.
Configuration Space
 configuration space was introduced with PCI to allow software to
control and check the status of devices in a standardized way.
 PCI Express was designed to be software backwards compatible with
PCI, so configuration space is still supported and used for the same
reason as it was in PCI.
 Even though configuration space was originally meant to hold
standardized structures (PCI‐defined headers, capability structures,
etc.), it is very common for PCIe devices to have device‐specific
registers mapped into their config space.
 In these cases, the device‐specific registers mapped into config space
are often control, status or pointer registers as opposed to data
storage locations
I Need an Address
 Almost all devices have internal registers or storage locations that software (and
potentially other devices) need to be able to access.
 These internal locations may control the device’s behavior, report the status of the
device, or may be a location to hold data for the device to process.
 Regardless of the purpose of the internal registers/storage, it is important to be
able to access them from outside the device itself.
 This means these internal locations need to be addressable.
 Software must be able to perform a read or write operation with an address that
will access the appropriate internal location within the targeted device.
 In order to make this work, these internal locations need to be assigned addresses
from one of the address spaces supported in the system.

 PCI Express supports the exact same three address spaces that were supported in
PCI:
 Configuration
 Memory
 IO
Memory and IO Address Spaces
 In the early days of PCs, the internal registers/storage in IO devices
were accessed via IO address space (as defined by Intel).
 However, because of several limitations and undesirable effects
related to IO address space, that we will not be going into here, that
address space quickly lost favor with software and hardware vendors.
 This resulted in the internal registers/storage of IO devices being
mapped into memory address space (commonly referred to as
memory mapped IO, or MMIO).
 However, because early software was written to use IO address space
to access internal registers/storage on IO devices, it became common
practice to map the same set of device‐specific registers in memory
address space as well as in IO address space.
 This allows new software to access the internal locations of a device
using memory address space (MMIO), while allowing legacy (old)
software to continue to function because it can still access the
internal registers of devices using IO address space.
Generic Memory and IO Address
Space 
The size of the memory map is a
function of the range of addresses that
the system can use (often dictated by
the CPU addressable range).
 The size of the IO map in PCIe is
limited to 32 bits (4GB), although in
many computers using Intel‐
compatible (x86) processors, only the
lower 16 bits (64KB) are used.
 PCIe can support memory addresses
up to 64 bits in size.
 showing MMIO and IO space being
claimed by Endpoints, but that ability
is not exclusive to Endpoints. It is very
common for Switches and Root
Complexes to also have device‐specific
registers accessed via MMIO and IO
addresses.
Prefetchable Memory Space
 Two different types of MMIO being claimed by PCIe devices:
 Prefetchable MMIO (P‐MMIO) and
 Non‐Prefetchable MMIO (NP‐MMIO).
 Prefetchable space has two very well defined attributes:
 Reads do not have side effects
 Write merging is allowed
 Defining a region of MMIO as prefetchable allows the data in that region to be
speculatively fetched ahead in anticipation that a Requester might need more data in
the near future than was actually requested.
 The reason it’s safe to do this minor caching of the data is that reading the data doesn’t
change any state info at the target device. That is to say there are no side effects from
the act of reading the location.
 For example, if a Requester asks to read 128 bytes from an address, the Completer
might prefetch the next 128 bytes as well in an effort to improve performance by having
it on hand when it’s requested. However, if the Requester never asks for the extra data,
the Completer will eventually have to discard it to free up the buffer space. If the act of
reading the data changed the value at that address (or had some other side effect), it
would be impossible to recover the discarded data. However, for prefetchable space, the
read had no side effects, so it is always possible to go back and get it later since the
original data would still be there.
Non Prefetchable Memory Space

 You may be wondering what sort of memory space might have read
side effects? One example would be a memory‐mapped status
register that was designed to automatically clear itself when read to
save the programmer the extra step of explicitly clearing the bits
after reading the status.
BAR’s  Base Address Registers
 Each device in a system may have
different requirements in terms of the
amount and type of address space
needed.
 For example, one device may have 256
bytes worth of internal
registers/storage that should be
accessible through IO address space
and another device may have 16KB of
internal registers/storage that should
be accessible through MMIO.
 PCI‐based devices are not allowed to
decide on their own, which addresses
should be used to access their internal
locations, that is the job of system soft‐
ware (i.e. BIOS and OS kernel).
BAR’s
 So the devices must provide a way for system software to determine the
address space needs of the device. Once software knows what the device’s
requirements are in terms of address space, then assuming the request
can be fulfilled, software will simply allocate an available range of
addresses, of the appropriate type (IO, NP‐MMIO or P‐MMIO), to that
device.
 Type 0 header has six BARs available (each one being 32 bits in size),
while a Type 1 header has only two BARs. Type 1 headers are found in all
bridge devices, which means every switch port and root complex port has
a Type 1 header. Type 0 headers are in non‐bridge devices like endpoints.
 System software must first determine the size and type of address space
being requested by a device. The device designer knows the collective size
of the internal registers/storage that should be accessible via IO or MMIO.
 Knowing this information, the device designer hard‐codes the lower bits of
the BARs to certain values indicating the type and size of the address
space being requested
BAR’s
 The upper bits of the BARs are writable by
software. Once system software checks the
lower bits of the BARs to determine the size
and type of address space requested, system
software will then write the base address of
the address range being allocated to this
device into the upper bits of the BAR.
 Since a single Endpoint (Type 0 header) has
six BARs, up to six different address space
requests can be made. However, this is not
common in the real world. Most devices will
request 1‐3 different address ranges
 Not all BARs have to be implemented. If a
device does not need all the BARs to map
their internal registers, the extra BARs are
hard‐coded with all 0’s notifying software that
these BARs are not implemented
BAR Example 1: 32-bit Memory Address Space Request
BAR Example 2: 64-bit Memory Address Space Request
BAR Example 3: IO Address Space Request
Any Queries