To perform programmed I/O on PCI devices on an SGI UV system, do the following to determine the resource filename (resourceN) and create an appropriate program to open the file and memory-map it:
Examine the output of the lspci(8) command to determine which device you want to map:
Record the domain, bus, slot, and function for the device (this information will help you locate the appropriate resource address file).
For example:
# lspci ... 0000:00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev 90) 0000:00:1f.0 ISA bridge: Intel Corporation 82801JIR (ICH10R) LPC Interface Controller 0000:00:1f.3 SMBus: Intel Corporation 82801JI (ICH10 Family) SMBus Controller 0000:01:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01) 0000:01:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01) 0000:04:00.0 SCSI storage controller: LSI Logic / Symbios Logic SAS1064ET PCI-Express Fusion-MPT SAS (rev 08) 0000:05:00.0 VGA compatible controller: Matrox Graphics, Inc. MGA G200e [Pilot] ServerEngines (SEP1) (rev 02) ...
The first field gives the information that is required to map the PCI registers into memory. The format is:
Domain:Bus:Slot.Function
In the above example, the highlighted output of 0000:01:00.1 for the Intel Corporation 82576 Gigabit Network card equates to domain 0, bus 1, slot 0, and function 1.
Determine the resource N numbers from the Region numbers in the lspci -vv output. The Region value corresponds directly to each resourceN value.
In the following example, the Region N output (highlighted) indicates that there are four resource N values (resource0, resource1, resource2 and resource3 ):
# lspci -n -s 0000:01:00.1 -vv 0000:01:00.1 0200: 8086:10c9 (rev 01) Subsystem: 10a9:8028 Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- SERR- <PERR- INTx- Latency: 0, Cache Line Size: 64 bytes Interrupt: pin B routed to IRQ 40 Region 0: Memory at b2140000 (32-bit, non-prefetchable) [size=128K] Region 1: Memory at b2120000 (32-bit, non-prefetchable) [size=128K] Region 2: I/O ports at 2000 [size=32] Region 3: Memory at b2240000 (32-bit, non-prefetchable) [size=16K] Expansion ROM at b2100000 [disabled] [size=128K] Capabilities: [40] Power Management version 3 Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+) Status: D0 PME-Enable- DSel=0 DScale=1 PME- ... ... Kernel driver in use: igb Kernel modules: igbA device can have both 32-bit and 64-bit base address registers (BARs). If a BAR is mapping a 64-bit address space, then two 32-bit BARs are used to map that 64-bit Region. As a result, Region numbers may not be consecutive. For example, in the following lspci output, there are three Region values (Region 0, Region 1 and Region 3):
# lspci -n -s 0000:04:00.0 -vv 0000:04:00.0 0100: 1000:0056 (rev 08) Subsystem: 1000:1000 Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0 Interrupt: pin A routed to IRQ 24 Region 0: I/O ports at 1000 [size=256] Region 1: Memory at b2010000 (64-bit, non-prefetchable) [size=16K] Region 3: Memory at b2000000 (64-bit, non-prefetchable) [size=64K] Expansion ROM at b1c00000 [disabled] [size=4M] Capabilities: [50] Power Management version 2 Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-) Status: D0 PME-Enable- DSel=0 DScale=0 PME- ... ... Kernel driver in use: mptsas Kernel modules: mptsasThere is no Region 2 because the card's second BAR is mapping a 64-bit region and thus uses two 32-bit BARs to do so. In this example, there would be three corresponding resource numbers ( resource0, resource1, and resource3 ) that would be used to memory-map the PCI registers.
Note: Only memory base-address registers (not I/O base-address registers) can be memory mapped. The base address must be page aligned.
Based on the information in step 1, determine the resource address file that you want to open:
/sys/bus/pci/devices/domain:bus:slot.function/resourceN
For the Intel example above, the resource address files are:
/sys/bus/pci/devices/0000:01:00.1/resource0 /sys/bus/pci/devices/0000:01:00.1/resource1 /sys/bus/pci/devices/0000:01:00.1/resource2 /sys/bus/pci/devices/0000:01:00.1/resource3
In the case of the LSI Logic® card example showing 64-bit Region values:
/sys/bus/pci/devices/0000:04:00.0/resource0 /sys/bus/pci/devices/0000:04:00.0/resource1 /sys/bus/pci/devices/0000:04:00.0/resource3
Create a program that opens the appropriate resource file for the domain, bus, slot, function, and resource in which you are interested. For example, the C program for the Intel card could include the following lines:
sprintf(path, "/sys/bus/pci/devices/%04x:%02x:%02x.%x/%s", (unsigned)domain, (unsigned)bus, (unsigned)slot, (unsigned)function, "resource0"); if ((fd = open(path, O_RDWR)) == -1) { perror("Couldn't open resource file"); exit(1); }
Add a line to the program that will memory-map the opened file from offset 0. For example, in C:
ptr = mmap( NULL, getpagesize(), PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
For details about kernel-level PCI device drivers, see the Linux Device Driver Programmer's Guide,Porting to SGI Altix Systems.