pci latency/interrupt delay on HP Z820 / Intel C602 chipset

Has anyone had any experience running QNX 6.5.0 on HP’s Z820 machines?

we’ve been trying to track down a periodic pci access latency of ~100us.

hardware:
HP z820
Matrox graphics card
Symmetronic PCI bc635 time card
axxon 2 serial / 1 parallel port combo card
Contec 32 Input /32 output digital I/O card
1 sata drive with bios configured for either ide compatible / ahci modes

test cases:

  • read to graphics card IO space
  • read time card time register
  • read/write parallel port combo card

i’ve stripped the system down to the bare essentials for testing
when running a 1620 hz function generator into the Digital card i’ve nominal seen periodic delays of up to ~100us in receiving the interrupt

  • tried setting the IRQ for the highest priority
  • running the thread at prio > 250

doing a simple read access of the time card at a priority of 255 still gets pre-empted up to ~100us during a read.

using ClockCycles to do a majority of the timing measurements
nominally the read takes ~3us, but can be up to ~100us during the delay periods when pci bus access is delayed.

Make sure SMI is disable (if the bios allows it). Also make sure USB emulation is disabled ( it’s often implemented via SMI).

i’ve gone as far in the bios as disabling all usb ports and slaying io-usb enum-usb, etc. with no effect.

i’m suspecting its something that’s not being configured properly by qnx on these newer z820 machines. the disk image/application seems to work fine on older chipsets / machines. Although one test case we’ve used is running the same image on a dell pc with the same chipset and we did not see the same delays.

Product Name
HP Z820 Workstation
Processor Type
Intel(R) Xeon(R) CPU E5-2630 @ 2.30GHz
Processor Speed
2300 MHz
Processor Stepping
000206D7 00000710
Cache Size (L1/L2/L3)
64KBx6 / 256KBx6 / 15360KBx1
Memory Size
32 GB ECC RDDR3 / 1333 MHz
Processor Type
Intel(R) Xeon(R) CPU E5-2630 @ 2.30GHz
Processor Speed
2300 MHz
Processor Stepping
000206D7 00000710
Cache Size (L1/L2/L3)
64KBx6 / 256KBx6 / 15360KBx1
System BIOS
J63 v03.50
Boot Block Date
12/28/2011
Chassis Serial Number
2UA2271B20
Asset Tracking Number
2UA2271B20
Family Name
103C_53335X G=D
ME Firmware Version
0.0.0.0
ME Management Mode
NONE
Removable Media Boot
*Enable
Disable
SATA Emulation
IDE

RAID+AHCI
*AHCI
EFI Boot Order
USB Floppy/CD Disabled
Hard Drive Disabled
ATAPI CD-ROM Drive
EFI HDD Order
USB Hard Drive Disabled
Legacy Boot Order
USB Floppy/CD Disabled
Hard Drive
ATAPI CD-ROM Drive
NIC Controller (AMT)
PnP Device #1
PnP Device #2
PnP Device #3
PnP Device #4
PnP Device #5
PnP Device #6
PnP Device #7
PnP Device #8
PnP Device #9
PnP Device #10
PnP Device #11
Legacy HDD Order
USB Hard Drive
AHCI HDD1
Lock Legacy Resources
Disable
*Enable
Network Server Mode
*Disable
Enable
Setup Browse Mode
Disable
*Enable
Stringent Password
*Disable
Enable
Password prompt on F9 F11 & F12
*Enable
Disable
Cover Lock
*Unlock
Lock
Cover Removal Sensor
*Disable
Notify User

Setup Password
Front USB Ports
Disable
*Enable
Front USB Port 1
Disable
*Enable
Rear USB Ports
Disable
*Enable
Rear USB Port 1
Disable
*Enable
Rear USB Port 2
Disable
*Enable
Rear USB Port 3
Disable
*Enable
Rear USB Port 4
Disable
*Enable
Internal USB Ports
*Disable
Enable
Internal USB Port 1
Disable
*Enable
Internal USB Port 2
Disable
*Enable
Internal USB Port 3
Disable
*Enable
Internal USB Port 4
Disable
*Enable
Internal USB Port 5
Disable
*Enable
Internal USB Port 6
Disable
*Enable
Slot 1 - PCI Express x4
Disable
*Enable
Slot 2 - PCI Express x16
Disable
*Enable
Slot 3 - PCI Express x8
Disable
*Enable
Slot 4 - PCI Express x16
Disable
*Enable
Slot 5 - PCI Express x4
Disable
*Enable
Slot 6 - PCI Express x16
Disable
*Enable
Slot 7 - PCI
Disable
*Enable
Azalia HD Audio
*Device hidden
Device available
1394 Controller
Device available
*Device hidden
NIC Controller (AMT)
Device hidden
*Device available
NIC Controller
Device hidden
*Device available
SCU Controller
*Device available
Device hidden
USB3 Controller
Device available
*Device hidden
Embedded Security Device
*Device hidden
Device available
SATA0
Device hidden
*Device available
SATA1
Device hidden
*Device available
Serial Port A
Device hidden
*Device available
Network Boot
*Disable
Enable
Enter Ownership Tag

Enter UUID
D05F7580-C027-11E1-AB20-2C41384F2565
Enter Feature Byte
3Q3X3Z3m47676Q6b7N7Q7U7W7a8H8h8j9Y.A3
Enter Build ID
11WWKYBW601#SABA#DABA
Data Execution Prevention
*Disable
Enable
Virtualization Technology (VTx)
*Disable
Enable
Intel(R) VT-d
*Disable
Enable
Coherency Support
*Disable
Enable
ATS Support
Disable
*Enable
Intel TXT(LT) Support
*Disable
Enable
Activate Embedded Security On Next Boot
*Disable
Enable
OS management of Embedded Security Device
*Enable
Disable
Reset of Embedded Security Device through OS
*Disable
Enable
Legacy Support
Disable
*Enable
Secure Boot
*Disable
Enable
Clear Secure Boot Keys
*Don’t Clear
Clear
Key Ownership
*HP Keys
Custom Keys
Fast Boot
*Disable
Enable
Runtime Power Management
Disable
*Enable
Idle Power Savings
Normal
*Extended
Turbo Mode
*Disable
Enable
Unique Sleep State Blink Rates
*Disable
Enable
S5 Maximum Power Savings
*Disable
Enable
Fan Idle Mode
*------
±----
+±—
++±–
+++±-
++++±
++++++
POST Messages
*Disable
Enable
Press ESC for Startup Menu Prompt
*Enable
Disable
System Recovery Boot Support
*Disable
Enable
Option ROM Prompt
*Enable
Disable
Mini Option ROM Display
*Enable
Disable
Remote Wakeup Boot Source
Remote Server
*Local Hard Drive
After Power Loss
*Off
On
Previous State
POST Delay (in seconds)
*None
5
10
15
20
Multi-Processor
*Enable
Disable
Active Processor Cores
*All
1
2
Hyper-Threading
*Disable
Enable
Bypass F1 Prompt on Configuration Changes
*Disable
Enable
Sunday
*Disable
Enable
Monday
*Disable
Enable
Tuesday
*Disable
Enable
Wednesday
*Disable
Enable
Thursday
*Disable
Enable
Friday
*Disable
Enable
Saturday
*Disable
Enable
BIOS Power-On Time (hh:mm)
00:00
NUMA
*Disable
Enable
PCI SERR# Generation
*Disable
Enable
PCI VGA Palette Snooping
*Disable
Enable
PCI Latency Timer
32 PCI Clocks
64 PCI Clocks
96 PCI Clocks
*128 PCI Clocks
160 PCI Clocks
192 PCI Clocks
224 PCI Clocks
248 PCI Clocks
CECP Mode
*Disable
Enable
PCIe Performance Mode
Disable
*Enable
Num Lock State at Power-On
Off
*On
S5 Wake on LAN
Enable
*Disable
Internal Speaker
*Disable
Enable
NIC Option ROM Download
Enable
*Disable
NIC1 Option ROM Download
Enable
*Disable
Slot 1 Option ROM Download
Disable
*Enable
Slot 2 Option ROM Download
Disable
*Enable
Slot 3 Option ROM Download
Disable
*Enable
Slot 4 Option ROM Download
Disable
*Enable
Slot 5 Option ROM Download
Disable
*Enable
Slot 6 Option ROM Download
Disable
*Enable
Slot 7 Option ROM Download
Disable
*Enable
SAS Option ROM Download
Enable
Disable
Serial Port A Settings
*Disabled
IO=3F8; IRQ=4
IO=3F8; IRQ=3
IO=2F8; IRQ=4
IO=2F8; IRQ=3
AMT
Disable
*Enable
Unconfigure AMT/ME
Disable
*Enable
Bypass Unconfigure AMT/ME Prompt
*Disable
Enable
WatchDog Timer
*Disable
Enable
OS WatchDog Timer
5
BIOS WatchDog Timer
5
Tpm PPI policy changed by OS allowed
*Disable
Enable
Tpm measure boot variables/devices to PCR1
*Disable
Enable
Tpm No PPI provisioning
*Disable
Enable
Tpm No PPI maintenance
*Disable
Enable
Power-On Password 32
c8d7d0ef0eedfa82d2ea1aa592845b9a6d4b02b7
Setup Password 32
c8d7d0ef0eedfa82d2ea1aa592845b9a6d4b02b7
SAS Controller
Device available
*Device hidden
VGA Configuration
*Slot 1 Matrox
Headless Mode
*Disable
Enable
Legacy ACPI CPU Tables
*Disable
Enable
PXE Option ROMS

EFI
*Legacy
Mass Storage Option ROMS

EFI
*Legacy
Video Option ROMS

EFI
*Legacy
SSC Alignment
*Enable
Disable
Slot 1 Limit PCIe Speed
*Auto
8 Gbs Gen3
5 Gbs Gen2
2.5 Gbs Gen1
Slot 2 Limit PCIe Speed
*Auto
8 Gbs Gen3
5 Gbs Gen2
2.5 Gbs Gen1
Slot 3 Limit PCIe Speed
*Auto
8 Gbs Gen3
5 Gbs Gen2
2.5 Gbs Gen1
Slot 4 Limit PCIe Speed
*Auto
8 Gbs Gen3
5 Gbs Gen2
2.5 Gbs Gen1
Slot 5 Limit PCIe Speed
*Auto
8 Gbs Gen3
5 Gbs Gen2
2.5 Gbs Gen1
Slot 6 Limit PCIe Speed
*Auto
8 Gbs Gen3
5 Gbs Gen2
2.5 Gbs Gen1
QPI Performance Counters
*Disable
Enable
Remote Optimized Performance
*Disable
Enable

What Mario said. The point is not slaying QNX processes, it’s -probably- about the BIOS interfering, either via SMI or USB/PS2 emulation. The OS cannot do anything about it.

I agree…what i’ve been trying to do is read the pci/intel chipset configuration registers to see if there is something that the bios is not doing or setting properly. i haven’t seen anything yet in the bios that would allow me to disable legacy usb emulation. the only thing i’ve been able to do is disable all the usb ports in the bios to no effect. i’ve been looking at the intel c602 chipset specification Table 5-29 which shows all the causes of SMI on the chipset and trying to track down which registers disable them.

finally tracked it down…

the HP z820 bios did not seem like it had an explicit way of disabling legacy usb2.0 SMI so i did it by mapping into the PMBASE register + offset for the SMI_EN control/status register (offset 0x30)

the legacy usb2.0 and software SMI timers were enabled in the register. disabling seemed to have resolved the issue.

pci_reg = lpc_if;
index = get_index(pci_reg, “PMBASE”);
pci_read_config32(info.BusNumber, info.DevFunc, pci_reg[index].offset, 1, (char*)&pmbase);
pmbase = pmbase & 0x0000ff80;
base = mmap(NULL,128,PROT_READ | PROT_WRITE, MAP_SHARED, NOFD, pmbase);
if(base == MAP_FAILED)
{
fprintf(stderr, “Error mapping to pmbase %s\n\n”, strerror(errno));
}
smi_en_reg = in32(pmbase + PMBASE_SMI_EN_OFF);
output_str = hex2bin(smi_en_reg, 32);
fprintf(stdout, “PMBASE: 0x%X SMIOFF: 0x%X SMI_EN_REG: 0x%X | %s\n”, pmbase, PMBASE_SMI_EN_OFF, smi_en_reg, output_str.c_str());
smi_en_reg = (0x01);

			out32(pmbase + PMBASE_SMI_EN_OFF, smi_en_reg);
			smi_en_reg = in32(pmbase + PMBASE_SMI_EN_OFF);
			output_str = hex2bin(smi_en_reg, 32);
			fprintf(stdout, "Updated PMBASE: 0x%X  SMIOFF: 0x%X SMI_EN_REG: 0x%X | %s\n", pmbase, PMBASE_SMI_EN_OFF, smi_en_reg, output_str.c_str());

output from this snippet:

Attach Vendor 0x8086 - Device 0x1d41 - Index 0
Base 0 - PCI Address 0x0 - CPU Address 0x0 - Size 0x0
Base 1 - PCI Address 0x0 - CPU Address 0x0 - Size 0x0
Base 2 - PCI Address 0x0 - CPU Address 0x0 - Size 0x0
Base 3 - PCI Address 0x0 - CPU Address 0x0 - Size 0x0
Base 4 - PCI Address 0x0 - CPU Address 0x0 - Size 0x0
Base 5 - PCI Address 0x0 - CPU Address 0x0 - Size 0x0
Revision 0x5

*** SMI Control/status REGISTER ****
PMBASE: 0x400 SMIOFF: 0x30 SMI_EN_REG: 0x8020063 | 0000 1000 0000 0010 0000 0000 0110 0011
Updated PMBASE: 0x400 SMIOFF: 0x30 SMI_EN_REG: 0x8000003 | 0000 1000 0000 0000 0000 0000 0000 0011

BusNumber 0
DeviceNumber 31
FunctionNumber 0
OFFSET: 0x0 VID : 0x 8086 | 1000 0000 1000 0110
OFFSET: 0x2 DID : 0x 1D41 | 0001 1101 0100 0001
OFFSET: 0x4 PCICMD : 0x 7 | 0000 0000 0000 0111
OFFSET: 0x6 PSTS : 0x 210 | 0000 0010 0001 0000
OFFSET: 0x8 RID : 0x 5 | 0000 0101
OFFSET: 0x9 PI : 0x 0 | 0000 0000
OFFSET: 0xA SCC : 0x 1 | 0000 0001
OFFSET: 0xB BCC : 0x 6 | 0000 0110
OFFSET: 0xD PMLT : 0x 0 | 0000 0000
OFFSET: 0xE HEADTYP : 0x 80 | 1000 0000
OFFSET: 0x2C SS : 0x 103C | 0001 0000 0011 1100
OFFSET: 0x34 CAPP : 0x E0 | 1110 0000
OFFSET: 0x40 PMBASE : 0x 401 | 0000 0000 0000 0000 0000 0100 0000 0001
OFFSET: 0x44 ACPI_CNTL : 0x 80 | 1000 0000
OFFSET: 0x48 GPIOBASE : 0x 501 | 0000 0000 0000 0000 0000 0101 0000 0001
OFFSET: 0x4C GC : 0x 11 | 0001 0001
OFFSET: 0x60 PIRQ[A]_ROUT : 0x B | 0000 1011
OFFSET: 0x61 PIRQ[B]_ROUT : 0x 5 | 0000 0101
OFFSET: 0x62 PIRQ[C]_ROUT : 0x 4 | 0000 0100
OFFSET: 0x63 PIRQ[D]_ROUT : 0x 3 | 0000 0011
OFFSET: 0x64 SIRQ_CNTL : 0x D0 | 1101 0000
OFFSET: 0x68 PIRQ[E] ROUT : 0x 7 | 0000 0111
OFFSET: 0x69 PIRQ[F] ROUT : 0x 0 | 0000 0000
OFFSET: 0x6A PIRQ[G] ROUT : 0x 0 | 0000 0000
OFFSET: 0x6B PIRQ[H] ROUT : 0x A | 0000 1010
OFFSET: 0x6C LPC_IBDF : 0x F0F8 | 1111 0000 1111 1000
OFFSET: 0x70 LPC_H0BDF : 0x F078 | 1111 0000 0111 1000
OFFSET: 0x72 LPC_H1BDF : 0x F078 | 1111 0000 0111 1000
OFFSET: 0x74 LPC_H2BDF : 0x F078 | 1111 0000 0111 1000
OFFSET: 0x76 LPC_H3BDF : 0x F078 | 1111 0000 0111 1000
OFFSET: 0x78 LPC_H4BDF : 0x F078 | 1111 0000 0111 1000
OFFSET: 0x7A LPC_H5BDF : 0x F078 | 1111 0000 0111 1000
OFFSET: 0x7C LPC_H6BDF : 0x F078 | 1111 0000 0111 1000
OFFSET: 0x7E LPC_H7BDF : 0x F078 | 1111 0000 0111 1000
OFFSET: 0x80 LPC_I/O_DEC : 0x 10 | 0001 0000
OFFSET: 0x82 LPC_EN : 0x 3F0E | 0011 1111 0000 1110
OFFSET: 0x84 GEN1_DEC : 0x FC0601 | 0000 0000 1111 1100 0000 0110 0000 0001
OFFSET: 0x88 GEN2_DEC : 0x FC0801 | 0000 0000 1111 1100 0000 1000 0000 0001
OFFSET: 0x8C GEN3_DEC : 0x 0 | 0000 0000 0000 0000 0000 0000 0000 0000
OFFSET: 0x90 GEN4_DEC : 0x 0 | 0000 0000 0000 0000 0000 0000 0000 0000
OFFSET: 0x94 ULKMC : 0x F00 | 0000 0000 0000 0000 0000 1111 0000 0000
OFFSET: 0x98 LGMR : 0x 0 | 0000 0000 0000 0000 0000 0000 0000 0000
OFFSET: 0xA0 GEN_PMCON_1 : 0x A18 | 0000 1010 0001 1000
OFFSET: 0xA2 GEN_PMCON_2 : 0x 20 | 0010 0000
OFFSET: 0xA4 GEN_PMCON_3 : 0x 1C49 | 0001 1100 0100 1001
OFFSET: 0xA6 GEN_PMCON_LOCK : 0x 6 | 0000 0110
OFFSET: 0xA9 CIR4 : 0x 47 | 0100 0111
OFFSET: 0xAA BM_BREAK_EN_2 : 0x 0 | 0000 0000
OFFSET: 0xAB BM_BREAK_EN : 0x 0 | 0000 0000
OFFSET: 0xB8 GPI_ROUT : 0x 11000 | 0000 0000 0000 0001 0001 0000 0000 0000
OFFSET: 0xD0 BIOS_SEL1 : 0x 112233 | 0000 0000 0001 0001 0010 0010 0011 0011
OFFSET: 0xD4 BIOS_SEL2 : 0x 4567 | 0100 0101 0110 0111
OFFSET: 0xD8 BIOS_DEC_EN1 : 0x FFCF | 1111 1111 1100 1111
OFFSET: 0xDC BIOS_CNTL : 0x 2A | 0010 1010
OFFSET: 0xE0 FDCAP : 0x 9 | 0000 0000 0000 1001
OFFSET: 0xE2 FDLEN : 0x C | 0000 1100
OFFSET: 0xE3 FDVER : 0x 10 | 0001 0000
OFFSET: 0xE4 FDVECIDX : 0x 0 | 0000 0000 0000 0000 0000 0000 0000 0000
OFFSET: 0xE8 FVECD : 0x C640290 | 0000 1100 0110 0100 0000 0010 1001 0000
OFFSET: 0xF0 RCBA : 0xFED1C001 | 1111 1110 1101 0001 1100 0000 0000 0001

Enter Commands(x to exit or NAME HEXVALUE:

Way to go man! :slight_smile: Thumbs up.