Interrupt Response Speed

QNX DEVELOPERS Realtime and Embedded
1

My basic question is what is the slowest speed for which QNX guarantees (if at all) it will respond to an interrupt. Next, if a thread is blocked on an InterruptWait, how long after the interrupt is serviced will it unblock and run.

In my system I am seeing response times of QNX scheduling my ISR of close to 460us. I killed all processes that had an equal or greater priority of my interrupt thread (24). The thread takes about 160us from the ISR returning to finish what it needs to do (50us of which are real work talking to the hardware).

I am measuring these values on an oscilloscope and logic analyzer I have connected to the target hardware I’m talking to with QNX.

Now can I make it go faster? I know I could move the 50us of real work into the ISR, but then I am still limited by the 460us slowest reaction time of QNX to schedule my ISR. And I will still be held back by the servicing thread to calculate the next set of parameters I need if I do the real work in the ISR. I will not be running at a constant ridiculous rate. The problem I’m working on has a variable rate, and I need to push for as fast as possible to give the other guys I’m working with an answer to how fast they can run.

Now I will say that the majority of the time QNX responds within 5-10us to schedule my ISR, but I have to design to the worst case.

Thanks in advance for the advice!

2

Maybe you first need to look at the worst case hardware performance. What is your hardware platform?

3

460us is a lot of time, giving from your ISR to your thread it is only 110us.

Hardware platform certainly is a good information to have, I would also looking if the perticular interrupt vect is shared
by other devices. Run an instrument kernel and have a trace is also useful.

4

First of all, I’ve never seen any general guarantee’s from QNX because any numbers have to be tied to a specific piece of hardware. If you are seen good response times of 5us, but bad times of 460us, then something is wrong. Interrupts are almost never held up by scheduled programs/applications running. The only way they could be is if a program turned off interrupts. I don’t think I’m going out on a limb when I say that no QNX program would do this. This leaves the kernel, and other interrupt handlers getting in the way. There’s a pretty convincing description in the system architecture manual, that shows the kernel turning off interrupts only for very short periods of time, nothing like what you are seeing. If you need more convincing, the source is now available for inspection. This leaves the possibility that another interrupt is your problem. 460us doesn’t sound right for a QNX supplied driver. Is it possible you have written one that is causing a problem. Interrupt priority can be controlled to some degree by the controller chips. You can pick the highest interrupt and all the rest are sequential. A minor complication is that there are two of these chips, and the secondary 8-15 chip is channeled through interrupt 2 on the primary. The question came up recently about how to set this, and I don’t remember the answer, but you should find it here at openqnx.

Doing a trace is probably the best way to track this problem down.

There are two other hardware “features” that could be causing you problems, both fairly unlikely. A really bad piece of hardware could hang the bus, holding up everything. The other is that some integrated processor chips I’ve run into have a “supervisor” mode that takes precedence over everything. The OS is not even aware of it other than the chunks of lost time. This feature allows a manufacturer to emulate some piece of missing hardware in software. I’ve seen it used to do pixel stretching on a flat panel, and SB sound card emulation.

5

Thanks for the advice so far guys, I’ll try and get a trace today.

info about my platform:

Ampro Little Board 800 (intel 1.4 GHz)
Target hardware is a custom PC104 DSP board
I’m using IRQ9

If any more info would help let me know and I’ll try and get it.

6

Interrupt 9 is the cascade interrupt to the second 8259-ish PIC. Do you know how the hardware interrupt priorities are assigned on your board? What other interrupts are being used? On QNX4 you used to be able to change interrupt hardware priorities from the command line, not sure how that works in QNX6 but it’s probably done in the startup code.

7

I finally got back to this project and did a trace.

The interrupt should have 2 noticeable rates, 2ms and 21.5ms. there should be 24 2ms events followed by 8 21.5ms events.

I don’t know if I caught a long latency interrupt or not. If anyone has some tips on how to catch it, let me know. Also any tips on how to analyze the data (calculate the actual time between interrupt calls) would be helpful

8

Is this QNX4 or QNX6?

9

QNX 6.3.2A

10

Couldn’t find a long latency interrupt.

The file was rather small so you can probable sample much longer ( if the system as enough memory). Can you post your ISR thread code.

At what priority is the ISR Thread set at.

Try sampling with Wide mode.

With Eclipse finding the time bewteen interrupts is as simple as dragging the mouse over the events in the time line view.

11

I had to do such a short run because I couldn’t post anything longer on the forum (size was too big).

The ISR thread is at 24.

const struct sigevent* DspSrvrISR(void* arg, int id)
{
(UINT16)arg = in16(0x1b8);
return &interruptEvent;
}

void* interruptThread(void arg)
{
int interruptId;
void interruptArgs;
DspServer_c* server;

server = (DspServer_c*)(arg);

// interrupt pulse with which everything is polled
sigevent event;
event.sigev_notify = SIGEV_INTR;
event.sigev_priority = 24;
event.sigev_coid = coid;
event.sigev_code = 333;

interruptEvent = event;

// get special priveleges for talking to hardware
ThreadCtl(_NTO_TCTL_IO,NULL);

interruptArgs = (void*)(&readInterruptStatus);
interruptId = InterruptAttach(9, DspSrvrISR, interruptArgs, sizeof(UINT16), 0);

setprio(0, 24);

while(1)
{
    InterruptWait(NULL, NULL);

    server->interruptServicing();

    MsgSendPulse(coid, 20, PULSE_MESSAGE_ONLY, 0);
}

InterruptDetach(interruptId);

}

I tried that, but I couldn’t find a print of the time delta anywhere.

12

The number are on the top of the time line view (in the ruler).

There are other interrupts in your system (0,1 and 15 ) these might be influencing your interrupt. If you can catch one with the system profiler that should tell you.

13

run tracelogger in ring mode and put a TraceEvent(_NTO_TRACE_STOP) into your code when you detect the long latency. That way you will get a dump of the point in time preceding the
long latency.

You probably want a bigger ring that the default 32 buffers in this case, try -k 1024 if you have enough memory.

Colin

14

That’s an interesting idea on running tracelogger in ring mode, but the only way I know that the CPU is responding slowly is because I’m watching the response speed on an oscilloscope. I’m not too sure how I’d catch the event in software to stop the trace properly.

15

You can use rdtsc to get a fairly accurate sense of time. Use ClockCycles() to snap the time in the ISR, and then snap it again immediately after exiting out of the InterruptWait. That will give you a reason idea of the length of time you are waiting to schedule.

Colin

16

Hello Colin,

I have a question regarding your reply based on the information QNX provides in the online help.

If I read the safety information for ClockCycles in the helpfiles, it states that it is not safe to call it from an ISR. Why do you suggest calling it from an ISR? It it safe to call it from and ISR eventhough the safety information states that it is not safe practice or does it depend on what you do in the ISR or do you mean the thread that gets scheduled when returning an SIGEV_INTR instead of the ISR?

Regards,
Freddy

17

It depends on the processor model, on x86 you are safe because ClockCycles() becomes a single assembly instruction

18

Also my problem isn’t so much going from the ISR to the handler thread, the biggest delay is when the IRQ is asserted to when QNX runs the ISR. That is the biggest delay in the system, and also seems to be the most difficult to debug…

19

It is very likely that the problem is another interrupt interrupting your interrupt. Or you might be getting bitten by SMI. That’s an interrupt used by the BIOS to implement various feature like USB keyboard, fan control, etc. There is nothing the OS can do about that, it has to be disable in the BIOS.

I tried to find some doc about this but I think interrupt 0 has the highest priority, so interrupt 0 and 1 can be interrupting yours.

Apparently start-bios has an undocumented -I option to set the highest priority so you might try -I2 ( since 9 is connected to 2)

20

Interrupt 0, the timer would cause a pretty minimal delay, and would not occur very often. Think of the probability. small-#-microseconds/timer-frequency.

Interrupt 1 is the keyboard interrupt, it also would cause a minimal delay, and would only occur if the user is typing at the time.

So changing the interrupt priority to 2 is probably not going to help.
On the other hand, any interrupt on the 2ndary interrupt controller (8-15) cascades through interrupt 2. If the interrupt being monitored is on the 1st controller, (0-7) then changing the priority to 3 might help.