Porting PCI driver in linux to QNX

QNX DEVELOPERS Realtime and Embedded
1

Hi,

I am working on EVMDM6437 TI board plugged in the PCI slot of my QNX machine. Right now, using a driver I need to configure the registers in DSP through host(qnx).Like I am able to use mmap_device_memory() and get a handle to the shared memory. But the killing need is to like get a handle to the device 's register space.How can I achieve this?

Also I coulndt enable the I/O space of the PCI device even though I write the control word in to the command register.Please help me out.

Giving you the output of pci -vvv:

[i]Class          = Data Acquisition (Unknown)
Vendor ID      = 104ch, Texas Instruments
Device ID      = b001h, Unknown Unknown
PCI index      = 0h
Class Codes    = 118000h
Revision ID    = 1h
Bus number     = 1
Device number  = 1
Function num   = 0
Status Reg     = 200h
Command Reg    = 556h
        I/O space access disabled
        Memory space access enabled
        Bus Master enabled
        Special Cycle operations ignored
        Memory Write and Invalidate enabled
        Palette Snooping disabled
        Parity Checking disabled
        Data/Address stepping enabled
        SERR# driver disabled
        Fast back-to-back transactions to different agents enabled
Header type    = 0h Single-function
BIST           = 0h Build-in-self-test not supported
Latency Timer  = 20h
Cache Line Size= 8h un-cacheable
PCI Mem Address = d8000000h prefetchable 32bit length 8388608 enabled
PCI Mem Address = da400000h 32bit length 65536 enabled
PCI Mem Address = da000000h 32bit length 4194304 enabled
PCI Mem Address = d8800000h prefetchable 32bit length 8388608 enabled
PCI Mem Address = d9000000h prefetchable 32bit length 8388608 enabled
PCI Mem Address = d9800000h prefetchable 32bit length 8388608 enabled
Max Lat        = 0ns
Min Gnt        = 0ns
PCI Int Pin    = INT A
Interrupt line = 11
CPU Interrupt  = bh
Capabilities Pointer = 40h
Capability ID        = 1h
Capabilities         = 2h - 0h
Device Dependent Registers:
0x40: 01 00 02 00 00 00 00 00 00 00 00 00 00 00 00 00
0x50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0xA0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0xB0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0xC0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0xD0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0xE0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0xF0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00[/i][/code]


Also, it would greatly help me if someone could write to me the equivalent function for ioremap() in linux to QNX.

Thanks,
Rama.[code]
2

Don’t over-complicate things by thinking like a Linux-head.

Want to write to a 32bit register?

out32(registerAddress, registerValue);

Want to read from a 32bit register?

registerValue=in32(registerAddress);

At the top of your main() function, make sure to ask the O/S for I/O privity.

ThreadCtl(_NTO_TCTL_IO, 0);

and you’ll need to:

#include <hw/inout.h>

That’s it. Simple.

3

Indeed its that simple but would it not be more portable if he would use the mapping functions? The only reason (as far as I know of) linux has ioremap on x86 platforms for portability issues.

4

Hi,

The TI’s EVMDM6437 is plugged in an x86 machine only. But the device has no I/O space provision.So I mapped all the address space of the DSP to memory mapped one. Now I am able to reach the device’s register and memory space.

Do someone have tried to write ISR to the PCI device? I am walking thru the QNX programming guide for interrupt handlers.
Any pointers would help me much!

Thanks
Rama.

5

Gila, it is actually more portable if you use the in/out macros, because on QNX they expand to the appropriate memory accesses on other processor types.

For instance out32() expands to a memory address dereference followed by an “eieio” on PPC…

6

Rama,

In QNX one does not use interrupt handlers, but instead interrupt threads. We call them IST’s (Interrupt Service Threads).

Why use threads instead of handlers?

  1. QNX is a RTOS, and a RTOS requires minimal scheduling latency and interrupt handlers increase scheduling latency.

  2. QNX is a RTOS, and a RTOS requires precise control of scheduling. IST’s are subject to the identical scheduling controls as are available with any thread (priority, adaptive partition membership, etc.) whereas a interrupt handler is not.

The API you need to read about is InterruptAttachEvent(). This call allows you to associate a pulse (with a priority) to a particular IRQ line. You then have a thread call MsgReceive(), and wait for a pulse to arrive. When you receive the pulse, the kernel has masked the IRQ, and you can service the device - remove all interrupt sources and then call InterruptUnmask().

Because this thread is a completely conventional thread, the interrupt processing can now be pre-empted by any thread with a higher priority (not just another interrupt).

For instance CardBus controllers have an insertion/removal interrupt. Typically an applications processing latency for an insertion/remove event is not a terribly tight constraint, so it might make sense to attach a priority 9 (1 level below normal ksh prompt priority) pulse to the IRQ for this. This means that ksh processing would pre-empt the insertion/removal interrupt processing. This level of control is not available on an operating system that does not support IST’s (i.e. Linux).

7

Hi,

I am not experienced to QNX and I feel panic now.Actually I want to stream my encoded data from my host hard disk to DSP board on the PCI slot. Assume I pass buffers of 33K everytime to DSP.I need to generate interrupt on every data shot to DSP and DSP should acknowledge the interrupt by updating a message.

When walking thru the QNX Neutrino Programming guide, I got a reference to use ISR. I thought I can use InterruptAttach, InterruptWait functions to generate PCI interrupt to the DSP. Will not work? Since you are suggesting to use the threads for interrupt, How can I start with? Any notes and help would greatly relieve me.

Thanking you,
Rama.

8

You cannot “generate” a PCI interrupt. It’s the hardware that generated the interrupt to the CPU not the other way around.

If you need to generate a interrupt to the DSP that is probably done through writing in some register on the card.

InterruptWait/InterruptAttach/InterruptAttachEvent are used when a device generated an interrupt to the CPU.

9

As Mario said, you generally enable the interrupt by setting a special register on your hardware.

The hardware will then fire the interrupt ever n usec or as soon as there is space in the buffer (or whenever it is planned to do so)
Your ISR then gets startet as soon as the interupt is fired, where you write your packets into the cards memory (or IO).

This loops until you disable the interrupt by unsetting (or setting different value) in the register of the hardwar.

10

Hi,
Sorry for not updating asap.
I am still to get my interrupt stuff! i could nt get interrupt from my PCI DSP board to QNX host.
Here goes my code.

[code]struct pci_dev_info inf;
main()
{
ThreadCtl(_NTO_TCTL_IO,0);
event.sigev_notify = SIGEV_INTR;
IntrId = InterruptAttach(inf.Irq,ISR_handler,NULL,0,_NTO_INTR_FLAGS_TRK_MSK);
while(1)
{
/* Sleep until the next interrupt /
InterruptWait( 0, NULL );
printf(“Interrupt \n”);
/ Reenable this interrupt */
InterruptUnmask( inf.Irq, IntrId );
}

InterruptDetach( IntrId );

}
//Here is my ISR handler;checks and clear the source of interrupt

const struct sigevent *ISR_handler(void * arg,int id)
{
Uint32 status;
printf("\n<<< In ISR routine >>>\n");
InterruptMask(inf.Irq,IntrId);
//look at the hardware to check whether interrupt is generated
status = HAL_CheckPciInterrupt ();
if (status == 1) {
HAL_PciClearDspInterrupt () ;
printf (“Interrupt received from DSP\n”) ;
}

return (&event);

}[/code]
Now the problem is that the IRQ number 11 is used by network controller,USB controller and my PCi DSP board.When I single step thru the code,
the program crashes in the sense the QNX machine hangs!Where am I commiting mistakes?

Thanks,
Rama.[code]

[/code]

11

You cannot call printf from inside an ISR. Check the documentation, for each function there is a table that specify if an function is ISR safe, thread safe and cancel point safe…

You also cannot put a break point in an ISR.

You might want to look at InterruptAttachEvent instead…

12

Thanks for reply.

My doubt is that, for every physical interrupt, we will be having an event number associated with it.It was that what I did in my Code composer studio.The physical interrupt for PCI DSP was mapped to a system event number.But in QNX I dont see any things like that.What I get from the structure pci_dev_info for my DSP board is IRQ11.But this number is for someother devices also. How can I uniquely map the IRQ11 to my PCI device?

As you suggested I have commented the printf statements.Even then no results.

Given below is the code with InterruptAttachEvent()

[code]struct pci_dev_info inf;
const struct sigevent event;
main()
{

ThreadCtl(_NTO_TCTL_IO,0); //request I/O privileage
pthread_create(NULL, NULL, PCI_isrThread, NULL); //thread for interrupt handling

}

Uint32 * PCI_isrThread(void *arg)
{

event.sigev_notify = SIGEV_INTR;

IntrId = InterruptAttachEvent( inf.Irq, &event, _NTO_INTR_FLAGS_TRK_MSK );

while (1)
{
/* Sleep until the next interrupt */
InterruptWait( 0, NULL );

	printf("<<Interrupt>>\n");
	
	/* Reenable this interrupt */
	InterruptUnmask( inf.Irq, IntrId );
	
}

// Detach interrupt
if( InterruptDetach( IntrId ) == -1 )
{
printf("%s: Interrupt detach failed <%s>", FUNCTION, strerror(errno));
}
}
[/code]

Please help me.

Thanks
Rama.

13

It’s the BIOS that assign the IRQ. You may be able to move it though BIOS option or by changing slots. That being said, it doesn’t really matter. PCI interrupt can be shared. The first thing your ISR should do is check the source of the interrupt. That’s usually done though some registers on the hardware. If the hardware indicates no interrupt was generated then the ISR simply return.

It seems to me you are rather a novice at this. You provide too little information about what you are trying to do and how you want to achieve it. Terms like Composer Studio and System Event Number don’t mean anything in the QNX world, hence it’s making it very hard to help you.

By the way in your ISR there is nothing that clears the interrupt on the hardware. When a hardware generate and interrupt that interrupt stays active until the processors confirm (clears) it has processed the interrupt. I suggest you get a book on real time / interrrupt programming.

14

Hi Master,

My working environment consists of

Machine 1:
Windows XP with Code composer studio 3.3 for debugging the DSP application

Machine 2:
QNX neutrino in which the DSP board is plugged in the PCI slot.

Its like I need to stream the encoded data from the QNX host to DSP slave in the PCI slot of the same machine. Since I am transferring the data in smaller packets i.e the data width of the PCI is 64KB.I need to transfer MegaBytes of data with smooth communication.For this I need to implement interrupts on both host and DSP slave. My host could interrupt the DSP thru PCI.But the DSP to host interrupt is giving me issues. I am a learner in QNX neutrino. My interrupt handling on the host side leads to crash…meaning as soon as the I write into DSP configuration register for triggering the interrupt, the QNX machine hangs! No response.I need to restart.

[code]struct pci_dev_info inf;
main()
{

ThreadCtl(_NTO_TCTL_IO,0); //request I/O privileage
pthread_create(NULL, NULL, PCI_isrThread, NULL); //thread for interrupt handling

}

Uint32 * PCI_isrThread(void *arg)
{

memset(&event,0,sizeof(event));
event.sigev_notify = SIGEV_INTR;

IntrId = InterruptAttachEvent( inf.Irq, &event, _NTO_INTR_FLAGS_TRK_MSK );

while (1)
{
/* Sleep until the next interrupt */
InterruptWait( 0, NULL );

  //Checking the source of the interrupt and 
  // clearing the interrupt
  
  /* Reenable this interrupt */
  InterruptUnmask( inf.Irq, IntrId );

}
// Detach interrupt
if( InterruptDetach( IntrId ) == -1 )
{
printf("%s: Interrupt detach failed <%s>", FUNCTION, strerror(errno));
}
} [/code]

As soon as single step the InterruptAttachEvent(),QNX machine hangs! Could spot the reason.

Thanks
Rama.

15

Where is the code that check and clear the interrupt? There is a comment but not code. This is probably where the problem is.

16

Here is the code for checking and clearing the interrupt.

void * PCI_isrThread(void *arg)
{
...
//i check the interrupt status here
volatile DM6437PCI_pciRegs * pciRegs = (DM6437PCI_pciRegs *) ((Uint32) regVirt + (DM6437PCI_PCIREG_BASE - 0x01C00000)) ;
    
Uint32                 status = 0 ;

if ( (pciRegs->PCICSRMIR & DM6437PCI_INTSTATUS_MASK) == (DM6437PCI_INTSTATUS_MASK)) 
{
	status = 1 ;
}

if ( (pciRegs->PCIHINTCLR & DM6437PCI_SOFTINT0_MASK ) == (DM6437PCI_SOFTINT0_MASK)) 
{
   	statusFlag = 1; //Interrupt from DSP
   
}
 pciRegs->PCIHINTCLR |= DM6437PCI_SOFTINT0_MASK ; //clear the interrupt

...
}

Thanks,
Rama

17

Where does regVirt comes from. What is the value of DM6437PCI_PCIREG_BASE.

18

Master,
Attached here is the source code for my pci driver.Hope this will help you.

regVirt is the handle for accessing the memory mapped registers of the DSP for PCI module.

Thanks
Rama.

19

I don’t see anything wrong aside the fact that you do check for PciInterrupt but clear the DspInterrupt and Unmask the interrupt anyway.

20

Does he loop checking for new sources and clearing until all sources are removed (must do this).