Thread Priority

QNX DEVELOPERS General Programming
1

Hello,

My resource manager starts a thread to manage IRQ events only after it receives some type of message from clients.

First the RM starts
241691 1 ./devc_dataaq 30o RECEIVE 1
241691 2 ./devc_dataaq 30o RECEIVE 1
241691 3 ./devc_dataaq 30o RECEIVE 1

To start the thread :

pthread_attr_init (&attr); pthread_attr_setdetachstate( &attr, PTHREAD_CREATE_DETACHED); pthread_attr_setinheritsched( &attr, PTHREAD_EXPLICIT_SCHED); pthread_attr_setschedpolicy( &attr, SCHED_FIFO); attr.param.sched_priority = 60; pthread_create(&th_interr, &attr, th_irq, &msg);

Then, if the RM gets new messages and need to re-start the thread, I do:

if (th_interr != 0) ThreadDestroy(th_interr, 60, NULL);

The RM works fine for the first thread,

241691 1 ./devc_dataaq 30o RECEIVE 1
241691 2 ./devc_dataaq 30o RECEIVE 1
241691 3 ./devc_dataaq 50o RECEIVE 1
241691 4 ./devc_dataaq 60f INTR

the other process running have the same priority as the RM and the new thread have the requested priority, you can see a thread with priority 50, I dont know where this came from (thread pool - write requests?), and when I need to destroy the running thread and start a new thread, the priority is all mixed up (as show by pidin) :

241691 1 ./devc_dataaq 60o RECEIVE 1
241691 2 ./devc_dataaq 30o RECEIVE 1
241691 3 ./devc_dataaq 50o RECEIVE 1
241691 4 ./devc_dataaq 60o RECEIVE 1
241691 5 ./devc_dataaq 60f INTR

One more thing, Now I get 5 threads instead of only 4. I think I’m having problems with the ThreadDestroy…

Note - Thread pool :

memset(&pool_attr, 0, sizeof pool_attr); pool_attr.handle = dpp; pool_attr.context_alloc = resmgr_context_alloc; pool_attr.block_func = resmgr_block; pool_attr.handler_func = resmgr_handler; pool_attr.context_free = resmgr_context_free; pool_attr.lo_water = 2; pool_attr.hi_water = 4; pool_attr.increment = 1; pool_attr.maximum = 50;

So… :unamused:

2

A couple things. You are mixing thread pools with your own create / destroy calls. So from what you have posted, you should get a maximum of 4 pool threads, plus whatever you created.

I don’t understand why you are using the posix calls to create the thread, but not the posix calls to destroy it (or cancel it). For that matter if you need a thread to handle interrupt, why create and destroy the thread at all (Just create it and go into a loop blocking on InterruptWait() or whatever is appropriate for your needs)

So far I only understand the need to two threads - one for the resource manager, and one for the interrupt.

The reason you may see different priorities is the thread will “float” to the level of the sender and although it will drop back down to 30 on the next request it processes with a level of 30 or less, it appears in the RECEIVE blocked state as the last priority it ran at.

A couple comments to end with. :slight_smile:

Don’t use any more threads than you need (in a lot of cases, that means 1 thread). If you think you need a multiple threaded resource manager, get it working as a single threaded version first, then add the thread pool later if you need it. If you think you need it, make sure you understand how the attribute locking works - otherwise you will get single threaded behaviour with a thread pool - it will serialize on the attribute lock mutex.

Don’t mess with priorities unless you need to - and generally only change one thread/process in the equation and re-evaluate the requirement.

Hope this helps… :slight_smile:

Rick…

3

Plus, with a resmgr, all the client interfacing threads are going to float the client’s priority anyways. And you shouldn’t keep destroying the interrupt thread, just have it wait on a condvar.

4

To make it clear:

I choose to use thread pool and create threads for my own because:

0 - The RM is part of a high complex robotics control architecture! This particular one is the RM for a Data Aquisition board.

1 - The RM may receive write calls from different programs at the same time. I choose to use thread pool to manage this calls for me.

2 - I choose to create and destroy the IRQ thread because it depends on a 8254 timer to generate the interrupt and this timer is configured using information received via write call to RM. So, when a special msg arrives with information to calculate the 8254 timer I have to calculate a new period. I think is much easier and secure (I have to stop other process with condvars, reconfigure and then restart again) to destroy the IRQ thread, configure the 8254 and start a new thread. I know I can just disable the interrupt first, re-configure the timer and just enable it again. But, to keep it simple (specially for other programmers to understand the logic, this system is for robotics research) I decided to destroy the last thread and start a new thread.

I want to use the thread pool just to manage open,read,write,close calls and I want to manage my IRQ thread by myself. All RMs will work like this, and just 2 or 3 will have an IRQ thread.

About the posix calls. Well… I was testing wich calll I should use to work as I wanted. So, no special reason at all. :astonished:

I know i can manage the priority for different threads, so why this mess with thread pool? My RM should work with a priority, and each thread I create (using pthread_attr_t) may have different priorities.

5

If all the threads from the thread pools are dealing with the same hardware, it doesn’t really make sense because, only one thread can talk to the hardware at a time (i’m assuming). Effectevely all threads will get serialized waiting for the hardware, that makes multiple threads to some extend useless. Also “at the same time” is only true if you are using multi-processor system.

I don’t see the simplicity in destroying/creating a thread because of a timer change. By extension it sounds to me as if one will restarted the RM to enable disable a new feature in the hardware. Plus destroying/creating is expensive. You don’t need to disable all interrupt you can just mask the one use by 8254, that’s real time friendly :wink:

As Rick said unless your driver support multiple path (hence multiple attribute) you will run into attribute locking issue and won’t get any benefit.

About the posix calls. Well… I was testing wich calll I should use to work as I wanted. So, no special reason at all. :o

I know i can manage the priority for different threads, so why this mess with thread pool? My RM should work with a priority, and each thread I create (using pthread_attr_t) may have different priorities.
[/quote]

6

What was your question again? :slight_smile:

So assuming you have multiple pieces of hardware (so it is not serialised at the hardware level), do you have multiple attribute structs? They each contain a mutex which is locked on open and released on close. You can bypass this if you know what you are doing, but otherwise you are serialised again.

And in the end, unless you are running on an SMP machine, you are ultimatly serilised by the CPU.

My original point about not using a thread pool is that you keep the app as simple as possible with everything else is working - makes debugging easier.

As far as creating / destroying the thread, our recommendation is that you don’t do that - and you did ask for our opinions on this, right? :slight_smile:

7

I think my biggest problems is with thread pool.
Let me try to explain the work of one specific RM in my system.

What the RM will provide :
1 - Comunication with hardware (DABoard - data aquisition board). All I/O functions.
2 - Work mode.
3 - Status report.

Explaining :
1 - All out* and in* needed to configure the board, the out* to get the data converted (ADC [analogic digital converter chip]), etc.
2 - Depending on variables information, the board must be configured as needed.
3 - Reply to clients (most performance and analisis client [b]programs[b]) based on values in variables, and in* from the board.

To get a bigger picture :
1 - Start the RM.
1.1 - The board is configured to reset mode (nothing is done, no AD conversion, status=reset);

2 - A configuration client (program : config.c) send a message to RM (via write()) within information to configure the timer, channels, etc (typical functions to Data aquisition boards).
2.1 - The RM makes all out* and in* needed to configure the board, and the RM starts the ISR (interrupt service routine) to get the data converted (ADC) and store in a shared memory area(SMA).
2.2 - Another program (fft_a.c) is “looking” to this SMA and making the calculations (this is a outside program, not a option of the RM, but i could put some options in the RM like FFT analysis for instance (a thread))
2.3 - Another program asks for the status of the board.
2.4 - Another program asks for the status of the RM.

3 - Reply to each client who asked for any option provided by the RM.

What was my analisis to this :

1 - Create a RM for the board.
2 - For each client (remember, different programs) who asks for any option provided by the RM i wanted to keep a thread for each io_write() call so, if a client asks for a FFT analisis (when, or IF it is an option of the RM), the RM should reply the result of the computation, so, while it is making computation, the client is blocked until the data is ready. If the io_write() is blocked, when the client who is asking for a new freq analysis will get his OK?

3 - When needed, a status client asks for the RM status and DABoard status.

Note : When I wrote “at the same time” i know the basics of multiprocessing. I know what means “same time” in single processors.

But you can see, all programs must be independent. And the RM must be able to response to each one!

So, i decided to keep the io_write(), io_read(), etc; functions to be managed by the thread pool, so no needed to worry if a client is blocked or not. If it is blocked it must not affect the other one who is trying to ask for a different option. And the ISR managed by myself.

But, again, this is a specific problem with this specific RM. There’s a lot of other programs who may needed this kill feature. And imagine how important is this priority “thing” :slight_smile: for the full system. Imagine if the FFT thread has a higher priority than the DABoard thread (ISR) !

This problem is because of the thread pool and pthread_create() ? If i stop using thread pool this problem will not happened?

But, I wanted to know why is this such a problem. I should manage thread priorities as I wanted. This is crucial for the system.

Sorry about such long posts, but we got into a nice discussion! :slight_smile:

8

In typical system you will want to thread to run at the priority of the client who made the request. That is the default behavior of all threads/process under QNX6.

In the example of the FFT, if a program at priorty 5 ask for FFT analysis, and RM runs it at priority 20, what you end up with is a low priority process having the possibility to indirectly steal the CPU from higher priority process (6-19).

As the FFT example I would not included FFT stuff in driver that deals with the hardware. I’d keep that separate.

I don’t think you should mix POSIX and non POSIX function, pthread_create and ThreadDestroy()…

9

I’m confuse too! :smiley:

Ok. I have to write a RM for Data acquisition board.
The RM must configure the board using information received via io_funcs (io_write()). It must provide a status report sub-system (again using io_funcs) and configure on-demand the board work-mode.
The RM must put the information converted in a shared memory area for other programs to access.
The RM must provide options for mathematical computations such as FFT as an option.

So, if i decided not to use thread pools, I have to agree it will be more reliable. I will have more control, and more work to keep it sincronized (wich i would have anyway), but better to debug. Yes, i prefer this approach. :laughing:

But, about the priority situation…

Sorry, I didn’t understand what you mean!

Should I keep in mind that, if using thread pools and create threads (pthread_create()) the priority system (for the process) get all mess up? :frowning:

What you think was the problem?

10

By default every time a thread receive a message/pulse it will inherit the priority of the pulse or of the program that send the message. That is what we mean by floating priority.

Basicaly when you design the RM you don`t care about the priority of the thread as they will adapt to the priority of the sender. This is great because the priority is up the the program making the request.

If you want to have a fixed priority you have to specify it (thread attribute)

Let"s take the example of an ISR. If the ISR returns a proxy event, it doesn`t matter what priority the thread is at. Even if you set it to priority 2, if the pulse send by the ISR is of priority 63 the kernel will bump the priority of the thread that receives it.

11

Another way to look at this is the RM does work on behalf of the client - this is inherint in the whole send/receive/reply IPC mechanism used in QNX. Since the work is being done on behalf of the client, it should be done at the priority of the client.

Remember again that if you only have one attribute (/dev/daq), only one request will be processed at a time.

12

Even if I use thread pool? I thought when you use thread pool a thread will start every time the RM receives an io_fuc() call. So no problem on blocking because of one client, you can still receive new calls.

Thank you very much for your help! :slight_smile:

13

You have one attribute strucuture for every “file” you register with the attach() function. Because this resource is shared between all the threads, it contains a mutex so that which ever thread is updating the structure doesn’t walk all over each other.

By default, this attribute is lock (the mutex is acquired) when the file is opened, and released (unlocked) when the file is closed. You can work around this, but it is a good indication that you need to rethink your design.

If you have 3 channels on your daq board, then register 3 “files”. If you only have one channel, you need to rethink why you want multiple threads again.

So the answer is – even a thread pool with not alleiviate the design issues in using a resource manager. Some of these design issues are to help a resource manager provide posix compliant behaviour - this may not matter to you, but you need to be aware of the issues. Read the section in the helpviewer about Locking and unlocking the attribute structure. It is in the Writing a Resource Manager section.