Hello,
I am facing a conundrum of sorts. I want to run the instrumented
micro-kernel and, of course, tracelogger because one of our software
engineers has decided on an ad-hoc basis to send pulses to a process at
priority 24. I noticed by default that the tracelogger runs at 15, 16 and
17, respectively. If I run the tracelogger will most likely be pre-empted by
the priority 24 process most of the time and hence not able to collect data
and will color the data I collect. Of course, I could raise tracelogger to
24. Any ideas?
Regards, Bill Halchin
“William Halchin” <bhalchin@syrres.com> wrote in message
news:d30spm$l54$1@inn.qnx.com…
Hello,
I am facing a conundrum of sorts. I want to run the instrumented
micro-kernel and, of course, tracelogger because one of our software
engineers has decided on an ad-hoc basis to send pulses to a process at
priority 24. I noticed by default that the tracelogger runs at 15, 16 and
17, respectively. If I run the tracelogger will most likely be pre-empted
by the priority 24 process most of the time and hence not able to collect
data and will color the data I collect. Of course, I could raise
tracelogger to 24. Any ideas?
The data collection is not done by tracelogger, it’s the kernel that does
it. Tracelogger gets the data generated by the instrumented kernel and save
it somewhere.
If you think tracelogger will not have enough CPU to empty the kernel buffer
fast enough then raise it’s priority. Yes it will affect lower priority
process, that’s the nature of the beast. Unless you have more then one CPU
Regards, Bill Halchin
Thanks for the reminder … I had a senior moment concerning who collects
data …
Regards, Bill
“Mario Charest” postmaster@127.0.0.1 wrote in message
news:d30tg8$lik$1@inn.qnx.com…
“William Halchin” <> bhalchin@syrres.com> > wrote in message
news:d30spm$l54$> 1@inn.qnx.com> …
Hello,
I am facing a conundrum of sorts. I want to run the instrumented
micro-kernel and, of course, tracelogger because one of our software
engineers has decided on an ad-hoc basis to send pulses to a process at
priority 24. I noticed by default that the tracelogger runs at 15, 16 and
17, respectively. If I run the tracelogger will most likely be pre-empted
by the priority 24 process most of the time and hence not able to collect
data and will color the data I collect. Of course, I could raise
tracelogger to 24. Any ideas?
The data collection is not done by tracelogger, it’s the kernel that does
it. Tracelogger gets the data generated by the instrumented kernel and
save it somewhere.
If you think tracelogger will not have enough CPU to empty the kernel
buffer fast enough then raise it’s priority. Yes it will affect lower
priority process, that’s the nature of the beast. Unless you have more
then one CPU >
Regards, Bill Halchin
Mario Charest wrote:
I am facing a conundrum of sorts. I want to run the instrumented
micro-kernel and, of course, tracelogger because one of our software
engineers has decided on an ad-hoc basis to send pulses to a process at
priority 24. I noticed by default that the tracelogger runs at 15, 16 and
17, respectively. If I run the tracelogger will most likely be pre-empted
by the priority 24 process most of the time and hence not able to collect
data and will color the data I collect. Of course, I could raise
tracelogger to 24. Any ideas?
The data collection is not done by tracelogger, it’s the kernel that does
it. Tracelogger gets the data generated by the instrumented kernel and save
it somewhere.
If you think tracelogger will not have enough CPU to empty the kernel buffer
fast enough then raise it’s priority. Yes it will affect lower priority
process, that’s the nature of the beast. Unless you have more then one CPU
In addition to what Mario mentioned - your 1 ms pulse isn’t going to
pre-empt tasks so often no forward progress will be made (it just seems
like a lot to a human).
Keep in mind that the OS does timer and scheduling services every ms
(tick), so one extra pulse isn’t going to make a huge change in overall
throughput.
The key is how much work gets done to service the pulse (same lines as
Mario’s previous comments about CPU usage). If you service time exceeds
you event interval time, you’re going to starve off lower prio tasks.
It’s not directly due to the rate of events, but rather your service
time is consuming CPU and is higher priority than the rest of the tasks.
–
Cheers,
Adam
With a PC, I always felt limited by the software available.
On Unix, I am limited only by my knowledge.
–Peter J. Schoenster <pschon@baste.magibox.net>
Adam,
It is not so much the 1 ms that bothers me rather it is the fact the
timer is a repeater not a one shot. In addition the receiving thread which
is in the process that owns the timer will be boosted to prio 27.
The above process shares a queue with another process for which the
former process is queue producer and the latter process is a consumer, which
has a thread running at prio 10. So basically there is one producer thread
and one comsumer thread for this queue implemented using shared memory. The
queue was implemented without a mutex to protect access. I still have
convinced myself that even if there is only one producer thread and one
consumer thread that the queue will not be corrupted by a race. ???
Regards, Bill
“Adam Mallory” <amallory@qnx.com> wrote in message
news:d31621$rm1$2@inn.qnx.com…
Mario Charest wrote:
I am facing a conundrum of sorts. I want to run the instrumented
micro-kernel and, of course, tracelogger because one of our software
engineers has decided on an ad-hoc basis to send pulses to a process at
priority 24. I noticed by default that the tracelogger runs at 15, 16
and 17, respectively. If I run the tracelogger will most likely be
pre-empted by the priority 24 process most of the time and hence not
able to collect data and will color the data I collect. Of course, I
could raise tracelogger to 24. Any ideas?
The data collection is not done by tracelogger, it’s the kernel that does
it. Tracelogger gets the data generated by the instrumented kernel and
save it somewhere.
If you think tracelogger will not have enough CPU to empty the kernel
buffer fast enough then raise it’s priority. Yes it will affect lower
priority process, that’s the nature of the beast. Unless you have more
then one CPU >
In addition to what Mario mentioned - your 1 ms pulse isn’t going to
pre-empt tasks so often no forward progress will be made (it just seems
like a lot to a human).
Keep in mind that the OS does timer and scheduling services every ms
(tick), so one extra pulse isn’t going to make a huge change in overall
throughput.
The key is how much work gets done to service the pulse (same lines as
Mario’s previous comments about CPU usage). If you service time exceeds
you event interval time, you’re going to starve off lower prio tasks. It’s
not directly due to the rate of events, but rather your service time is
consuming CPU and is higher priority than the rest of the tasks.
–
Cheers,
Adam
With a PC, I always felt limited by the software available.
On Unix, I am limited only by my knowledge.
–Peter J. Schoenster <> pschon@baste.magibox.net
William Halchin wrote:
It is not so much the 1 ms that bothers me rather it is the fact the
timer is a repeater not a one shot. In addition the receiving thread which
is in the process that owns the timer will be boosted to prio 27.
The reload doesn’t make much of a difference, and I cannot speak
intelligently about the priority choice since I don’t know how your
system interactions/relationships are working.
The above process shares a queue with another process for which the
former process is queue producer and the latter process is a consumer, which
has a thread running at prio 10. So basically there is one producer thread
and one comsumer thread for this queue implemented using shared memory. The
queue was implemented without a mutex to protect access. I still have
convinced myself that even if there is only one producer thread and one
consumer thread that the queue will not be corrupted by a race. ???
This is new information!
You probably will have corruption - the producer has no way of knowing
if it pre-empted the consumer mid ‘consumption’. You need to do some
sort of synchronization.
–
Cheers,
Adam
With a PC, I always felt limited by the software available.
On Unix, I am limited only by my knowledge.
–Peter J. Schoenster <pschon@baste.magibox.net>
The above process shares a queue with another process for which the
former process is queue producer and the latter process is a consumer,
which has a thread running at prio 10. So basically there is one producer
thread and one comsumer thread for this queue implemented using shared
memory. The queue was implemented without a mutex to protect access. I
still have convinced myself that even if there is only one producer
thread and one consumer thread that the queue will not be corrupted by a
race. ???
This is new information!
You probably will have corruption - the producer has no way of knowing if
it pre-empted the consumer mid ‘consumption’. You need to do some sort of
synchronization.
^^^ I am not happy at all with this legacy code. Very little
documentation and a lot of commented out experimental code. Anyway enough of
the kvetching. Adam, are you sure that the case of 1 producer and 1 consumer
is not a special degenerate case where synchronization is not necessary? Can
you show some counter-example to prove corruption? I have been trying to
convince myself that this degenerate case is not in some way special in not
needing synchronization. Question: would I put a mutex here? Answer: in a
heartbeat!
Regards, Bill
–
Cheers,
Adam
With a PC, I always felt limited by the software available.
On Unix, I am limited only by my knowledge.
–Peter J. Schoenster <> pschon@baste.magibox.net
“Adam Mallory” <> amallory@qnx.com> > wrote in message
news:d31f8t$5si$1@inn.qnx.com…
William Halchin wrote:
It is not so much the 1 ms that bothers me rather it is the fact the
timer is a repeater not a one shot. In addition the receiving thread
which is in the process that owns the timer will be boosted to prio 27.
The reload doesn’t make much of a difference, and I cannot speak
intelligently about the priority choice since I don’t know how your system
interactions/relationships are working.
The above process shares a queue with another process for which the
former process is queue producer and the latter process is a consumer,
which has a thread running at prio 10. So basically there is one producer
thread and one comsumer thread for this queue implemented using shared
memory. The queue was implemented without a mutex to protect access. I
still have convinced myself that even if there is only one producer
thread and one consumer thread that the queue will not be corrupted by a
race. ???
This is new information!
You probably will have corruption - the producer has no way of knowing if
it pre-empted the consumer mid ‘consumption’. You need to do some sort of
synchronization.
–
Cheers,
Adam
With a PC, I always felt limited by the software available.
On Unix, I am limited only by my knowledge.
–Peter J. Schoenster <> pschon@baste.magibox.net
William Halchin wrote:
^^^ I am not happy at all with this legacy code. Very little
documentation and a lot of commented out experimental code. Anyway enough of
the kvetching. Adam, are you sure that the case of 1 producer and 1 consumer
is not a special degenerate case where synchronization is not necessary? Can
you show some counter-example to prove corruption? I have been trying to
convince myself that this degenerate case is not in some way special in not
needing synchronization. Question: would I put a mutex here? Answer: in a
heartbeat!
Scenario:
An empty queue where the producer puts in the first element, signals the
no-empty transition, and then blocks waiting for the next element. The
consumer wakes, begins consuming the element (the only one in the list).
While this is happening, an event occurs, which wakes the producer
and since it’s higher priority, pre-empts the consumer mid-consumption.
Now the one and only element in the list is in a state of flux with
regards to the linked list (I’m assuming it’s a linked list of some
type) and the producer attempts to add an element to that list while
it’s in a state of flux - now you have undefined behaviour.
It’s hard to speak in concrete terms when I’m not sure what the data
structures look like and what the interface to them is. You could avoid
a lock using a read-copy-update paradigm. Or I guess you might be able
to work something such that the producer only works on one end of the
list and the consumer on the other and you somehow guarantee that the
queue never runs < 2 elements at any time when either the producer or
consumer needs to operate on the queue. This would be hard to do (and
more importantly hard to verify)
I know some people make assumptions that certain operations are atomic
(such as an assignment etc). This can be a very dangerous assumption
which falls under one or more of these criteria:
- a completely false assumption
- assumes the code will always be running on x86 (and banks on certain
x86isms)
- Works in uni-processor case, but not in an multi-processor one
FYI - There is an API for doing cross-platform atomic operations (check
out atomic_*() functions) safely.
–
Cheers,
Adam
With a PC, I always felt limited by the software available.
On Unix, I am limited only by my knowledge.
–Peter J. Schoenster <pschon@baste.magibox.net>
Adam,
Please see below.
Vasili
"> Scenario:
An empty queue where the producer puts in the first element, signals the
^^^ the situation is
actually more hilarious than this! ;^). The producer does not signal the
consumer when it enqueues data rather the consumer sends itself a pulse
every 5 ms at prio 10 (whereas the producer is boosted to 24 and timer
firing every 1 ms) to check the queue, i.e. there is no “coordination” or
“synchronization” between the producer and consumer, e.g. like a condition
variable or counting semaphore. I have already strongly suggested that the
producer directly(!!!) pulse the consumer after an enqueuing operation.
!!!
no-empty transition, and then blocks waiting for the next element. The
consumer wakes, begins consuming the element (the only one in the list).
While this is happening, an event occurs, which wakes the producer and
since it’s higher priority, pre-empts the consumer mid-consumption. Now
the one and only element in the list is in a state of flux with regards to
the linked list (I’m assuming it’s a linked list of some type) and the
producer attempts to add an element to that list while it’s in a state of
flux - now you have undefined behaviour.
It’s hard to speak in concrete terms when I’m not sure what the data
structures look like and what the interface to them is. You could avoid a
lock using a read-copy-update paradigm. Or I guess you might be able to
work something such that the producer only works on one end of the list
and the consumer on the other and you somehow guarantee that the queue
never runs < 2 elements at any time when either the producer or consumer
needs to operate on the queue. This would be hard to do (and more
importantly hard to verify)
I know some people make assumptions that certain operations are atomic
(such as an assignment etc). This can be a very dangerous assumption
which falls under one or more of these criteria:
- a completely false assumption
- assumes the code will always be running on x86 (and banks on certain
x86isms)
- Works in uni-processor case, but not in an multi-processor one
FYI - There is an API for doing cross-platform atomic operations (check
out atomic_*() functions) safely.
–
Cheers,
Adam
With a PC, I always felt limited by the software available.
On Unix, I am limited only by my knowledge.
–Peter J. Schoenster <> pschon@baste.magibox.net
“Adam Mallory” <amallory@qnx.com> wrote in message news:d3414t$344=>
Scenario:
An empty queue where the producer puts in the first element, signals the
no-empty transition, and then blocks waiting for the next element. The
consumer wakes, begins consuming the element (the only one in the list).
While this is happening, an event occurs, which wakes the producer and
since it’s higher priority, pre-empts the consumer mid-consumption. Now
the one and only element in the list is in a state of flux with regards to
the linked list (I’m assuming it’s a linked list of some type) and the
producer attempts to add an element to that list while it’s in a state of
flux - now you have undefined behaviour.
It’s hard to speak in concrete terms when I’m not sure what the data
structures look like and what the interface to them is. You could avoid a
lock using a read-copy-update paradigm. Or I guess you might be able to
work something such that the producer only works on one end of the list
and the consumer on the other and you somehow guarantee that the queue
never runs < 2 elements at any time when either the producer or consumer
needs to operate on the queue. This would be hard to do (and more
importantly hard to verify)
I know some people make assumptions that certain operations are atomic
(such as an assignment etc). This can be a very dangerous assumption
^^ I had a former manager that tried to convince me of this. He said I
was diificult to manage because I was impatient. I was impatient with this
kind of stupidity!
which falls under one or more of these criteria:
- a completely false assumption
- assumes the code will always be running on x86 (and banks on certain
x86isms)
- Works in uni-processor case, but not in an multi-processor one
FYI - There is an API for doing cross-platform atomic operations (check
out atomic_*() functions) safely.
^^ I have used before on QNX and like a lot this API!
Regards, Vasili
–
Cheers,
Adam
With a PC, I always felt limited by the software available.
On Unix, I am limited only by my knowledge.
–Peter J. Schoenster <> pschon@baste.magibox.net