evaluation of duration time for message passing

Hi everybody,
i m a new qnx-user.
i wrote a client/server application to measure the duration of the message
passing on any Qnx-computersystem.
For this purpose, the client automatically generates a sample of messages
that are send to the server, then it gets the send time. After receiving the
reply from the server , the client will get the receive time for each
message and will build the time difference to determine the duration of the
message passing.
My Problem:
the measurement result are as following:
(all messages have the same Property)
message Nr time(microsec)
1 77
2 29
3 21
4 21
5 21
6 20,5
7 20,5
8 21.
the duration of the first message passing is long (77microsec). Could you
tell me the causes of this behavior? Is there anything to adjust in the
Qnx-system to resolve this Problem, so that i get a constant duration for
every message passings.
Thanks in advance.
Dadji.

dadji wrote:

Hi everybody,
i m a new qnx-user.
i wrote a client/server application to measure the duration of the message
passing on any Qnx-computersystem.
For this purpose, the client automatically generates a sample of messages
that are send to the server, then it gets the send time. After receiving the
reply from the server , the client will get the receive time for each
message and will build the time difference to determine the duration of the
message passing.
My Problem:
the measurement result are as following:
(all messages have the same Property)
message Nr time(microsec)
1 77
2 29
3 21
4 21
5 21
6 20,5
7 20,5
8 21.
the duration of the first message passing is long (77microsec). Could you
tell me the causes of this behavior? Is there anything to adjust in the
Qnx-system to resolve this Problem, so that i get a constant duration for
every message passings.
Thanks in advance.
Dadji.

Well, in order to insure that your thread isn’t being pre-empted (and
making your measurements invalid), you’d have to run it as the highest
priority. Is your thread (in the client, since the server will float)
at prio 255 ?

Also, what timesource are you using for a timestamp ?

Rennie

Another thing to consider is cache effects…

Rennie Allen wrote:

dadji wrote:

Hi everybody,
i m a new qnx-user.
i wrote a client/server application to measure the duration of the
message
passing on any Qnx-computersystem.
For this purpose, the client automatically generates a sample of messages
that are send to the server, then it gets the send time. After
receiving the
reply from the server , the client will get the receive time for each
message and will build the time difference to determine the duration
of the
message passing.
My Problem:
the measurement result are as following:
(all messages have the same Property)
message Nr time(microsec)
1 77
2 29
3 21
4 21
5 21
6 20,5
7 20,5
8 21.
the duration of the first message passing is long (77microsec). Could you
tell me the causes of this behavior? Is there anything to adjust in the
Qnx-system to resolve this Problem, so that i get a constant duration for
every message passings.
Thanks in advance.
Dadji.


Well, in order to insure that your thread isn’t being pre-empted (and
making your measurements invalid), you’d have to run it as the highest
priority. Is your thread (in the client, since the server will float)
at prio 255 ?

Also, what timesource are you using for a timestamp ?

Rennie


cburgess@qnx.com

Thank you for your hepl.
i’m using the system function ClockCycles() to get the time in clockcycle
units, and then determine the time by using the system variable
SYSPAGE_ENTRY->cycles_per_sec.
i also run the Programm with the highest system priority , but the first
measurement is still corrupt (ca 77microsec). i still don’t know the causes
of this behavior, i will just ignore this first value.
I still thank you for your suggestions.
Dadji.
“Rennie Allen” <rnogspamallen@comcast.net> schrieb im Newsbeitrag
news:d9u6r6$8ms$1@inn.qnx.com

dadji wrote:
Hi everybody,
i m a new qnx-user.
i wrote a client/server application to measure the duration of the
message
passing on any Qnx-computersystem.
For this purpose, the client automatically generates a sample of
messages
that are send to the server, then it gets the send time. After receiving
the
reply from the server , the client will get the receive time for each
message and will build the time difference to determine the duration of
the
message passing.
My Problem:
the measurement result are as following:
(all messages have the same Property)
message Nr time(microsec)
1 77
2 29
3 21
4 21
5 21
6 20,5
7 20,5
8 21.
the duration of the first message passing is long (77microsec). Could
you
tell me the causes of this behavior? Is there anything to adjust in the
Qnx-system to resolve this Problem, so that i get a constant duration
for
every message passings.
Thanks in advance.
Dadji.

Well, in order to insure that your thread isn’t being pre-empted (and
making your measurements invalid), you’d have to run it as the highest
priority. Is your thread (in the client, since the server will float)
at prio 255 ?

Also, what timesource are you using for a timestamp ?

Rennie

dadji wrote:

Thank you for your hepl.
i’m using the system function ClockCycles() to get the time in clockcycle
units, and then determine the time by using the system variable
SYSPAGE_ENTRY->cycles_per_sec.
i also run the Programm with the highest system priority , but the first
measurement is still corrupt (ca 77microsec). i still don’t know the causes
of this behavior, i will just ignore this first value.
I still thank you for your suggestions.
Dadji.

Then Colin is most likely correct that the problem is cache effect. The
first message pass is non-cached.

Rennie

thanks.
“Rennie Allen” <rnogspamallen@comcast.net> schrieb im Newsbeitrag
news:da3lpo$cef$1@inn.qnx.com

dadji wrote:
Thank you for your hepl.
i’m using the system function ClockCycles() to get the time in
clockcycle
units, and then determine the time by using the system variable
SYSPAGE_ENTRY->cycles_per_sec.
i also run the Programm with the highest system priority , but the first
measurement is still corrupt (ca 77microsec). i still don’t know the
causes
of this behavior, i will just ignore this first value.
I still thank you for your suggestions.
Dadji.

Then Colin is most likely correct that the problem is cache effect. The
first message pass is non-cached.

Rennie

i’m using the system function ClockCycles() to get the time in clockcycle
units, and then determine the time by using the system variable
SYSPAGE_ENTRY->cycles_per_sec.
i also run the Programm with the highest system priority , but the first
measurement is still corrupt (ca 77microsec). i still don’t know the
causes
of this behavior, i will just ignore this first value.

the measurement result are as following:
(all messages have the same Property)
message Nr time(microsec)
1 77
2 29
3 21
4 21
5 21
6 20,5
7 20,5
8 21.
the duration of the first message passing is long (77microsec). Could you
tell me the causes of this behavior? Is there anything to adjust in the
Qnx-system to resolve this Problem, so that i get a constant duration for
every message passings. [dadji]

Another thing to consider is cache effects… [Collin Burgess]

  • is there a possibility for the programmer to influence QNX’s caching
    mechanism?
  • How does it work for messages in general?

Caching, in general, is intended to speed up memory access if the originally
accessed memory location was not changed after the preceding access, right?
When QNX caches the message’s memory, it should be sufficient to perform a
dummy-message-pass in order to cache the message. From this moment on, all
subsequent message passings for this message would be cached. Right? But,
suppose there are many different processes within the system sending and
receiving messages at not determined times. When does QNX throw one
particular message belonging to a specific process out of the cache, so that
the next message sent by this process would be un-cached again, causing the
initial time delay for re-caching?

Nnamdi

Nnamdi Kohn wrote:

Another thing to consider is cache effects… [Collin Burgess]

\

  • is there a possibility for the programmer to influence QNX’s caching
    mechanism?

MMU can be setup to control what is cached and what is not cached and maybe what addresses are locked in cache but that’s prolly about it.


  • How does it work for messages in general?

No diff from any other memory accesses, the MMU is in charge.


Caching, in general, is intended to speed up memory access if the originally
accessed memory location was not changed after the preceding access, right?

Writing to memory is cached so that any subsequent reads also get the benefit.


When QNX caches the message’s memory, it should be sufficient to perform a
dummy-message-pass in order to cache the message. From this moment on, all
subsequent message passings for this message would be cached. Right? But,

Yep.


suppose there are many different processes within the system sending and
receiving messages at not determined times. When does QNX throw one
particular message belonging to a specific process out of the cache, so that
the next message sent by this process would be un-cached again, causing the
initial time delay for re-caching?

I presume it’ll be oldest data in the cache gets dropped to make way for the new arrivals so you don’t get much predictable timing, just faster on average. Average throughput is what general computing speed is all about.

Locking memory permanently into cache might be possible if you really need it but I doubt there’s any API that will help you. You’ll need to hack the MMU/Cache registers directly.


Evan

Nnamdi Kohn <nnamdi.kohn@tu-bs.de> wrote:

i’m using the system function ClockCycles() to get the time in clockcycle
units, and then determine the time by using the system variable
SYSPAGE_ENTRY->cycles_per_sec.
i also run the Programm with the highest system priority , but the first
measurement is still corrupt (ca 77microsec). i still don’t know the
causes
of this behavior, i will just ignore this first value.

the measurement result are as following:
(all messages have the same Property)
message Nr time(microsec)
1 77
2 29
3 21
4 21
5 21
6 20,5
7 20,5
8 21.
the duration of the first message passing is long (77microsec). Could you
tell me the causes of this behavior? Is there anything to adjust in the
Qnx-system to resolve this Problem, so that i get a constant duration for
every message passings. [dadji]

Another thing to consider is cache effects… [Collin Burgess]

Note, it might be CODE cacheing in the instruction cache, as well as
data caching in the data cache.

Have you tried your test cycling through different send buffers on
each cycle (and different receive buffers) vs trying the test
re-using the same send buffer?

That would help to isolate the data vs instruction cache issues.

  • is there a possibility for the programmer to influence QNX’s caching
    mechanism?

Cacheing is done at the hardware level, the OS doesn’t (generally) do
anything with it, other than to allow the cache controller to do its
work. (There are some exceptions, for instance memory can be allocated
through mmap*() to be PROT_NOCACHE to disable cacheing for a region.)

But, with at least 6.3.0 SP1 (documented with 6.3.0 SP2) there is a
cache control library. Take a look at <sys/cache.h>.

  • How does it work for messages in general?

In general, we do absolutely nothing special with messages.

Caching, in general, is intended to speed up memory access if the originally
accessed memory location was not changed after the preceding access, right?
When QNX caches the message’s memory, it should be sufficient to perform a
dummy-message-pass in order to cache the message.

In general, QNX doesn’t cache memory – the CPU cache controller decides
what to load into cache, do pre-reads or whatever it feels is neccessary.

From this moment on, all
subsequent message passings for this message would be cached. Right? But,
suppose there are many different processes within the system sending and
receiving messages at not determined times. When does QNX throw one
particular message belonging to a specific process out of the cache, so that
the next message sent by this process would be un-cached again, causing the
initial time delay for re-caching?

Again, QNX doesn’t do this. It will be done by the CPU’s cache controller,
generally on an LRU (least-recently-used) basis.

In a common case:
write data into buffer
send buffer
The buffer is likely to be in CPU cache, because it has just been updated.

But, if the data to be sent is accumulated over time, then quite possibly
the buffer won’t be in CPU cache.

-David

David Gibbs
QNX Training Services
dagibbs@qnx.com

suppose there are many different processes within the system sending and
receiving messages at not determined times. When does QNX throw one
particular message belonging to a specific process out of the cache, so
that
the next message sent by this process would be un-cached again, causing
the
initial time delay for re-caching? [Nnamdi Kohn]

Locking memory permanently into cache might be
possible if you really need it but I doubt there’s any
API that will help you. You’ll need to hack the
MMU/Cache registers directly. [Evan Hillas]

Great… back to the methods of the good old times… I loved it :slight_smile:

Where do I find detailed information about wheres and hows of the MMU/Cache?

Thanks. Nnamdi.

Nnamdi Kohn wrote:

Where do I find detailed information about wheres and hows of the MMU/Cache?

Sorry, not me.


Evan

Evan Hillas wrote:

Nnamdi Kohn wrote:

Where do I find detailed information about wheres and hows of the
MMU/Cache?


Sorry, not me.

I was only speaking from general knowledge.


Evan

Nnamdi Kohn wrote:

suppose there are many different processes within the system sending and
receiving messages at not determined times. When does QNX throw one
particular message belonging to a specific process out of the cache, so

that

the next message sent by this process would be un-cached again, causing

the

initial time delay for re-caching? [Nnamdi Kohn]


Locking memory permanently into cache might be
possible if you really need it but I doubt there’s any
API that will help you. You’ll need to hack the
MMU/Cache registers directly. [Evan Hillas]


Great… back to the methods of the good old times… I loved it > :slight_smile:

Where do I find detailed information about wheres and hows of the MMU/Cache?

The MMU is handled by QNX … hands off.
AFAIK … the Cache is handled by the microcode of the CPU. No chance to
fiddle here around.

You can only choose the behaviour of the cache by the BIOS setup … or
simply switch it off and look if it helps.

The SMI services have more influence about the real-time behaviour of
your system … IMHO

–Armin










Thanks. Nnamdi.

The MMU is handled by the kernel, but caching can be controlled to some
extent indirectly. Perhaps try to ‘touch’ your buffer with a simple loop
immediately before sending it - that should put it into the cache. It will
not really cause any significant performance hit - the kernel call to send
the message takes a LOT more than filling the buffer.

Different CPUs may behave differently depending on buffer sizes and
particular software set being executed. For example, AMD’s L2 cache is
‘victim cache’ (populated by lines evicted from L1), but Intel uses a
different policy (supposedly smarter). Pentium4’s L1 instruction cache is
quite esoteric - it caches instructions after decoding (so they call it
‘trace cache’), which helps the loops since instructions only need to be
decoded once (and then the first pass of a loop inherently takes longer).

– igor

“Armin Steinhoff” <a-steinhoff@web.de> wrote in message
news:dd9tk7$lel$1@inn.qnx.com

Nnamdi Kohn wrote:
suppose there are many different processes within the system sending and
receiving messages at not determined times. When does QNX throw one
particular message belonging to a specific process out of the cache, so

that

the next message sent by this process would be un-cached again, causing

the

initial time delay for re-caching? [Nnamdi Kohn]


Locking memory permanently into cache might be
possible if you really need it but I doubt there’s any
API that will help you. You’ll need to hack the
MMU/Cache registers directly. [Evan Hillas]


Great… back to the methods of the good old times… I loved it > :slight_smile:

Where do I find detailed information about wheres and hows of the
MMU/Cache?

The MMU is handled by QNX … hands off.
AFAIK … the Cache is handled by the microcode of the CPU. No chance to
fiddle here around.

You can only choose the behaviour of the cache by the BIOS setup … or
simply switch it off and look if it helps.

The SMI services have more influence about the real-time behaviour of your
system … IMHO

–Armin











Thanks. Nnamdi.