_beginthread crashes

“Brown, Richard” <brownr@aecl.ca> wrote in message
news:9bp64m$cs7$1@inn.qnx.com

How many semaphores are in use system wide? The >default Proc options will
support a max of 125.

I have increased the number of semaphores just in case to 200 and it didn’t
help.

Can you make a thread that does not require any
semaphores?

I can’t even start a thread, _beginthread() fails every time, I have tried

another test program to create a simple thread but after starting my driver
and running it for a while I can’t create threads from anywhere.

On 19 Apr 2001 19:28:36 GMT, Wojtek Lerch <wojtek@qnx.com> wrote:

Mario Charest <> mcharest@nozinformatic.com> > wrote:
On 19 Apr 2001 15:16:53 GMT, Wojtek Lerch <> wojtek@qnx.com> > wrote:
Mario Charest <> mcharest@nozinformatic.com> > wrote:

Common mistake with thread is to use printf in each thread. In my
experience that can cause problem because file descriptor are
not thread safe.

I was under the impression that they are, provided that you use
_beginthread() rather than raw tfork().


My understanding is function using FILE * are thread safe but the file
descriptor itself isn’t.

I have no idea what you mean but that distinction. To me, the concept
of being thread-safe applies to functions, but not to data structures…

Let that the example of fprintf. It gets pass FILE *, data in FILE *
is modified by fprintf, possibly making FILE * somewhat meaningless
to the NEXT fprintf running in another thread. I say NEXT because
you just mention fprintf will not run simultaniously because of
semaphores.

Now thinking about it, this situation may not cause corruption of any
data. I could however cause problem for example if one thread does
and unget while another thread does a fgetc.

So from what you are saying it has more to do with “file locking”
issues then threads.

In that case I probably misinterpreted/distorted the information that
was given, or maybe it comes from the pre _beginthread era.

Anyway, from a few experiments I did on my machine, it seems that even
though the putc() and getc() macros are not thread-safe (which is easy
to guess from looking at their implementation in stdio.h), other
functions (like printf()) are trying to use a semaphore to make
themselves thread-safe.

According to WD, those functions use two function pointers called
_AccessFileH and _ReleaseFileH. Initially, these variables point to an
empty function, but the first call to _beginthread() sets them to point
to real functions called __AccessFileH() and __ReleaseFileH() that do
some semafore stuff.

Of course, this can only work reliably if you’re not using the
thread-unsafe getc() or putc()…


Wojtek Lerch (> wojtek@qnx.com> ) QNX Software Systems Ltd.

Mariola Solodko <mariola.m.solodko@ca.abb.com> wrote:


According to WD, those functions use two function pointers called
_AccessFileH and _ReleaseFileH. Initially, these variables point to an
empty function, but the first call to _beginthread() sets them to point
to real functions called __AccessFileH() and __ReleaseFileH() that do
some semafore stuff.

Any idea why calling the _beginthread after I have about 4 other threads
created (use semaphores a lot) does not create a thread ?

I can think of a few possible reasons… What does errno say?


Wojtek Lerch (wojtek@qnx.com) QNX Software Systems Ltd.

Mario Charest <mcharest@nozinformatic.com> wrote:

On 19 Apr 2001 19:28:36 GMT, Wojtek Lerch <> wojtek@qnx.com> > wrote:

Mario Charest <> mcharest@nozinformatic.com> > wrote:
On 19 Apr 2001 15:16:53 GMT, Wojtek Lerch <> wojtek@qnx.com> > wrote:
Mario Charest <> mcharest@nozinformatic.com> > wrote:

My understanding is function using FILE * are thread safe but the file
descriptor itself isn’t.

I have no idea what you mean but that distinction. To me, the concept
of being thread-safe applies to functions, but not to data structures…

Let that the example of fprintf. It gets pass FILE *, data in FILE *
is modified by fprintf, possibly making FILE * somewhat meaningless
to the NEXT fprintf running in another thread. I say NEXT because
you just mention fprintf will not run simultaniously because of
semaphores.

After fprintf() has returned, the contents of the FILE structure are
consistent and safe for use in the next call to fprintf(). Why would it
matter whether the next call is made from the same or another thread?

But there’s another thing… Hey, isn’t that what you originally meant:
since QNX4 “threads” are really separate processes that share their data
space, a file descriptor is only meaningful to the thread that opened
it, and to its children. If one thread calls fopen() and then another
thread calls fprintf() on the same FILE, the fd that the FILE is using
might be invalid in the other thread – or refer to a completely
different file. Even though you don’t crash in fprintf (because the
consistency of the contents of the FILE structure is protected by the
semaphore), printf() may either fail or write to the wrong file.

Now thinking about it, this situation may not cause corruption of any
data. I could however cause problem for example if one thread does
and unget while another thread does a fgetc.

Only if the first thread expects that its next call to fgetc() will
return the same character that was given to ungetc(). But it would be
silly to expect something like that in an application that has another
thread reading from the same file… As a matter of fact, it would be
silly to write an application that has multiple threads reading from the
same file without any synchronization…

By definition, thread-safe means that if two threads call the function
simultaneously, the result is guaranteed to be the same as it would be
if one thread’s call had finished before the other thread’s call had
started – but it does not guarantee that it always makes sense to write
code that doesn’t care which thread was first. For instance, even
though fclose() and fprintf() are thread safe, it’s not a good idea to
let two threads call these two functions on the same FILE pointer at the
same time, just like it’s not a good idea to call fprintf() after
fclose().

So from what you are saying it has more to do with “file locking”
issues then threads.

You mean the kind of file locking that fcntl(F_SETLK) implements?
I don’t see how…

In that case I probably misinterpreted/distorted the information that
was given, or maybe it comes from the pre _beginthread era.

Or maybe it comes from experiences with the thread-unsafe getc() and
putc()…


Wojtek Lerch (wojtek@qnx.com) QNX Software Systems Ltd.

On 20 Apr 2001 14:52:03 GMT, Wojtek Lerch <wojtek@qnx.com> wrote:

But there’s another thing… Hey, isn’t that what you originally meant:
since QNX4 “threads” are really separate processes that share their data
space, a file descriptor is only meaningful to the thread that opened
it, and to its children. If one thread calls fopen() and then another
thread calls fprintf() on the same FILE, the fd that the FILE is using
might be invalid in the other thread – or refer to a completely
different file. Even though you don’t crash in fprintf (because the
consistency of the contents of the FILE structure is protected by the
semaphore), printf() may either fail or write to the wrong file.

No I don’t think that’s the case, although I’m now very confused :wink:


I see it on two levels. I’ll use a link list as an example.
However since those functions don’t make use of global variable they
can be use in two differents threads simulatenously if they are not
working on the same list.

It would be innefficient to have a single semaphore to protect the
link list function. In the case of each thread working on different
list there is no point in protected them, worse if this would be
running on SMP, it would be real bad. Hence what needs to be
protected is the DATA, so each list should have its OWN semaphore.


According to the doc memcpy is thread safe. If you memcpy to the same
destination in two different threads, the end result might be
corrupted data. Hence some level of protection at a higher level
is required. That how I understood fprintf before you indicated
it was protected by a semaphore. I though fprintf was thread-safe
(meaning you can have multiple fprintf happening) but not if
you use the same file descriptor, since the FILE structure don’t
appear to have its own semaphores.

If all buffered filebase functions have the SAME semaphores,
then I think that’s not really good for performance as an fprintf on a
file could get block/delayed by a getc on the keyboard. On
a single CPU that’s probably not critical but on SMP that
could be desastrous, but we don’t have to worry about that
with QNX4 :wink:

So from what you are saying it has more to do with “file locking”
issues then threads.

You mean the kind of file locking that fcntl(F_SETLK) implements?
I don’t see how…

No, I put “file locking” in quote because I couldn’t find a better
word. The right word ( it just came to me) is synchronisation.

In that case I probably misinterpreted/distorted the information that
was given, or maybe it comes from the pre _beginthread era.

Or maybe it comes from experiences with the thread-unsafe getc() and
putc()…

:wink:

  • Mario

But there’s another thing… Hey, isn’t that what you originally
meant:
since QNX4 “threads” are really separate processes that share their
data
space, a file descriptor is only meaningful to the thread that
opened
it, and to its children. If one thread calls fopen() and then another
thread calls fprintf() on the same FILE, the fd that the FILE is using
might be invalid in the other thread – or refer to a completely
different file. Even though you don’t crash in fprintf (because the
consistency of the contents of the FILE structure is protected by the
semaphore), printf() may either fail or write to the wrong file.

No I don’t think that’s the case, although I’m now very confused > :wink:

What do you mean by “I don’t think that’s the case” ? It certainly is
the case that “threads” in QNX4 are processes with shared data, and this
certainly does mean that a fd obtained by open() in one thread is
completely meaningless in another thread (since from the resource
managers POV it sees 2 different pids, and is indexing its ocbs based on
pid/fd).

Mario Charest <mcharest@nozinformatic.com> wrote:

On 20 Apr 2001 14:52:03 GMT, Wojtek Lerch <> wojtek@qnx.com> > wrote:

But there’s another thing… Hey, isn’t that what you originally meant:
since QNX4 “threads” are really separate processes that share their data
space, a file descriptor is only meaningful to the thread that opened
it, and to its children. If one thread calls fopen() and then another
thread calls fprintf() on the same FILE, the fd that the FILE is using
might be invalid in the other thread – or refer to a completely
different file. Even though you don’t crash in fprintf (because the
consistency of the contents of the FILE structure is protected by the
semaphore), printf() may either fail or write to the wrong file.

No I don’t think that’s the case, although I’m now very confused > :wink:

I see it on two levels. I’ll use a link list as an example.
However since those functions don’t make use of global variable they
can be use in two differents threads simulatenously if they are not
working on the same list.

It would be innefficient to have a single semaphore to protect the
link list function. In the case of each thread working on different
list there is no point in protected them, worse if this would be
running on SMP, it would be real bad. Hence what needs to be
protected is the DATA, so each list should have its OWN semaphore.

Watcom seems to use a single semaphore for all your FILEs. I just ran a
test case where each thread opens /dev/tty and calls fgets() – and one
thread ended up reply-blocked on Dev while all the other threads were
semaphore-blocked.

But that’s not how POSIX wants it to work – POSIX wants each FILE to
have its own lock, implemented with flockfile(), and requires that it
should be safe for multiple threads to give the same FILE to fprintf()
or other functions at the same time.

According to the doc memcpy is thread safe. If you memcpy to the same
destination in two different threads, the end result might be
corrupted data. Hence some level of protection at a higher level

Yes… It seems that memcpy() doesn’t really fit into the definition
that I cited, does it…

I don’t have a copy of POSIX here, but according to
http://www.UNIX-systems.org/whitepapers/reentrant.html:

In POSIX.1c, a “reentrant function” is defined as a “function whose
effect, when called by two or more threads, is guaranteed to be as
if the threads each executed the function one after another in an
undefined order, even if the actual execution is interleaved”
(ISO/IEC 9945:1-1996, 2.2.2).

is required. That how I understood fprintf before you indicated
it was protected by a semaphore. I though fprintf was thread-safe
(meaning you can have multiple fprintf happening) but not if
you use the same file descriptor, since the FILE structure don’t
appear to have its own semaphores.

Well, printf() is supposed to be thread-safe, too.

If all buffered filebase functions have the SAME semaphores,
then I think that’s not really good for performance as an fprintf on a
file could get block/delayed by a getc on the keyboard. On
a single CPU that’s probably not critical but on SMP that
could be desastrous, but we don’t have to worry about that
with QNX4 > :wink:

If you have an application where one thread does

puts( “Press Enter to interrupt” );
getchar();
kill_all_threads();

while other threads are trying to process some files, having the
getchar() share the semaphore with all the other files would be pretty
bad even on a single CPU.

So from what you are saying it has more to do with “file locking”
issues then threads.
You mean the kind of file locking that fcntl(F_SETLK) implements?
No, I put “file locking” in quote because I couldn’t find a better
word. The right word ( it just came to me) is synchronisation.

POSIX calls it “stream locking”. And, of course, it does have to do
with threads.


Wojtek Lerch (wojtek@qnx.com) QNX Software Systems Ltd.

Wojtek Lerch <wojtek@qnx.com> wrote in message
news:9bpcov$jun$1@nntp.qnx.com

Mariola Solodko <> mariola.m.solodko@ca.abb.com> > wrote:


According to WD, those functions use two function pointers called
_AccessFileH and _ReleaseFileH. Initially, these variables point to an
empty function, but the first call to _beginthread() sets them to point
to real functions called __AccessFileH() and __ReleaseFileH() that do
some semafore stuff.

Any idea why calling the _beginthread after I have about 4 other threads
created (use semaphores a lot) does not create a thread ?

I can think of a few possible reasons… What does errno say?

Reading the errno from the driver which is creating all the threads the
errno is 0 but when running another program at the same time, _beginthread
will fail and the errno will be set to 11.

mariola <mariola@sympatico.ca> wrote:

Wojtek Lerch <> wojtek@qnx.com> > wrote in message
news:9bpcov$jun$> 1@nntp.qnx.com> …
Mariola Solodko <> mariola.m.solodko@ca.abb.com> > wrote:


According to WD, those functions use two function pointers called
_AccessFileH and _ReleaseFileH. Initially, these variables point to an
empty function, but the first call to _beginthread() sets them to point
to real functions called __AccessFileH() and __ReleaseFileH() that do
some semafore stuff.

Any idea why calling the _beginthread after I have about 4 other threads
created (use semaphores a lot) does not create a thread ?

I can think of a few possible reasons… What does errno say?

Reading the errno from the driver which is creating all the threads the
errno is 0 but when running another program at the same time, _beginthread
will fail and the errno will be set to 11.

11 is EAGAIN, and according to the docs, EAGAIN means that “there are
too many threads”. In QNX, this probably means that you have run out of
process entries.

It’s possible that you’re filling up the table with zombie processes.
Remember that QNX4 “threads” are really processes – and a child process
that terminates becomes a zombie until the parent calls wait() or
waitpid().


Wojtek Lerch (wojtek@qnx.com) QNX Software Systems Ltd.

11 is EAGAIN, and according to the docs, EAGAIN means that “there are
too many threads”. In QNX, this probably means that you have run out of
process entries.

It’s possible that you’re filling up the table with zombie processes.
Remember that QNX4 “threads” are really processes – and a child process
that terminates becomes a zombie until the parent calls wait() or
waitpid().


Wojtek Lerch (> wojtek@qnx.com> ) QNX Software Systems Ltd.

I don’t terminate any threads unless I exit the main application and

creating threads is a problem the first time I run the program (after
rebooting). I do however create a proxy and a timer about every second but
I remove them right after I’m done. One thread is in a while loop and it
has a 1 second delay time before going into the loop again. Since delay()
is not thread safe I have a function called OS_Delay() that will create a
proxy and a timer every time you call it, it will arm the timer and signal
the proxy when the time is up, then I delete the proxy, timer and exit. The
one thread calls it every second, the other two threads will call it if
there is an interrupt coming and there could be interrupts coming every
second. I was watching the number of timers created during the execution but
it never went higher than the limit.

sin or osinfo does not show any zombies and I’ve increased the number of
processes to 2000.

I just tried a small test application that creates threads until
_beginthread fails and was able to create close to 410 threads before it
failed with resource unavailable. Using osinfo I was able to see that this
was within 2 or 3 proc entries of the limit. The threads I created just did
a Receive and nothing more.

So … it looks like your exhausting some other resource. I would try
simplifying your thread by commenting out all resources (e.g., timers,
proxies, shared memory, file descriptors, dynamic memory, etc.) and then
start adding the resources back one at a time until you come across the
limiting factor.

Best of Luck

  • Richard

“Brown, Richard” <brownr@aecl.ca> wrote in message
news:9c1nta$mjm$1@inn.qnx.com

I just tried a small test application that creates threads until
_beginthread fails and was able to create close to 410 threads before it
failed with resource unavailable. Using osinfo I was able to see that this
was within 2 or 3 proc entries of the limit. The threads I created just
did
a Receive and nothing more.

So … it looks like your exhausting some other resource. I would try
simplifying your thread by commenting out all resources (e.g., timers,
proxies, shared memory, file descriptors, dynamic memory, etc.) and then
start adding the resources back one at a time until you come across the
limiting factor.

Best of Luck

  • Richard

Thanks to you all,
I have finally found the cause of my _beginthread problems and it’s a
thread safe related problem . I was initializing a semaphore from one of
the threads, however this semaphore was originally initialized by the main
application and initializing the same semaphore from more than one thread is
not safe (so I found out). Instead of sem_init() I now use sem_trywait() in
a while loop until the sem_trywait() returns -1. I’m glad this is over,
thanks again.

I have the old beta Function Safety Matrix which lists sem_init as “Depends”
under thread safe. I guess you were on the wrong side of that “Depends”.

  • Richard

Brown, Richard <brownr@aecl.ca> wrote:
: I have the old beta Function Safety Matrix which lists sem_init as “Depends”
: under thread safe. I guess you were on the wrong side of that “Depends”.
: - Richard

Here’s what the current version says:

Safety:

Interrupt handler No

Signal handler Yes, but modifies errno

Thread Read the Caveats

Caveats:

Don’t initialize the same semaphore from more than one thread. It’s best
to set up semaphores before starting any threads.


Steve Reid stever@qnx.com
TechPubs (Technical Publications)
QNX Software Systems

Where can I find this version?

  • Richard

I have downloaded the Patch B for Watcom 10.6 and with the documentation you
will have a list of all the thread safe and not safe functions. Also when
you look up each function you will see a table showing if it is
thread/signal/handler safe.

“Brown, Richard” <brownr@aecl.ca> wrote in message
news:9c77cf$a3o$1@inn.qnx.com

Where can I find this version?

  • Richard

According to my /etc/version/wcc I have Patch B installed but I do not have
those doc changes that you mention. I investigated a little and noticed that
the Patch A release is larger than the patch B release. The release note
seems to indicate that the Patch B release contains everything from Patch A
but the size of the archives tells a different story. (From
/updates/watcom10.6/Released). I also checked the contents of the Patch B
archive and found 0 helpfiles.

I think there is a booboo here.

  • Richard

“Brown, Richard” <brownr@aecl.ca> wrote in message
news:9ck3ve$b06$1@inn.qnx.com

According to my /etc/version/wcc I have Patch B installed but I do not
have
those doc changes that you mention. I investigated a little and noticed
that
the Patch A release is larger than the patch B release. The release note
seems to indicate that the Patch B release contains everything from Patch
A
but the size of the archives tells a different story. (From
/updates/watcom10.6/Released). I also checked the contents of the Patch B
archive and found 0 helpfiles.

I think there is a booboo here.

  • Richard



    I am using the online ducumentation (html files from that patch) and this

is the PatchB I’ve downloaded:
/watcom10.6/wcc/Released/wcc10.6.patchB.tar.F

I am using the online ducumentation (html files from that patch) and this
is the PatchB I’ve downloaded:
/watcom10.6/wcc/Released/wcc10.6.patchB.tar.F

I believe I have installed the same archive but no help files are included.
What do you get from the following?

cksum wcc10.6.patchB.tar.F produces:
3747462726 3794650 wcc10.6.patchB.tar.F

This matches the Update_info.

fcat wcc10.6.patchB.tar.F | pax -v *help*
shows no help files

I know that the PhAB help files have the safety matrix associated with each
function but the watcom help files do not.

  • Richard

“Brown, Richard” <brownr@aecl.ca> wrote in message
news:9cmo79$5d$1@inn.qnx.com

I am using the online ducumentation (html files from that patch) and
this
is the PatchB I’ve downloaded:
/watcom10.6/wcc/Released/wcc10.6.patchB.tar.F

I believe I have installed the same archive but no help files are
included.
What do you get from the following?

cksum wcc10.6.patchB.tar.F produces:
3747462726 3794650 wcc10.6.patchB.tar.F

This matches the Update_info.

fcat wcc10.6.patchB.tar.F | pax -v *help*
shows no help files

I know that the PhAB help files have the safety matrix associated with
each
function but the watcom help files do not.

  • Richard


    Yes I get the same results, I don’t know anything about watcom help files

but I do know that there are help html files, we have our documentation
online so html files are all we need,
we use microsoft explorer to connect to our qnx server and view all the help
files (html files)