qnx 4.25, Watcom 10.6 patch B
I am starting several threads in my application and now and then my program
misbehaves after one of the threads allocates memory by calling calloc().
What happens is the thread will allocate memory and right after that another
structure in my main program gets overwritten. It doesn’t happen all the
time. I thought that maybe the thread needs more stack space so insted of
2000 I’ve changed the size to 4000 when calling _beginthread but it crashes
with a violation access. I have also 2M of shared memory opened. Is there
any special options for compiling when you create threads? What about the
stack size, what is the min and max stack size I can have when creating a
thread.
From what I recall QSSL recommends NOT using threads with QNX4.
That being said there is a “-bm” compiler switch which is supposed to be
turned on for multithreading.
Hope this helps,
Previously, Brown, Richard wrote in qdn.public.qnx4:
From what I recall QSSL recommends NOT using threads with QNX4.
i don’ think calloc() is thread safe. You better check with the list
of thread safe functions. The list is small, so there are a lot of
commonly used functions that are not safe!
That being said there is a “-bm” compiler switch which is supposed to be
turned on for multithreading.
Hope this helps,
From an older 10.6 Beta Function Safety Matrix calloc() is listed as thread
safe but NOT signal or interrupt safe. In the Re-entrant function list
(which replaced the function safety matrix) calloc() is not listed.
thanks for your help, I think calloc works ok, I had some memory leaking
problems and now that I’ve fixed it the _beginthread function doesn’t crash
but it doesn’t create a thread either. I have created 4 threads and when
I’m trying for the 5th one the _beginthread returns -1 but the error number
is 0. At the same time I’m trying to run another test application that
doesn’t do anything except for creating one thread and when my main program
can’t create threads the test program can’t either. If my main program
doesn’t do much I will be able to create the 5th thread but when the program
creates structures and allocates more memory the thread won’t be created.
I’ve checked the system memory and I still have 14M of RAM when this happens
but I don’t know if this is enough or not. Any suggestions would help at
this point
“Kevin Warkentin” <kevin.p.warkentin@ca.abb.com> wrote in message
news:9al2bf$d99$2@inn.qnx.com…
qnx 4.25, Watcom 10.6 patch B
I am starting several threads in my application and now and then my
program
misbehaves after one of the threads allocates memory by calling calloc().
What happens is the thread will allocate memory and right after that
another
structure in my main program gets overwritten. It doesn’t happen all the
time. I thought that maybe the thread needs more stack space so insted of
2000 I’ve changed the size to 4000 when calling _beginthread but it
crashes
with a violation access. I have also 2M of shared memory opened. Is
there
any special options for compiling when you create threads? What about the
stack size, what is the min and max stack size I can have when creating a
thread.
\
Your first post sounds like a heap corruption problem (not simply a
memory leak) so fixing the memory leak has probably just changed the
symptoms. Isn’t multi-threaded debugging wonderful ? One quick
question, are you allocating your thread stack off the heap (this would
include letting _beginthread allocate it for you), or statically (the
first method can cause heap corruption directly if the stack is
overwritten, the second can cause heap corruption also if the stack is
overwritten, but only indirectly) ?
btw: QNX4 or QNX6, a heap corruption induced in one thread will nuke the
whole process, if dynamic memory allocation is involved (i.e. this has
nothing to do with tfork vs. posix threads).
-----Original Message-----
From: Kevin Warkentin [mailto:kevin.p.warkentin@ca.abb.com]
Posted At: Tuesday, April 10, 2001 7:25 AM
Posted To: qnx4
Conversation: _beginthread crashes
Subject: Re: _beginthread crashes
thanks for your help, I think calloc works ok, I had some memory leaking
problems and now that I’ve fixed it the _beginthread function doesn’t
crash
but it doesn’t create a thread either. I have created 4 threads and
when
I’m trying for the 5th one the _beginthread returns -1 but the error
number
is 0. At the same time I’m trying to run another test application that
doesn’t do anything except for creating one thread and when my main
program
can’t create threads the test program can’t either. If my main program
doesn’t do much I will be able to create the 5th thread but when the
program
creates structures and allocates more memory the thread won’t be
created.
I’ve checked the system memory and I still have 14M of RAM when this
happens
but I don’t know if this is enough or not. Any suggestions would help
at
this point
“Kevin Warkentin” <kevin.p.warkentin@ca.abb.com> wrote in message
news:9al2bf$d99$2@inn.qnx.com…
qnx 4.25, Watcom 10.6 patch B
I am starting several threads in my application and now and then my
program
misbehaves after one of the threads allocates memory by calling
calloc().
What happens is the thread will allocate memory and right after that
another
structure in my main program gets overwritten. It doesn’t happen all
the
time. I thought that maybe the thread needs more stack space so
insted of
2000 I’ve changed the size to 4000 when calling _beginthread but it
crashes
with a violation access. I have also 2M of shared memory opened. Is
there
any special options for compiling when you create threads? What about
the
stack size, what is the min and max stack size I can have when
creating a
thread.
\
I am not sure how to find the heap corruption problem if it exists. I am
allocating thread stack first, using calloc() then I call _beginthread but
I’ve tried it both ways and I get the same results. The other test program
I am trying to run in the background will return an error equal to 11 when
trying to create a thread, which means too many threads. I am working with
a huge program and trying to go through every single memcpy and strcpy is
impossible but I might have to do that. Any suggestions?
“Rennie Allen” <RAllen@csical.com> wrote in message
news:D4907B331846D31198090050046F80C903C321@exchangecal.hq.csical.com…
Your first post sounds like a heap corruption problem (not simply a
memory leak) so fixing the memory leak has probably just changed the
symptoms. Isn’t multi-threaded debugging wonderful ? One quick
question, are you allocating your thread stack off the heap (this would
include letting _beginthread allocate it for you), or statically (the
first method can cause heap corruption directly if the stack is
overwritten, the second can cause heap corruption also if the stack is
overwritten, but only indirectly) ?
btw: QNX4 or QNX6, a heap corruption induced in one thread will nuke the
whole process, if dynamic memory allocation is involved (i.e. this has
nothing to do with tfork vs. posix threads).
-----Original Message-----
From: Kevin Warkentin [mailto:> kevin.p.warkentin@ca.abb.com> ]
Posted At: Tuesday, April 10, 2001 7:25 AM
Posted To: qnx4
Conversation: _beginthread crashes
Subject: Re: _beginthread crashes
thanks for your help, I think calloc works ok, I had some memory leaking
problems and now that I’ve fixed it the _beginthread function doesn’t
crash
but it doesn’t create a thread either. I have created 4 threads and
when
I’m trying for the 5th one the _beginthread returns -1 but the error
number
is 0. At the same time I’m trying to run another test application that
doesn’t do anything except for creating one thread and when my main
program
can’t create threads the test program can’t either. If my main program
doesn’t do much I will be able to create the 5th thread but when the
program
creates structures and allocates more memory the thread won’t be
created.
I’ve checked the system memory and I still have 14M of RAM when this
happens
but I don’t know if this is enough or not. Any suggestions would help
at
this point
“Kevin Warkentin” <> kevin.p.warkentin@ca.abb.com> > wrote in message
news:9al2bf$d99$> 2@inn.qnx.com> …
qnx 4.25, Watcom 10.6 patch B
I am starting several threads in my application and now and then my
program
misbehaves after one of the threads allocates memory by calling
calloc().
What happens is the thread will allocate memory and right after that
another
structure in my main program gets overwritten. It doesn’t happen all
the
time. I thought that maybe the thread needs more stack space so
insted of
2000 I’ve changed the size to 4000 when calling _beginthread but it
crashes
with a violation access. I have also 2M of shared memory opened. Is
there
any special options for compiling when you create threads? What about
the
stack size, what is the min and max stack size I can have when
creating a
thread.
\
On Wed, 11 Apr 2001 14:37:46 -0400, “Kevin Warkentin”
<kevin.p.warkentin@ca.abb.com> wrote:
I am not sure how to find the heap corruption problem if it exists. I am
allocating thread stack first, using calloc() then I call _beginthread but
I’ve tried it both ways and I get the same results. The other test program
I am trying to run in the background will return an error equal to 11 when
I’m not sure of the context of that error code but 11 is also the
signal number for a SIGSEGV ???
trying to create a thread, which means too many threads. I am working with
a huge program and trying to go through every single memcpy and strcpy is
impossible but I might have to do that. Any suggestions?
“Rennie Allen” <> RAllen@csical.com> > wrote in message
news:> D4907B331846D31198090050046F80C903C321@exchangecal.hq.csical.com> …
Your first post sounds like a heap corruption problem (not simply a
memory leak) so fixing the memory leak has probably just changed the
symptoms. Isn’t multi-threaded debugging wonderful ? One quick
question, are you allocating your thread stack off the heap (this would
include letting _beginthread allocate it for you), or statically (the
first method can cause heap corruption directly if the stack is
overwritten, the second can cause heap corruption also if the stack is
overwritten, but only indirectly) ?
btw: QNX4 or QNX6, a heap corruption induced in one thread will nuke the
whole process, if dynamic memory allocation is involved (i.e. this has
nothing to do with tfork vs. posix threads).
-----Original Message-----
From: Kevin Warkentin [mailto:> kevin.p.warkentin@ca.abb.com> ]
Posted At: Tuesday, April 10, 2001 7:25 AM
Posted To: qnx4
Conversation: _beginthread crashes
Subject: Re: _beginthread crashes
thanks for your help, I think calloc works ok, I had some memory leaking
problems and now that I’ve fixed it the _beginthread function doesn’t
crash
but it doesn’t create a thread either. I have created 4 threads and
when
I’m trying for the 5th one the _beginthread returns -1 but the error
number
is 0. At the same time I’m trying to run another test application that
doesn’t do anything except for creating one thread and when my main
program
can’t create threads the test program can’t either. If my main program
doesn’t do much I will be able to create the 5th thread but when the
program
creates structures and allocates more memory the thread won’t be
created.
I’ve checked the system memory and I still have 14M of RAM when this
happens
but I don’t know if this is enough or not. Any suggestions would help
at
this point
“Kevin Warkentin” <> kevin.p.warkentin@ca.abb.com> > wrote in message
news:9al2bf$d99$> 2@inn.qnx.com> …
qnx 4.25, Watcom 10.6 patch B
I am starting several threads in my application and now and then my
program
misbehaves after one of the threads allocates memory by calling
calloc().
What happens is the thread will allocate memory and right after that
another
structure in my main program gets overwritten. It doesn’t happen all
the
time. I thought that maybe the thread needs more stack space so
insted of
2000 I’ve changed the size to 4000 when calling _beginthread but it
crashes
with a violation access. I have also 2M of shared memory opened. Is
there
any special options for compiling when you create threads? What about
the
stack size, what is the min and max stack size I can have when
creating a
thread.
\
Could it be that you have exhausted all of Proc’s process entries? osinfo is
a good tool for monitoring this resource.
I am not sure how to find the heap corruption problem if it exists.
I think there is a debugging malloc library for QNX4 (there is one for
QNX6) on the website.
I am
allocating thread stack first, using calloc() then I call _beginthread
but
I’ve tried it both ways and I get the same results.
Try static allocation of the stack:
eg.
stack[NUM_THREADS][2048];
probably not what you want, but a good diagnostic step.
The other test program
I am trying to run in the background will return an error equal to 11
when
trying to create a thread, which means too many threads.
Yes, but once there is heap (or stack) corruption, it isn’t unlikely to
see meaningless error returns. You could check to make sure you haven’t
exhausted process entries (maybe you are allocating a lot of proxies ?).
I am working with
a huge program and trying to go through every single memcpy and strcpy
is
impossible but I might have to do that. Any suggestions?
Look for callocs/mallocs/memcpy/strcpy assignments where sizeof() is being user rather than sizeof(*variable). If it is
a huge program there is a chance that a variable pointer was wrongly
declared as some type of structure, and (at the same time as the first
mistake) the calloc for the pointer was done as a sizeof(). When the programmer later tried to access a member that
wasn’t part of the wrongly assigned type they couldn’t compile, and
noticed their error, however when they fixed it they didn’t update the
calloc type. If the sizeof() the correct type is bigger than the
sizeof() the initial (wrong) type, then heap corruption will occur (the
same logic applies to memcpy’s but in reverse - i.e. if the correct type
is smaller than the wrong type, then heap corruption occurs).
Hope this helps Kevin
thanks Rennie, I was using Kevin’s computer to post these messages but I’m
not Kevin, he seems to know you though.
I am still at the same point, I don’t think I have a heap corruption, I was
running some utilities to check the heap and it looked ok, I’ve checked
every single calloc(), memset(), memcpy() and strcpy(). The code I’m
working with is a working code for a different operating system and the
additions I’ve made were to make it compatible with QNX. It’s a PCI
driver, uses a interrupt handler, semaphores and shared memory. The program
works fine if I create all the threads at the start up.
I’ve checked my resources when running the program and I’m not exceeding
any.
I ran out of ideas, I will have make additions to this program to support
redundancy and maybe then something will come up.
thanks again
“Rennie Allen” <RAllen@csical.com> wrote in message
news:D4907B331846D31198090050046F80C903C80B@exchangecal.hq.csical.com…
I am not sure how to find the heap corruption problem if it exists.
I think there is a debugging malloc library for QNX4 (there is one for
QNX6) on the website.
I am
allocating thread stack first, using calloc() then I call _beginthread
but
I’ve tried it both ways and I get the same results.
Try static allocation of the stack:
eg.
stack[NUM_THREADS][2048];
probably not what you want, but a good diagnostic step.
The other test program
I am trying to run in the background will return an error equal to 11
when
trying to create a thread, which means too many threads.
Yes, but once there is heap (or stack) corruption, it isn’t unlikely to
see meaningless error returns. You could check to make sure you haven’t
exhausted process entries (maybe you are allocating a lot of proxies ?).
I am working with
a huge program and trying to go through every single memcpy and strcpy
is
impossible but I might have to do that. Any suggestions?
Look for callocs/mallocs/memcpy/strcpy assignments where sizeof() is being user rather than sizeof(*variable). If it is
a huge program there is a chance that a variable pointer was wrongly
declared as some type of structure, and (at the same time as the first
mistake) the calloc for the pointer was done as a sizeof(). When the programmer later tried to access a member that
wasn’t part of the wrongly assigned type they couldn’t compile, and
noticed their error, however when they fixed it they didn’t update the
calloc type. If the sizeof() the correct type is bigger than the
sizeof() the initial (wrong) type, then heap corruption will occur (the
same logic applies to memcpy’s but in reverse - i.e. if the correct type
is smaller than the wrong type, then heap corruption occurs).
Hope this helps Kevin
The code I’m
working with is a working code for a different operating system and
the
additions I’ve made were to make it compatible with QNX.
Well, just because something runs on another environment, that doesn’t
mean that it is error free. Memory layouts vary between operating
systems.
It’s a PCI
driver, uses a interrupt handler, semaphores and shared memory. The
program
works fine if I create all the threads at the start up.
Not that I am suggesting tht you gloss over the error, but pre-creating
your threads is a good idea in a real-time environment anyway.
On Wed, 18 Apr 2001 09:57:19 -0400, “Mariola Solodko”
<mariola.m.solodko@ca.abb.com> wrote:
Common mistake with thread is to use printf in each thread. In my
experience that can cause problem because file descriptor are
not thread safe.
thanks Rennie, I was using Kevin’s computer to post these messages but I’m
not Kevin, he seems to know you though.
I am still at the same point, I don’t think I have a heap corruption, I was
running some utilities to check the heap and it looked ok, I’ve checked
every single calloc(), memset(), memcpy() and strcpy(). The code I’m
working with is a working code for a different operating system and the
additions I’ve made were to make it compatible with QNX. It’s a PCI
driver, uses a interrupt handler, semaphores and shared memory. The program
works fine if I create all the threads at the start up.
I’ve checked my resources when running the program and I’m not exceeding
any.
I ran out of ideas, I will have make additions to this program to support
redundancy and maybe then something will come up.
thanks again
“Rennie Allen” <> RAllen@csical.com> > wrote in message
news:> D4907B331846D31198090050046F80C903C80B@exchangecal.hq.csical.com> …
I am not sure how to find the heap corruption problem if it exists.
I think there is a debugging malloc library for QNX4 (there is one for
QNX6) on the website.
I am
allocating thread stack first, using calloc() then I call _beginthread
but
I’ve tried it both ways and I get the same results.
Try static allocation of the stack:
eg.
stack[NUM_THREADS][2048];
probably not what you want, but a good diagnostic step.
The other test program
I am trying to run in the background will return an error equal to 11
when
trying to create a thread, which means too many threads.
Yes, but once there is heap (or stack) corruption, it isn’t unlikely to
see meaningless error returns. You could check to make sure you haven’t
exhausted process entries (maybe you are allocating a lot of proxies ?).
I am working with
a huge program and trying to go through every single memcpy and strcpy
is
impossible but I might have to do that. Any suggestions?
Look for callocs/mallocs/memcpy/strcpy assignments where sizeof() is being user rather than sizeof(*variable). If it is
a huge program there is a chance that a variable pointer was wrongly
declared as some type of structure, and (at the same time as the first
mistake) the calloc for the pointer was done as a sizeof(). When the programmer later tried to access a member that
wasn’t part of the wrongly assigned type they couldn’t compile, and
noticed their error, however when they fixed it they didn’t update the
calloc type. If the sizeof() the correct type is bigger than the
sizeof() the initial (wrong) type, then heap corruption will occur (the
same logic applies to memcpy’s but in reverse - i.e. if the correct type
is smaller than the wrong type, then heap corruption occurs).
Hope this helps Kevin
Mario Charest <mcharest@nozinformatic.com> wrote:
On Wed, 18 Apr 2001 09:57:19 -0400, “Mariola Solodko”
mariola.m.solodko@ca.abb.com> > wrote:
Common mistake with thread is to use printf in each thread. In my
experience that can cause problem because file descriptor are
not thread safe.
Are you sure?
I was under the impression that they are, provided that you use
_beginthread() rather than raw tfork().
–
Wojtek Lerch (wojtek@qnx.com) QNX Software Systems Ltd.
thanks, I was going to ask you the same thing , I’m using printf() thoughout
my threads and I in the Watcom 10.6 manual printf() is said to be thread
safe. Anyway just to see I’ve removed all the calls to printf() and it
didn’t help ( I was hoping it would). I’m thinking maybe not being able to
create threads has something to do with the shared memory. I open it with
shm_open, then I map to it (mmap()) three times in three different places
but I don’t use ltrunc(). My driver writes to the shared memory locations
and the PCI card reads the memory and mirrows it to the communication memory
on the card.
I’m also suspecting stack corruption but don’t know how to prove it.
“Mario Charest” <mcharest@nozinformatic.com> wrote in message
news:3adddee5.20388617@inn.qnx.com…
On Wed, 18 Apr 2001 09:57:19 -0400, “Mariola Solodko”
mariola.m.solodko@ca.abb.com> > wrote:
Common mistake with thread is to use printf in each thread. In my
experience that can cause problem because file descriptor are
not thread safe.
thanks Rennie, I was using Kevin’s computer to post these messages but
I’m
not Kevin, he seems to know you though.
I am still at the same point, I don’t think I have a heap corruption, I
was
running some utilities to check the heap and it looked ok, I’ve checked
every single calloc(), memset(), memcpy() and strcpy(). The code I’m
working with is a working code for a different operating system and the
additions I’ve made were to make it compatible with QNX. It’s a PCI
driver, uses a interrupt handler, semaphores and shared memory. The
program
works fine if I create all the threads at the start up.
I’ve checked my resources when running the program and I’m not exceeding
any.
I ran out of ideas, I will have make additions to this program to support
redundancy and maybe then something will come up.
thanks again
“Rennie Allen” <> RAllen@csical.com> > wrote in message
news:> D4907B331846D31198090050046F80C903C80B@exchangecal.hq.csical.com> …
I am not sure how to find the heap corruption problem if it exists.
I think there is a debugging malloc library for QNX4 (there is one for
QNX6) on the website.
I am
allocating thread stack first, using calloc() then I call
_beginthread
but
I’ve tried it both ways and I get the same results.
Try static allocation of the stack:
eg.
stack[NUM_THREADS][2048];
probably not what you want, but a good diagnostic step.
The other test program
I am trying to run in the background will return an error equal to 11
when
trying to create a thread, which means too many threads.
Yes, but once there is heap (or stack) corruption, it isn’t unlikely to
see meaningless error returns. You could check to make sure you
haven’t
exhausted process entries (maybe you are allocating a lot of proxies
?).
I am working with
a huge program and trying to go through every single memcpy and
strcpy
is
impossible but I might have to do that. Any suggestions?
Look for callocs/mallocs/memcpy/strcpy assignments where
sizeof() is being user rather than sizeof(*variable). If it
is
a huge program there is a chance that a variable pointer was wrongly
declared as some type of structure, and (at the same time as the first
mistake) the calloc for the pointer was done as a sizeof(). When the programmer later tried to access a member that
wasn’t part of the wrongly assigned type they couldn’t compile, and
noticed their error, however when they fixed it they didn’t update the
calloc type. If the sizeof() the correct type is bigger than the
sizeof() the initial (wrong) type, then heap corruption will occur (the
same logic applies to memcpy’s but in reverse - i.e. if the correct
type
is smaller than the wrong type, then heap corruption occurs).
Hope this helps Kevin
\
On 19 Apr 2001 15:16:53 GMT, Wojtek Lerch <wojtek@qnx.com> wrote:
Mario Charest <> mcharest@nozinformatic.com> > wrote:
On Wed, 18 Apr 2001 09:57:19 -0400, “Mariola Solodko”
mariola.m.solodko@ca.abb.com> > wrote:
Common mistake with thread is to use printf in each thread. In my
experience that can cause problem because file descriptor are
not thread safe.
Are you sure?
No 
I was under the impression that they are, provided that you use
_beginthread() rather than raw tfork().
My understanding is function using FILE * are thread safe but the file
descriptor itself isn’t.
–
Wojtek Lerch (> wojtek@qnx.com> ) QNX Software Systems Ltd.
Mario Charest <mcharest@nozinformatic.com> wrote:
On 19 Apr 2001 15:16:53 GMT, Wojtek Lerch <> wojtek@qnx.com> > wrote:
Mario Charest <> mcharest@nozinformatic.com> > wrote:
…
Common mistake with thread is to use printf in each thread. In my
experience that can cause problem because file descriptor are
not thread safe.
…
I was under the impression that they are, provided that you use
_beginthread() rather than raw tfork().
My understanding is function using FILE * are thread safe but the file
descriptor itself isn’t.
I have no idea what you mean but that distinction. To me, the concept
of being thread-safe applies to functions, but not to data structures…
Anyway, from a few experiments I did on my machine, it seems that even
though the putc() and getc() macros are not thread-safe (which is easy
to guess from looking at their implementation in stdio.h), other
functions (like printf()) are trying to use a semaphore to make
themselves thread-safe.
According to WD, those functions use two function pointers called
_AccessFileH and _ReleaseFileH. Initially, these variables point to an
empty function, but the first call to _beginthread() sets them to point
to real functions called __AccessFileH() and __ReleaseFileH() that do
some semafore stuff.
Of course, this can only work reliably if you’re not using the
thread-unsafe getc() or putc()…
–
Wojtek Lerch (wojtek@qnx.com) QNX Software Systems Ltd.
Wojtek Lerch <wojtek@qnx.com> wrote:
Mario Charest <> mcharest@nozinformatic.com> > wrote:
…
My understanding is function using FILE * are thread safe but the file
descriptor itself isn’t.
I have no idea what you mean but that distinction. To me, the concept
^^^
“by”
of being thread-safe applies to functions, but not to data structures…
…
–
Wojtek Lerch (wojtek@qnx.com) QNX Software Systems Ltd.
According to WD, those functions use two function pointers called
_AccessFileH and _ReleaseFileH. Initially, these variables point to an
empty function, but the first call to _beginthread() sets them to point
to real functions called __AccessFileH() and __ReleaseFileH() that do
some semafore stuff.
Any idea why calling the _beginthread after I have about 4 other threads
created (use semaphores a lot) does not create a thread ?
How many semaphores are in use system wide? The default Proc options will
support a max of 125. Can you make a thread that does not require any
semaphores?