I have a couple of large programs that call strtok(). Strtok() is
always returning NULL, regardless of the input. When I try to create
a small test application, the test works correctly, but the large apps
continue to fail. The same large apps calling strtok_r() work
correctly. Both apps work correctly under QNX4 and Linux.
This happens with Patch A. Has anybody else seen this behaviour?
Andrew
Previously, Alain Magloire wrote in qdn.public.qnxrtp.os:
Andrew Thomas <> Andrew@cogent.ca> > wrote:
: I have a couple of large programs that call strtok(). Strtok() is
: always returning NULL, regardless of the input. When I try to create
: a small test application, the test works correctly, but the large apps
: continue to fail. The same large apps calling strtok_r() work
: correctly. Both apps work correctly under QNX4 and Linux.: This happens with Patch A. Has anybody else seen this behaviour?
char *
strtok (char *s1, const char *s2)
{
static char *word;
return strtok (s1, s2, &word);
}if you say strtok_r () works and not strtok () maybe there is more to it.
I put this in a common library that both programs use:
#ifdef BROKEN_STRTOK
#define strtok(c,d) strtok_ok(c,d)
extern char* strtok_ok (char* c, const char* d);
#endif
#ifdef BROKEN_STRTOK
char* strtok_ok (char* str, const char* delim)
{
static char *last;
char *out;
… and it works perfectly. I reported a case where GCC is producing
erroneous code in qdn.public.qnxrtp.devtools that actually might cause
strtok() to fail but strtok_r() to succeed. It appears that with -O
specified, GCC can produce code that trashes registers, and neither
the caller nor the receiver takes reponsibility for PUSHing and POPing
them. Want to see some code?
Cheers,
Andrew
Previously, Igor Kovalenko wrote in qdn.public.qnxrtp.os:
Somebody said there is some internal limit on string size imposed by
libc, which would be removed by Dinkum libc when it is out…
One of my failure cases is:
strtok (“1:1”, “:”);
This returns NULL.
I’m going to go out on a limb here and guess that i haven’t breached
very many internal limits. 
Cheers,
Andrew
Previously, Andrew Thomas wrote in qdn.public.qnxrtp.os:
… and it works perfectly. I reported a case where GCC is producing
erroneous code in qdn.public.qnxrtp.devtools that actually might cause
strtok() to fail but strtok_r() to succeed. It appears that with -O
specified, GCC can produce code that trashes registers, and neither
the caller nor the receiver takes reponsibility for PUSHing and POPing
them. Want to see some code?
My strtok bug still exists, but the GCC generating bad code was a
false alarm. Code optimizers are sneakier than people. 
Andrew
Andrew Thomas <Andrew@cogent.ca> wrote:
: I have a couple of large programs that call strtok(). Strtok() is
: always returning NULL, regardless of the input. When I try to create
: a small test application, the test works correctly, but the large apps
: continue to fail. The same large apps calling strtok_r() work
: correctly. Both apps work correctly under QNX4 and Linux.
: This happens with Patch A. Has anybody else seen this behaviour?
char *
strtok (char *s1, const char *s2)
{
static char *word;
return strtok (s1, s2, &word);
}
Aussi haut que l’on soit assis, on n’est toujours assis que sur son cul !!!
Somebody said there is some internal limit on string size imposed by
libc, which would be removed by Dinkum libc when it is out…
Alain Magloire wrote:
Andrew Thomas <> Andrew@cogent.ca> > wrote:
: I have a couple of large programs that call strtok(). Strtok() is
: always returning NULL, regardless of the input. When I try to create
: a small test application, the test works correctly, but the large apps
: continue to fail. The same large apps calling strtok_r() work
: correctly. Both apps work correctly under QNX4 and Linux.: This happens with Patch A. Has anybody else seen this behaviour?
char *
strtok (char *s1, const char *s2)
{
static char *word;
return strtok (s1, s2, &word);
}if you say strtok_r () works and not strtok () maybe there is more to it.
–
au revoir, alainAussi haut que l’on soit assis, on n’est toujours assis que sur son cul !!!
Previously, Alain Magloire wrote in qdn.public.qnxrtp.os:
Alain Magloire <> alain@qnx.com> > wrote:
: Andrew Thomas <> Andrew@cogent.ca> > wrote:
: : I have a couple of large programs that call strtok(). Strtok() is
: : always returning NULL, regardless of the input. When I try to create
: : a small test application, the test works correctly, but the large apps
: : continue to fail. The same large apps calling strtok_r() work
: : correctly. Both apps work correctly under QNX4 and Linux.: : This happens with Patch A. Has anybody else seen this behaviour?
: char *
: strtok (char *s1, const char *s2)
: {
: static char *word;
: return strtok (s1, s2, &word);
^^^^^^^^^^^
hum … it should be strtok_r ().
The typo was obvious but, you never somebody cut&pasting carelessly
Are you saying that strtok() in the library actually has a typo and is
therefore buggy, or are you saying that the posting had a typo and the
library is fine?
Andrew
Alain Magloire <alain@qnx.com> wrote:
: Andrew Thomas <Andrew@cogent.ca> wrote:
: : I have a couple of large programs that call strtok(). Strtok() is
: : always returning NULL, regardless of the input. When I try to create
: : a small test application, the test works correctly, but the large apps
: : continue to fail. The same large apps calling strtok_r() work
: : correctly. Both apps work correctly under QNX4 and Linux.
: : This happens with Patch A. Has anybody else seen this behaviour?
: char *
: strtok (char *s1, const char *s2)
: {
: static char *word;
: return strtok (s1, s2, &word);
^^^^^^^^^^^
hum … it should be strtok_r ().
The typo was obvious but, you never somebody cut&pasting carelessly
: }
: if you say strtok_r () works and not strtok () maybe there is more to it.
Aussi haut que l’on soit assis, on n’est toujours assis que sur son cul !!!
Andrew Thomas <Andrew@cogent.ca> wrote:
: Previously, Andrew Thomas wrote in qdn.public.qnxrtp.os:
:> … and it works perfectly. I reported a case where GCC is producing
:> erroneous code in qdn.public.qnxrtp.devtools that actually might cause
:> strtok() to fail but strtok_r() to succeed. It appears that with -O
:> specified, GCC can produce code that trashes registers, and neither
:> the caller nor the receiver takes reponsibility for PUSHing and POPing
:> them. Want to see some code?
: My strtok bug still exists, but the GCC generating bad code was a
: false alarm. Code optimizers are sneakier than people.
The strtok() example in the other post was to point that
the symptoms were(may be) misleading. Writing strtok() is trivial and
I do not think that the library strtok() is the fault.
You could take the one in glibc or *BSD, to confirm, but this
may change the memory layout and make it crash somewhere else.
Aussi haut que l’on soit assis, on n’est toujours assis que sur son cul !!!
Andrew Thomas <Andrew@cogent.ca> wrote:
: Previously, Alain Magloire wrote in qdn.public.qnxrtp.os:
:> Alain Magloire <alain@qnx.com> wrote:
:> : Andrew Thomas <Andrew@cogent.ca> wrote:
:> : : I have a couple of large programs that call strtok(). Strtok() is
:> : : always returning NULL, regardless of the input. When I try to create
:> : : a small test application, the test works correctly, but the large apps
:> : : continue to fail. The same large apps calling strtok_r() work
:> : : correctly. Both apps work correctly under QNX4 and Linux.
:>
:> : : This happens with Patch A. Has anybody else seen this behaviour?
:>
:> : char *
:> : strtok (char *s1, const char *s2)
:> : {
:> : static char *word;
:> : return strtok (s1, s2, &word);
:> ^^^^^^^^^^^
:> hum … it should be strtok_r ().
:> The typo was obvious but, you never somebody cut&pasting carelessly
: Are you saying that strtok() in the library actually has a typo and is
: therefore buggy, or are you saying that the posting had a typo and the
: library is fine?
My code was buggy. The lib is fine.
Aussi haut que l’on soit assis, on n’est toujours assis que sur son cul !!!