I was using popen to start a daemon in the background and using fgets to
read the output (which there is none).
FILE * file=popen(p_cmd,“r”);
if (file) {
while (fgets(p_out,out_size,file) != NULL) {
printf("%s", p_out);
} // while
rc=pclose(file);
if (rc != -1) // if not shell error, get command rc
rc=WEXITSTATUS(rc);
} // if file
The problem is that the fgets hangs until I kill the daemon I started
(p_out is “mydaemon &”). I guess this is expected behaviour?
Joseph Witherspoon wrote:
I was using popen to start a daemon in the background and using fgets to
read the output (which there is none).
FILE * file=popen(p_cmd,“r”);
if (file) {
while (fgets(p_out,out_size,file) != NULL) {
printf("%s", p_out);
} // while
rc=pclose(file);
if (rc != -1) // if not shell error, get command rc
rc=WEXITSTATUS(rc);
} // if fileThe problem is that the fgets hangs until I kill the daemon I started
(p_out is “mydaemon &”). I guess this is expected behaviour?
Yes. The fgets() will block reading from the pipe set up from the
command output, and will keep blocking/reading until EOF (typically when
the pipe is empty and the process closes the write end, perhaps by
terminating). What are you trying to achieve (I mean, when you popen
you normally want to collect all output from the subprocess, which means
waiting until it exits)? If you only want initialisation errors you may
be able to have your daemon do a “fclose(stdout)” once it has started
(if you’re sure it will never try to output again), which should let
your fget unblock … [just a guess as to what you’re trying to do]
Yes, this happens in Linux too, although in Linux if the daemon
takes the trouble to close its standard output before sleeping or
whatever, the fgets will unblock. In both QNX and Linux, however,
blocking will occur on the pclose until the daemon exits, even if
there has been no attempt to read anything from it.
I have to admit I prefer the Linux behaviour on the fgets. (John
Garvey’s message has just shown up, and I think he must also
prefer it, as he seems to assume that’s what happens in Neutrino
too, though it actually doesn’t!)
For the pclose, it is a pity there is no way defined in POSIX
(that I know of) to instruct the system whether to
Note that it is not easy, short of implementing your own popen and
pclose, to send a signal to a subprocess whose pid you probably
don’t know…or at least I don’t know an easy POSIX-defined way
to find out a subprocess pid from a pipe stream. Otherwise choice
#2 would be redundant, as you could just send your own signal
before the pclose.
dB
jspoon@us.ibm.com (Joseph Witherspoon) wrote ~ 23 Jun 2004 17:37:01 GMT:
I was using popen to start a daemon in the background and using fgets to
read the output (which there is none).
FILE * file=popen(p_cmd,“r”);
if (file) {
while (fgets(p_out,out_size,file) != NULL) {
printf("%s", p_out);
} // while
rc=pclose(file);
if (rc != -1) // if not shell error, get command rc
rc=WEXITSTATUS(rc);
} // if fileThe problem is that the fgets hangs until I kill the daemon I started
(p_out is “mydaemon &”). I guess this is expected behaviour?
David Bacon wrote:
Yes, this happens in Linux too, although in Linux if the daemon
takes the trouble to close its standard output before sleeping or
whatever, the fgets will unblock. In both QNX and Linux, however,
blocking will occur on the pclose until the daemon exits, even if
there has been no attempt to read anything from it.
I have to admit I prefer the Linux behaviour on the fgets. (John
Garvey’s message has just shown up, and I think he must also
prefer it, as he seems to assume that’s what happens in Neutrino
too, though it actually doesn’t!)
He did only say he wanted to unblock the fgets(), which is what closing
the write end of the pipe will do (or are you trying to say it
doesn’t?!). Yes, plcose() will do a waitpid(), which is blocking (but
at this point we are beyond the fgets (the original question); you can
also stack popen calls, so there is no/little harm to not pclose() this
one instantiation if necessary). Then it also depends on what the
“daemon” does; if it makes a suitable procmgr_daemon() call (eg, without
PROCMGR_DAEMON_NOCLOSE) then that will both unblock the fgets() and the
waitpid() of pclose. I tested QNX6.3 with this example below (where
popen exits immediately, not after 30 seconds). So I fail to see what
you are saying or assuming or what “it actually doesn’t” means?
— popen.c —
int main(int argc, char *argv[])
{
FILE *file;
int rc;
char out[128];
file = popen("/home/jgarvey/testcode/daemon &",“r”);
while (fgets(out,sizeof(out),file) != NULL) {
printf("%s", out);
}
rc = pclose(file);
printf(“result = %04x\n”, rc);
return(0);
}
— daemon.c —
int main(int argc, char *argv[])
{
procmgr_daemon(0, 0);
sleep(30);
return(0);
}
The salient difference between my test program and yours is that
I didn’t “background” the subprocess with an ampersand. While I
must say that probably fits the description of a “daemon” process
best, the ampersand kind of makes the example trivial with respect
to the fgets(), because the direct subprocess of the top program
then of course exits immediately, while its child continues.
What I found interesting (and this is what the “it actually doesn’t”
was about) was that if a non-ampersanded subprocess like your daemon
(but without the procmgr_daemon() call) closes its stdout and then
sleeps, the Linux implementation of fgets() returns immediately,
while the Neutrino one doesn’t, in either 6.2.1B or 6.3.0. My
feeling is that Neutrino should do this too.
dB
John Garvey wrote ~ Thu, 24 Jun 2004 11:51:00 +1200:
David Bacon wrote:
Yes, this happens in Linux too, although in Linux if the daemon
takes the trouble to close its standard output before sleeping or
whatever, the fgets will unblock. In both QNX and Linux, however,
blocking will occur on the pclose until the daemon exits, even if
there has been no attempt to read anything from it.
I have to admit I prefer the Linux behaviour on the fgets. (John
Garvey’s message has just shown up, and I think he must also
prefer it, as he seems to assume that’s what happens in Neutrino
too, though it actually doesn’t!)He did only say he wanted to unblock the fgets(), which is what closing
the write end of the pipe will do (or are you trying to say it
doesn’t?!). Yes, plcose() will do a waitpid(), which is blocking (but
at this point we are beyond the fgets (the original question); you can
also stack popen calls, so there is no/little harm to not pclose() this
one instantiation if necessary). Then it also depends on what the
“daemon” does; if it makes a suitable procmgr_daemon() call (eg, without
PROCMGR_DAEMON_NOCLOSE) then that will both unblock the fgets() and the
waitpid() of pclose. I tested QNX6.3 with this example below (where
popen exits immediately, not after 30 seconds). So I fail to see what
you are saying or assuming or what “it actually doesn’t” means?— popen.c —
int main(int argc, char *argv[])
{
FILE *file;
int rc;
char out[128];file = popen("/home/jgarvey/testcode/daemon &",“r”);
while (fgets(out,sizeof(out),file) != NULL) {
printf("%s", out);
}
rc = pclose(file);
printf(“result = %04x\n”, rc);return(0);
}— daemon.c —
int main(int argc, char *argv[])
{
procmgr_daemon(0, 0);
sleep(30);
return(0);
}
In article <cbepro$5cp$1@inn.qnx.com>, Wojtek Lerch <Wojtek_L@yahoo.ca> wrote:
David Bacon wrote:
Actually, it’s not about fgets() at all. It’s about the shell. Under
QNX, even if your daemon closes the pipe, the shell keeps its stdout
open, and that’s what’s preventing fgets() from returning. Under Linux,
the shell execs into the daemon, which makes the daemon’s stdout the
only fd pointing to the writable end of the pipe.I just made an experiment that confirmed it:
$ sh -c ‘ps -p$$’
Under QNX, this reports the shell. Under Red Hat, it reports ps.
As far as I know, POSIX never mentions that what Linux does is allowed.
It’s not QNX vs Linux - it’s ksh vs bash. Do:
Brian Stecher (bstecher@qnx.com) QNX Software Systems, Ltd.
phone: +1 (613) 591-0931 (voice) 175 Terence Matthews Cr.
+1 (613) 591-3579 (fax) Kanata, Ontario, Canada K2M 1W8
David Bacon wrote:
What I found interesting (and this is what the “it actually doesn’t”
was about) was that if a non-ampersanded subprocess like your daemon
(but without the procmgr_daemon() call) closes its stdout and then
sleeps, the Linux implementation of fgets() returns immediately,
while the Neutrino one doesn’t, in either 6.2.1B or 6.3.0. My
feeling is that Neutrino should do this too.
Actually, it’s not about fgets() at all. It’s about the shell. Under
QNX, even if your daemon closes the pipe, the shell keeps its stdout
open, and that’s what’s preventing fgets() from returning. Under Linux,
the shell execs into the daemon, which makes the daemon’s stdout the
only fd pointing to the writable end of the pipe.
I just made an experiment that confirmed it:
$ sh -c ‘ps -p$$’
Under QNX, this reports the shell. Under Red Hat, it reports ps.
As far as I know, POSIX never mentions that what Linux does is allowed.
Brian Stecher wrote:
It’s not QNX vs Linux - it’s ksh vs bash. Do:
ksh -c ‘ps -p$$’ => reports ksh
bash -c ‘pc -p$$’ => reports ps
The part that is about QNX vs Linux is that /bin/sh points to bash under
Linux and to ksh under QNX.
This rings true, and IIRC, POSIX does allow shells some latitude
in deciding whether to hang around waiting for their children or
not. Bash too will hang around if the command line you feed it is
a little more complicated. For example, a simple redirection will
do it:
bash -c ‘ps -p$$ >&2’ => reports bash
dB
bstecher@qnx.com (Brian Stecher) wrote ~ 24 Jun 2004 15:11:38 GMT:
It’s not QNX vs Linux - it’s ksh vs bash. Do:
ksh -c ‘ps -p$$’ => reports ksh
bash -c ‘pc -p$$’ => reports ps
Just to relate this back to the original question, if the
p_cmd in the line
FILE * file=popen(p_cmd,“r”);
begins with the word “exec”, Joseph Witherspoon will have a
portable way of starting the subprocess which, if it closes
its stdout immediately, will not hold up the fgets().
Note that
ksh -c ‘exec ps -p$$’ => reports ps
dB
David Bacon wrote ~ 24 Jun 2004 16:06:38 GMT:
This rings true, and IIRC, POSIX does allow shells some latitude
in deciding whether to hang around waiting for their children or
not. Bash too will hang around if the command line you feed it is
a little more complicated. For example, a simple redirection will
do it:bash -c ‘ps -p$$ >&2’ => reports bash
dB
bstecher@qnx.com > (Brian Stecher) wrote ~ 24 Jun 2004 15:11:38 GMT:
It’s not QNX vs Linux - it’s ksh vs bash. Do:
ksh -c ‘ps -p$$’ => reports ksh
bash -c ‘pc -p$$’ => reports ps
David Bacon wrote:
This rings true, and IIRC, POSIX does allow shells some latitude
in deciding whether to hang around waiting for their children or
not.
Can you point me to where it says that? I looked but didn’t manage to
find it…
No, that’s why the “IIRC”. I’m pretty sure I read it somewhere,
perhaps not in a POSIX document after all, but I do remember that
when I read it, it helped explain an observation.
Sorry for the false claim. I guess if POSIX says nothing about it,
my claim is at least vacuously true, anyway
dB
Wojtek Lerch wrote ~ Thu, 24 Jun 2004 15:02:22 -0400:
David Bacon wrote:
This rings true, and IIRC, POSIX does allow shells some latitude
in deciding whether to hang around waiting for their children or
not.Can you point me to where it says that? I looked but didn’t manage to
find it…
David Bacon wrote:
Sorry for the false claim. I guess if POSIX says nothing about it,
my claim is at least vacuously true, anyway >
Well, not exactly; if the general rule is that an executable must run as
a separate process, and POSIX says nothing about the case in question
being an exception, than it is no exception.
I am reasonably sure that POSIX does not mention an exception; trouble
is, the general rule doesn’t explicitly mention a separate process,
either. But it seems quite clear from the following part that a command
started from the shell should not be able to close an fd that belongs to
the original shell’s environment, doesn’t it:
The environment of the shell process shall not be changed
by the utility unless explicitly specified by the utility
description (for example, cd and umask).
If you want to read all the details:
http://www.opengroup.org/onlinepubs/009695399/utilities/xcu_chap02.html
John Garvey wrote:
Joseph Witherspoon wrote:
What’s really going on here is that someone is calling a program
as a subroutine. QNX has great interprocess communication. Yet
doing this operation with QNX primitives is much harder than
using the classic pipe/string output parsing approach from
the early days of UNIX.
Starting a subprogram and setting up QNX interprocess
communication with it is surprisingly painful under QNX.
We’ve done it, and we had to write a sizable program to
help deal with the problems of establishing parent/child
communication.
It would be useful to have a function like “popen”
which establishes a channel for MsgSend/MsgReceive.
The child process can create a channel easily enough,
but passing it to the child process requires somehow
passing the node ID/process ID/channel ID from the child
to the parent. If there’s an intervening shell process,
this becomes even more complicated, because the parent
needs to know the PID of the real process, not the
shell. Doing this across nodes is even harder, because
of a previously reported problems in io-net.
Using connect_attach for parent/child communication
is “too global”; you get into namespace and security issues.
A great strength of QNX is the interprocess messaging.
But there’s a big price of entry to use it. That could
be reduced.
John Nagle
Team Overbot