I have a question about TCP timing.
I have an application using TCP in QNX4.25, shuttling lots of streaming
data from local hardware to the network. Normally the TCP driver takes
my data, which consists of lots of relatively small messages,
concatenates them and transmits them quite quickly, not usually waiting
for an ACK from the receiver.
The receiver’s window size is about 60K, and we are sending typically
30-120 KB/sec. The MSS is 1460.
When the data stops for one second or more then restarts here is what
happens:
- The first message only (e.g. small message, about 35 bytes) is sent on
the network - The QNX machine waits for an ACK before sending any more data, which
usually takes about 100 ms. - In the meantime the application continues to send messages, but they
often fill the buffer up before the ACK is received. - When the ACK is finally received the buffered messages get sent, and
the system works fine from that point on.
(Note: This does NOT happen when the silence period is less than one
second)
I believe what is happening is that the driver is timing out during the
period of inactivity and “forgetting” the window sent in the last ACK
from the receiver. Therefore the driver only attempts to send a small
message, fully anticipating an ACK sometime later to inform it of the
actual window size. Unfortunately for our application we often don’t
have that 100 ms. to spare.
We suspect that maybe the Nagle algorithm is working here, but it does
not fit the Nagle algorithm description completely. Although the Nagle
algorithm does indicate that subsequent data is not sent until an ACK
arrives, my inerpretation is that once enough data is received by the
TCP driver to fill a full TCP packet that the packet is sent immediately
and an ACK is not waited for.
Here is what I have tried:
- Set the TCP_NODELAY option. No effect.
- Increase the send buffer size to the max. This works for most
situations, but in some cases the buffer still fills up. - Buffer individual messages in the application for a large atomic send
to the network, thinking that this might defeat the Nagle algorithm by
presenting a message large enough for a full TCP packet. This did not
work.
Does anyone have:
- A better explanation of what is going on here?
- A better suggestion for overcoming this problem?
Thanks
Matt Schrier