Take the 2-minute tour ×

We have an application where we send data through TCP sockets. We use 8 TCP connections for this. The socket send and receive is called in a background thread. There is just one thread which iterates over the array of sockets to send data through all of them (sequential).

The code in sender thread is something like:


    nBytesWrriten = send (tcpsock[i], data2, nleft, 0))
    //error handling and process more data


and the receiver thread is like:


    sz[i] = recv (tcpsock[i], data, MAX_UDT_SIZE, 0);
    //process data

Everything works fine and the data gets transferred, but sometimes it just takes too long. On checking logs, I found that in most cases, the sender thread works just fine , but sometimes, there is a huge delay in timestamps(sometimes more than a second) before and after 'send' call.

All of the send and receive action is taking place in a worker thread. Is it something to do with pre-emption of the thread just before/on send call? Can I avoid the pre-emption of the thread just before the send call? Or is it that the receiver thread has not received the data on the socket while send it ready with more data, and therefore it causes the delay?

How do I optimize this as it is taking too long to send data?


share|improve this question
You can use tools like Wireshark to check if there are any delays on a lower level than your application. –  Default Nov 12 '12 at 13:05
Are your sockets no-blocking (O_NONBLOCK)? If not, the send() call won't return until all the data has been accepted by the local TCP stack (which is only somewhat related to what goes out on the wire and may be immediate or may take some time). Including a wireshark/tcpdump trace might help. –  Brian White Nov 12 '12 at 14:59

2 Answers 2

You should use non-blocking sockets for sending. What might be happening is that one (or more than one) cannot send right away, so it waits until it can send some data, maybe buffers full or whatever.

Using non-blocking sockets it won't stop, but you must check that the data was not sent to some sockets and try again later.

share|improve this answer
but he said that the sockets are in background threads –  Default Nov 12 '12 at 13:26
@Default and he said the send delays (block) and is only one thread. –  aaronps Nov 12 '12 at 13:27

Do a select on each socket before the send to see if you can write without blocking, otherwise you'll block the sends on the other sockets. You'll want to do the same thing on the read side or the lack of readability of one could block available read on others.

share|improve this answer
From a design perspective, what you ought to do is do a select on ALL of the sockets at once, on both ends, and do your loop writes and reads on each that is ready. That would prevent spin-polling if all sockets are not ready... –  mark Nov 12 '12 at 13:34

Your Answer


