Channels ▼


Use Threads Correctly = Isolation + Asynchronous Messages

Example 2: Sockets

Another classic example of event-driven code is client-server communications. A server listens for incoming connection requests; as each connection arrives, the server has to accept it and then process inbound and outbound traffic on that connection until the connection is closed by one side or the other. At any given time, the server has to be able to deal with multiple active connections from different clients, each at a different stage in its processing.

As with the previous GUI example, we have the option of handling the work synchronously or asynchronously. The synchronous code looks something like this in a classic socket-oriented API. The methods are spelled a little differently in POSIX, Java, and other environments, but all follow the basic operation.

// Example 2: Doing it all on one thread/process
// (pseudocode, and assumes select returns single events)
while( /*... */ ) {
   select( our socket list ); // blocks until something is ready
   if( a connection request came in )
      accept() and add it into to our socket list
   else if( a connection closed )
      close() and remove it from our socket list
   else if( one of our read sockets is ready )
                           // synchronous processing
      read more data from that socket and process it
   else if( one of our write sockets is ready )
                           // synchronous processing
   write the next chunk of data to that socket

A key problem with this is that all the work is being handled by a single thread. For one thing, that tends to make the code more complex; one thread has to do it all, so all the code to handle all the different kinds of clients appears in one place. For another, running all the work on one thread means that all the work will run sequentially on one core (or, equivalently, one core at a time if the operating system migrates the thread from core to core). If the total work saturates the core, then additional work will be throttled and slow down the server's responsiveness for all clients as later requests queue up and wait for earlier work to complete.

A common solution is to use Option 2: Let the server service connections simultaneously by launching a new thread to handle each incoming client. Here's the basic logic:

// Applying Option 2 (pseudocode)
// Listener thread: Only accepts connections
// and launches new threads to deal with them

while( /*…*/ ) {
   wait for connection request
   ... = new Thread( [] { HandleIt( theConnection) ); } );
// Each handler thread
void HandleIt( ... ) {
   while( wait for event on this connection ) {
                       // this could block
      if( this connection was closed ) {
      else if( incoming data ready to read )
                    // synchronous processing
        read more data from this connection and process it
      else if( connection is ready to write )
                   // synchronous processing
        write the next chunk of data to this connection

This code is both clearer and potentially faster. The program is clearer because the code to handle each kind of client is nicely wrapped up in its own method or class, instead of being intermingled. The program is faster because it keeps different connections asynchronous and independent, so that the work on one connection doesn't have to wait to be processed sequentially behind work on another connection. This gives better responsiveness even on a single-core server that isn't heavily loaded, and it delivers better scalability under load on servers that do have parallel hardware.

Related Reading

More Insights

Currently we allow the following HTML tags in comments:

Single tags


<br> Defines a single line break

<hr> Defines a horizontal line

Matching tags


<a> Defines an anchor

<b> Defines bold text

<big> Defines big text

<blockquote> Defines a long quotation

<caption> Defines a table caption

<cite> Defines a citation

<code> Defines computer code text

<em> Defines emphasized text

<fieldset> Defines a border around elements in a form

<h1> This is heading 1

<h2> This is heading 2

<h3> This is heading 3

<h4> This is heading 4

<h5> This is heading 5

<h6> This is heading 6

<i> Defines italic text

<p> Defines a paragraph

<pre> Defines preformatted text

<q> Defines a short quotation

<samp> Defines sample computer code text

<small> Defines small text

<span> Defines a section in a document

<s> Defines strikethrough text

<strike> Defines strikethrough text

<strong> Defines strong text

<sub> Defines subscripted text

<sup> Defines superscripted text

<u> Defines underlined text

