![]() |
Winsock Programmer's FAQ |
![]() |
Which I/O Strategy Should I Use?by Warren Young There are several different conventions for communicating with Winsock, and each method has distinct advantages. The question of the hour is, what are these advantages, and how does someone choose the convention that makes the most sense for their application? The choices are:
Further confusing the issue are threads, because each of the above mechanisms changes in nature when used with threads. In trying to find an answer to the "which I/O strategy" question,
it becomes apparent that there are only a few major kinds of programs,
and the successful ones follow the same patterns. From those patterns
and practical experience The heuristics are ordered in terms of compatibility, then speed, and finally functionality. Compatibility is first, because if a given I/O strategy won't work on the platforms you need to support, it doesn't matter how fast or functional it is. Speed is next because performance requirements are easy to determine, and often important. Functionality is last, because once you decide the compatibility and speed issues, your choices become much more subjective. Heuristic 1: Narrow your choices by deciding which operating systems you need to support.There are currently two major varieties of Windows, which I call Win9x and WinNT. "Win9x" currently includes Windows 95, Windows 98 and Windows ME. "WinNT" currently includes Windows NT 4.0, Windows 2000 and Windows XP. These two major Windows varieties are "upwards compatible" within the series. For example, if Windows NT 4.0 has a given feature, that feature will also work on Windows 2000 and Windows XP. This article treats all operating systems outside these two series separately. Specifically: Windows NT 3.x, Win16, Windows CE, Unix, and all the non-Windows platforms that support Winsock. You may also need to support Unix. By "Unix", I mean operating systems with BSD sockets and POSIX threads (a.k.a. "pthreads"). In addition to the traditional Unixes, for the purposes of this article you can consider Linux, MacOS X, QNX, and BeOS to all be "Unixes". Given this wide array of operating systems under consideration, it is no surprise that there are several I/O strategies to choose from. Most of these varieties are Winsock's extensions to BSD sockets, added to take advantage of new Windows features.
Heuristic 2: Avoid pure non-blocking sockets.Pure non-blocking sockets are almost never necessary, and
a good thing, too: their inefficiency makes them a poor architecture
choice for Windows programs. (By "pure non-blocking", I mean using a
non-blocking socket without also using one of the "select" functions:
When a socket is set as non-blocking, every Winsock call on that socket will return immediately, whether it was able to do anything or not. This is useful because it lets your program do other things while the network is busy. Most programs don't have something to do all the time: they're usually waiting on user input, or the network, or some other slow thing. For this reason, Winsock provides the "select" functions. The three functions vary in how they work, but they either put your program to sleep while waiting for network I/O so you don't waste CPU time, or they let your program return to its GUI event loop until something happens on one of the program's sockets. Heuristic 3: Avoid select().I mentioned that the About the only time you should use Heuristic 4: Avoid asynchronous sockets in programs that must deal with high volumes of data.Window messages are the slowest way (aside from Heuristic 5: For high-performance servers, prefer overlapped I/O.Of all the various I/O strategies, overlapped I/O has the highest performance. (I/O completion ports are even more efficient, but are nonstandard vis-a-vis Winsock proper, so I don't cover them in the FAQ.) With careful use of overlapped I/O (and boatloads of memory in the server!) you can support tens of thousands of connections with a single server. No other I/O strategy comes close to the scalability of overlapped I/O. Heuristic 6: To support a moderate number of connections, consider asynchronous sockets and event objects.If your server only has to support a moderate number of
connections Programmed correctly, asynchronous sockets are a reasonable choice for a dedicated server supporting a moderate number of connections. The main problem with doing this is that many servers don't have a user interface, and thus no message loop. A server without a UI using asynchronous sockets would have to create an invisible window solely to support its asynchronous sockets. If your program already has a user interface, though, asynchronous sockets can be the least painful way to add a network server feature to it. Another reasonable choice for handling a moderate number of connections is event objects. These are very efficient in and of themselves. The main problem you run into with them is that you cannot block on more than 64 event objects at a time. To block on more, you need to create multiple threads, each of which blocks on a subset of the event objects. Before choosing this method, consider that handling 1024 sockets requires 16 threads. Any time you have many more active threads than you have processors in the system, you start causing serious performance problems. Thus, call 1024 sockets a hard practical limit. One caution: it's very easy to underestimate the number of simultaneous connections you will get on a public Internet server. It may make sense to design for massive scalability even if your estimates don't currently predict thousands of simultaneous clients. On the other hand, it's becoming clear that usable-but-weak code today always beats wonderful code next month. Heuristic 7: Low-traffic servers can use most any I/O strategy.For low-traffic servers, there isn't much call to be super-efficient.
Some servers just don't have to support very many connections, and
if you're deploying on Win9x you're already going to be limited to 100 sockets at a time. Suitable
strategies for 1-100 connections are event objects, non-blocking sockets
with We've covered the first three methods already, so let's consider threads with blocking sockets. This is often the simplest way by far to write a server. You just have a main loop that accepts connections and spins each new connection off to its own thread, where it's handled with blocking sockets. Blocking sockets have several advantages. They are efficient, because when a thread blocks, the operating system immediately lets other threads run. Also, synchronous code is more straightforward than equivalent non-synchronous code. There are two main problems with this method. First, threads often require a lot of synchronization work, which is hard to get right; this may outstrip the benefits of using blocking sockets. Second, threads don't scale well at all. Recall the discussion of event objects: if the number of active threads outnumbers the number of processors in the system to a great degree, you run into efficiency problems. So, this method is only suitable for a fairly small number of connections, or a moderate number of connections that are mostly idle. Heuristic 8: Do not block inside a user interface thread.This heuristic sounds more like a straightforward rule of Windows programming, but I bring it up because most programs are single-threaded. In a single-threaded GUI program, any time you call a Winsock function that blocks the UI thread, buttons can't be pressed, menus won't pull down, scroll bars won't move, keypresses are ignored...your UI freezes. Heuristic 9: For GUI client programs, prefer asynchronous sockets.There are two reasons for this heuristic:
Heuristic 10: Threads are rarely helpful in client programs.When a programmer first learns about threads, he is eager to try them out in his own programs. He sees that they have several advantages, but he doesn't yet see the drawbacks. Unfortunately for the soon-to-be-educated newbie, these drawbacks can have very significant consequences. One real benefit of threads is that a thread doing I/O on a blocking socket has a linear control flow, and is therefore easier to understand. Asynchronous code is more spread out, so it is harder to write and debug. Another perceived benefit of threads is a kind of encapsulation: a programmer can split a program up into a number of threads, each of which has a single well-defined task. But, this is only valid if each thread is mostly independent from the rest of the program. If not, the threads will have to share data through a common data structure, destroying any potential encapsulation. In the end, the biggest problem with threads is also related to shared data structures: synchronization. This issue is covered better elsewhere, so I won't spend many words on it here. In short, synchronization is hard to get right: poorly-synchronized threads are subject to serialization delays, context switching overhead, deadlocks, race conditions and corrupted data. These are hard problems, and for most programs the benefits are not large enough to make them worth overcoming. A saner alternative is to use asynchronous I/O. This buys you the
synchronization benefits described in the previous heuristic. You can
even partition the application in a similar manner to threads by creating
an invisible window for each socket. If you have two different types of
sockets, each socket can have its notifications sent to a different type
of window. In straight API terms it means a separate Heuristic 11: Use threads only when their effect on the rest of the program is easily contained.The previous heuristic cautions that threads are often very hard to program correctly, but the truth is that they are sometimes very useful. You can make an educated guess about whether threads will improve the program by doing a bit of design work: is there a clean interface between each thread and the rest of the program? If so, synchronization becomes simple. If not, you're going to end up with a mess that crashes and destroys data unpredictably. Examples where threads are viable are:
Heuristic 12: Design around your protocol.Some network protocols are inherently synchronous, and others are not. An example of a synchronous protocol is the POP3 e-mail protocol: send a user name, get a response, send a password, get a response, send a request to get the list of emails, get a response... With POP, you have to send these commands in a specific order: you can't send the password before the user name, and you can't get the list of emails without sending the user name and password. Writing a POP client with a non-synchronous socket type would require also writing a state machine. On the other hand, if your protocol is non-synchronous, you might as well use non-synchronous sockets. Non-synchronous protocols tend to resemble a set of function calls. Consider, for example, a program to retrieve data from a networked SQL database: send a SQL statement, and retrieve the result set. At the end of each "function call", the program is back to its original state: you don't need to maintain a state machine to keep track of where you are in the protocol. Heuristic 13: Blocking sockets are simpler, non-blocking sockets are more powerful.This heuristic is almost a restatement of all the above material. It just bears repeating that, while blocking sockets are attractive for their simplicity, you may find that their disadvantages eventually force you to redesign your program to use some form of non-blocking sockets. This is especially true if your program will be supporting more than one socket. (Virtuall all server programs fall into this category.) The only reasonable way to use multiple blocking sockets at once is to use threads, but with non-blocking sockets, you have many more design options. ConclusionIt is my hope that you find these heuristics helpful. Although you may not agree with each of them, I think that they will at least make you think about your own choices. Design is a highly subjective enterprise, and this list is based mainly on my own thoughts and preferences. Special thanks go to Philippe Jounin for his comments on the 1998 version of this paper. The 2000 version reflects my greater experience, as well as commentary from David Schwartz and Alun Jones, both of whom expanded my ideas of the proper way to build a Winsock server. Copyright © 1998-2001 by Warren Young. All rights reserved. |
<< Winsock for the Impatient | Effective TCP/IP >> |
Last modified on 19 October 2001 at 15:32 UTC-7 | Please send corrections to tangent@cyberport.com. |
< Go to the main FAQ page | << Go to the Home Page |