Four Reasons to Use Multithreading

6 Apr 2007

There’s an old saying, “When you’re a hammer, everything looks like a nail.” Nowhere does that seem more relevant than when watching a developer, newly proselytized in the gospel of concurrency, his dual-monitor kit propped up on hardbound copies of Threads Primer: A Guide to Multithreaded Programming and Patterns for Parallel Programming, while he listens to his backlog of Merlin Mann podcasts, IM’ing three friends and coding up TDD test suites for a multithreaded way to collate TPS reports.

Now, I’m not saying that that developer is you or that your TPS reports aren’t worth collating. I’m simply saying that there is a time and place for everything, and threads are no exception.

There are a number of useful resources, online and in print, that explain how to use threads: how to create them, synchronize them, and incorporate them in various design patterns. Here, though, I’ll take a look at why and whether or not you should bother.

1. Keep a Process Responsive

You may or may not remember this, but there was a time when you would print a document in Microsoft Word and the application would freeze until the print job finished. You had one fleeting opportunity to cancel before you were firmly committed to watching your dot-matrix printer slog through to the last page. Eventually, Microsoft fixed this behavior by running print jobs on a separate thread, but somebody somewhere, each and every time, has had to make a decision that a particular task would (or could or might maybe) take an annoyingly long time to complete and that task should be run apart from the user interface thread. It’s not automatic, though, and even today you can find corners of Microsoft Office that respond with Tommy Chong-like moments of pot-induced blankness when you pull the network cable.

To be clear, though, it’s not just GUI apps. Network services have to keep an ear to the ground for new clients, dropped connections, and cancellation requests. In either case, it’s critical to do heavy lifting on a secondary thread to keep the customer satisfied.

2. Keep a Processor Busy

No matter how fast your fancy new CPU is, if it’s not actively chewing on something, it might as well be an 8086. Keeping a processor pegged can be tough task, though, and it’s a vexing pursuit even for top programmers.

You see, there are all kinds of things that could cause a processor to stall. The most prevalent and most mundane is common disk and network I/O. When a single-threaded application accesses a file, calls out to a web service, or streams from your USB webcam, your processor is just killing time. It’s sitting on its thumb thinking about clouds and Rorschach Tests.

Occasionally, there isn’t much you can do but wait for the I/O to complete. But normally, just like planning a date for Valentine’s Day, there is something else you could spin up while you wait for that “in” restaurant to call you back. Like ordering flowers. Or picking up your dry cleaning. Or coming up with a backup in case the velvet rope for that “in” restaurant decides to keep out disorganized riff-raff like you. If you’re processing a series of files, you can process one file while reading the next. If you’re calling multiple web services, call them in parallel; maybe one will come back before the others. Try to think ahead.

3. Keep Multiple Processors Busy

Let’s say you’ve that a single-threaded application that has a monomaniacal focus on computation. It uses minimal I/O, makes effective use of caching, and has predictable loops that your compiler can turn into data-parallel operations. You’re golden.

Unless you have more than one processor.

You see, today’s hardware isn’t getting more Hertz, it’s getting more pipelines (as with HyperThreading), more cores (such as the Intel Core Duo), and additional processors. This means that even if you’re maxing out one processor calculating a billion digits of π, the rest of the processors on your multi-way Dell are drinking mint juleps on the veranda wondering what all the fuss is about.

If you run multiple processes, that is, multiple distinct applications or multiple instances of one application, the operating system will come to the rescue and goad those other processors into action. But if you want your specific application to wrestle maximum horsepower from the hardware it’s on, you’ve got to give the OS scheduler multiple things to work on simultaneously. And the way you do that is through multiple threads.

4. Simplify Coding of Cooperating Tasks

If you think about it, most software bugs come from either doing something difficult (pretty rare) or doing something simple in a difficult way (pretty commonplace). Sometimes, you end up writing code that trips over itself trying to advance multiple tasks at once. Just as many programming problems have both an iterative and recursive implementations, many programming problems also have both serial and concurrent variations. The version you choose depends on the nature of the specific problem.

For example, the recursive version of quicksort makes a whole lot more sense that the iterative version. Similarly, trying to write a multi-client network service with just one thread, though possible, particularly if you’re savvy with Sockets API, isn’t fun and isn’t natural. When you refactor it into threads, the code becomes logically simpler. One thread per client, one thread to rule them all.

The key in using threads to simplify your code is to try to keep your functions and methods on one train of thought. Rather than turn your code into an arthritic mess trying to interleave multiple simple simultaneous tasks, code each task separately. Code each line block with one mind. Let the OS thread scheduler do the interleaving.

Of course, I’m glossing over some of the difficulty in synchronizing multiple threads of execution. Coding toward this goal can trade one set of complexities for another. It may not necessarily be easier, but it may be clearer.


7 Responses to “Four Reasons to Use Multithreading”

  1. Sergey Lipnevich Says:

    Just for the sake of discussion, why not mention that you can do all these things using multiple processes as well? Sometimes the solution is much cleaner if you avoid sharing memory state altogether.

    #1 is the most difficult candidate for separation into multiple processes, but then it’s the item on this list that seems to include most “hairy” tasks :-).

    #2 and #3 look similar, and I think a developer must always think about #3, multiple CPU, and never assume #2, a single CPU. Even single CPU scenarios may include GPUs offloading computations. Those two items don’t seem to have any specific requirement for multithreaded as opposed to multiprocess code.

    #4 looks like the best candidate for multiple processes.

    All that said, it seems that Windows developers too often limit themselves to using multiple threads as opposed to UNIX developers using both multiple threads and multiple processes for different goals.

  2. Marc Jacobs Says:

    Sergey, your comments about multi-process options are definitely on point. In fact, given my interests in distributed computing, I tend to have a bias toward software design with process decomposition.

    That said, though, I really wasn’t trying to compare and contrast multi-thread vs multi-process models in this article. The focus here was to comment on an observation that many developers I’ve worked with are adding threading to projects without full consideration or concept of why threading would be helpful. Concurrency is the real story here, not necessarily threads or processes. With multi-processes, of course, the developer has to ensure that the cost of the work exceeds the cost of the IPC/RPC overhead to make it worthwhile.

    On your final point comparing Windows and UNIX developers, it’s important to point out that, in common UNIX, threads are implemented as a special case of processes and have similar performance characteristics. On Windows, however, the operating system implements fibers and threads as first class scheduled objects. Fibers and threads are much lighter-weight than Windows processes and their UNIX counterparts. Your observation that Windows developers tend to use threading more than UNIX developers may be due to threading being a more performant option on Windows.

  3. “With multi-processes, of course, the developer has to ensure that the cost of the work exceeds the cost of the IPC/RPC overhead to make it worthwhile.”

    This isn’t necessarily true if the developer is using a system that translates multiple processes and IPC to simple function calls using light threads. One such system is Erlang (interesting exposition about this here:

  4. Marc Jacobs Says:

    Hey, Reuben, nice to hear from you.

    Systems like Erlang and the even more fabulous CCR/DSS platform that is part of Microsoft Robotics Studio are only simulating threads, effectively running a custom scheduler within each process and performing inter-process communication through simple messaging. These systems trade a certain amount of programmer convenience and scheduler control (i.e. they are non-preemptive) for raw performance and the ability to have many, many more concurrent virtual threads than any OS is capable of.

    That said, while these systems can significantly drop the cost of IPC/RPC in many cases, I believe the maxim still applies: if it costs more to call the function than execute it, it’s probably not worth calling it as a function.

  5. Ross Says:

    I thought the saying was “When all you have is a hammer, everything looks like a nail”. Sorry to be pedantic, good article.

  6. Marc Jacobs Says:

    Punditry obeys no conventions. :)

  7. […] centric blog and has recently posted a fairly high level article on the four reasons that you should be multithreading your applications. The focus of the article is an observation that many developers he’s […]

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: