Threads Are Optimizations

The proto-pattern has been moved to AvoidThreadsForOptimizations based on debate from this page. This was originally from SynchronizationStrategies.

Threads are optimizations. They exist to maximize processor usage.

[I strongly disagree with this! I think ThreadsAreComputationalTasks.]

AAAAAUUUUUGGGGGHHHHHH!!!!!!! No, no, no, no, and no. And some more no. Or at least not always. Yes, in many cases, threads do exist to tolerate latency. (And on some machines (http://www.tera.com), they're necessary.) However, threads also open up more organizational techniques.

For example, take an event-driven system like a caching http proxy. In fact, take a specific example: Squid (http://squid.nlanr.net). Look at its code. Every action is explicitly broken up into a bunch of smaller actions linked together through callbacks. Squid has a limited number of tasks it needs to perform, and these tasks decompose naturally into smaller units, so this works very well.

It doesn't take much imagination to come up with tasks that do not decompose nicely. For example (working on writing the concrete example ;)), take a distributed sparse matrix factorization system (scientific computing). There are a lot of tasks that can be overlapped, and almost none of these tasks have a natural decomposition into efficiently-sized smaller tasks. The smaller tasks are all too small, and you lose big-time if you try to break them up in a manner like in Squid. It's not just a performance loss; the code becomes incomprehensible. And undebuggable. I've been there. It's not a sane task. There's actually a package that kinda manages this in Fortran90 (MUMPS). It's gross.

Now introduce threads. In this case, the synchronization points are completely natural. And the code becomes clean. You give up a bit of control in scheduling, and you get back much, much more maintainable and debuggable code.

The moral of the story is 'not to think of threads as optimizations' (unless you know you need it). Think of threads as organizational units. Synchronization is organizational, so it's a much better fit. If you find that your application's organization doesn't fit into a thread model, you're not likely to be able to force it into an efficient one even when optimizing. So ThreadsAreNotOptimizations?.

(unfortunately, in the concrete example i'm writing, i can't always use threads. some machines (t3e) don't have thread-safe versions of crucial libraries. sucks.)

-- JasonRiedy

In your example, you've already written in threads. It's just that you're doing the context management yourself instead of moving it to a different layer (the operating system). However, you are correct. Certainly, threads seem sometimes more natural than other solutions. They are, after all, closer to the problem domain than polling because in reality many tasks run in parallel. However, sometimes, tasks run in parallel only as a matter of efficiency, not simplicity. In those cases, ThreadsAreOptimizations. I'll see if I can change the wording above to reflect this clearly. Better? -- SunirShah

Somewhat. I wish I had time to explain more thoroughly... In a few years (5-10), you won't have any choice but to write very threaded systems, but locking will be much more natural. The Tera machine I cited uses a nifty system originally from MIT's J-machine: Full/empty bits on every word of memory. You have hardware-supported, fine-grained locking with basically no performance hit. And it's very natural to use (kinda like tuple spaces, you can check out and check in information easily). This type of thing will grow. Sun's MAJC processor is threaded (iirc), and the multiple-cpus-on-a-die processors (Power4, Alpha) are similar... They don't have the fine-grained locking, though, so they're still a pain to use. You already know the locking and synchronization issue is the key. Simple and fast fine-grained locking makes building more sophisticated synchronization systems accessible. It doesn't sound like it until you try it. Tera's a kinda slimy company, and they're going fast into a disappearing niche, so this particular architecture won't be around too long, but the ideas are moving out to industry.

Still, if you think of threads as optimizations, you're already hosed. You won't see how they work their way through the organization of the system being designed. That's the source of DaveHarris's comment above about having to design them in from the beginning. It's difficult, but a lot of that is from the programming environment and education.

People are trained to think of threads as a way to make things faster, when they should primarily be a way to make things more clear. The speed then comes from both better utilization of resources and better changes to clear code. (There are exceptions, of course, but they tend to be related to very regular data structures, and then the code still remains pretty clear.)

And the environments layer lots of complexity atop threads that don't necessarily need to be there. Java's support is a start (I tend not to like Java), but a skim though DougLea's book on concurrent Java shows many of its weaknesses. The Java `community` (aka Sun) is attempting to address them through Jini and the transaction system, which is a good start. There are many aspects of Java that I detest, but it's a good place to look for thread management ideas. -- JasonRiedy, being wordy but not saying enough

My principle source for this story/pattern is from our current project here at work where we used threads as optimizations. We certainly could have gotten away with simpler code without as many threads, but it would have been less efficient. It had nothing to do with code complexity and everything to do with CPU usage. On the other hand, other parts of the system were simpler with threads. Maybe the name of this page is wrong (I renamed the section in my report to ThreadsForOptimizations?). Maybe avoid using threads for optimizations, just like any other optimization. I kind of see what you're saying, and I see that I've written this wrong, so, I have once again reformulated the above text. Better? By the way, I've just acknowledged you, Jason, in my work term report for helping to clarify this section. ;) -- SunirShah

In ErlangLanguage, threads (with a few special properties) are the fundamental means of program structuring - so ThreadsArentAlwaysOptimisations?. -- LukeGorrie

Um, that's one of the FunctionalProgrammingLanguages. The (ideal) lack of state renders moot most of SunirShah's points. And some of mine. FunctionalProgramming is all about directing and organizing the flow of the program over the data. Many don't even bother you with the threading aspects; they're automatic (especially in Clean, iirc). Some aspects of FP are pretty opposite to some of OO. However, they can also be complementary. Thinking of your problem in both contexts can give nice insights that are relevant to the threading issue.

I definitely agree that ThreadsForOptimizations? are pretty nasty. It's difficult to make them fit where they don't want to go, and you often don't get a worth-while performance boost in those cases.

Anyways, I'll try to give this matter some more attention this weekend. It's finals time. Yippee. -- JasonRiedy