Threads Considered Harmful
Threads Considered HarmfulPlease attack/defend/discuss the following proposition (yes, a ThreadMode page about threading):
Java 1.5 has added additional synchronization primitives and additional JavaCollectionClasses? which implement various BlockingQueue? strategies. -- RobertField
Here's an attempt at summarizing this page. Please don't add discussion here, put it later and add to this summary only if there's something genuinely missing.
By telling that threads are harmful, you're also telling that multicore CPU are harmful. However, they have been built though, and they certainly improve the performance of a PC or server a lot. So, the old serialised times are gone, and that's exactly the reason why we MUST program with threads nowadays. Otherwise you use only 1 CPU and your performance won't be improved at all. Anyhow, deadlocks are no problem in Java if you use (guarded) synchronized blocks and encapsulate the synchronisation commands into each other (like "synchronise(a), then synchronize(b), then do what you need and finish it!). Anyway, hash tables and arrays don't need synchronisation for changements, only for addition or removal of elements, so we don't have performance problems here, still using arrays isn't really great for it (since you need to block the hole array for a search, blocking the access of elements for other threads!). At best you could minimize the synchronisation efforts in Java by using hash tables. Do you know what? Hash tables and threads should be learned in every beginners class, because altogether they are nothing complicated at all! They are like a team, like man or woman needing each other. The same as good hardware needs good software to be useful, and good software need good programmers using threads. Cheers, Mark Weiler
The problem isn't that the idea of threads is bad (though a lot of programmers have gotten the wrong idea of threads from the available implementations)...The problem is that the implementations provided by most of the existing OSes are inefficient, and the primitives provided by those OSes to manage the interactions between threads (a) are many and varied, (b) are complicated, (c) are asynchronous, and (d) make you explicitly do both communication and synchronization in all the right places. I have no doubt that there is a better way. :) See ThreadsAreComputationalTasks and SendReceiveReply.
-- ShamelessPromoterOfSynchronousSystems? (er... WylieGarvin)
I have seen this premise begin to take root several places: the policies of some development shops, the design of EnterpriseJavaBeans specifically prohibits the use of threading within a bean, and some web application servers (like WebLogic) discourage it as well. They say "We handle the threading... you just write your code single-threaded or multi-threadable (tell us which), and our framework will schedule it all. You must never start a thread of your own, nor perform wait() or sleep() calls which affect thread scheduling."
Attacking the premise: Threading is a powerful tool, which allows us to do things which were extremely difficult without it (you might even say impossible). I once built an EnterpriseJavaBean with a synchronous interface (you called the function doTheWork(), and it returned an Answer object). Behind the scenes, it needed to call an asynchronous module (it needed to call startTheWork(), which returned a Key, then later call getTheAnswer(key) which returned an Answer).
To do this, you need threads. Well, you could maintain a list of active Keys and run a tight loop which called getTheAnswer(key) with a timeout repeatedly, interspersed with calls to startTheWork() for new requests coming in. But essentially what you're doing is implementing your own threading... which is one of the FiveSignsAprojectIsSpiralingOutOfControl?. Besides, because of details of java's IO model, calling getTheAnswer(key) blocked until an Answer was ready.
So what I did was to violate the specifications for an EnterpriseJavaBean. The specifications explicitly say that you can't launch a new thread or execute any code which affects thread synchronization. I did it anyhow, and fortunately it worked. But can't we have something better? I admit that writing robust threaded programs can be quite difficult -- that even experts often err -- but to take the tool away seems too extreme a reaction. -- MichaelChermside
Defending the premise: Have you ever worked with a beginning developer - one just out of school - on a project involving threading? If so, I rest my case.
That's the best time to get a beginning developer into threading - while they're fresh, and before they learn a bunch of prejudices and bad habits that they'll have to unlearn later! -- MikeSmith
Have you ever worked with a beginning developer using a linked list? A similar set of problems arises. This is more a trait of new developers than the technologies they use. -- PeteHardie
Yeah, I wrote the thread library and taught all the experienced programmers how not to deadlock the system. What's your point? Old people are dumb? I cautiously disagree.
Are we on the same wavelength here? I am saying that new developers tend to have problems with things that experienced developers do not, and thus stating that threads are bad because new developers cannot use them correctly is poor logic, because new developers also make mistakes using other constructs that we know are good and useful. -- PeteHardie
Are the limitations on threads in EnterpriseJavaBeans not also there to make EJB implementations easier for the application server vendors?
Yes, but that's just too bad. Lots of things about writing an EJB container are hard. You have to support lots of complicated transaction handling. You have to handle remote invocation. The intent of the whole EJB thing is to set standards that container vendors must meet so that the beans will be portable across servers. There is no reason to impose unnecessary limits on the bean authors. No one suggested it would be easy to write the containers, and this is something that the container writers CAN handle.
Threads are necessary in complex systems for many reasons. If you haven't used them, young or old, doesn't matter, it's difficult. The problem is having threads exposed to developers. Java having suspend/resume available is almost criminal. Place a message passing agent model on top of threads and deadlock becomes very rare and the usability of the system becomes high. -- AnonymousFool?
Java deprecated the suspend and resume methods a long time ago.
Like any other tool, threading is a good thing to have around in case you meet a task it's suited to. It's a way to sacrifice a bit of efficiency for clarity.
I'd say ThreadOveruse? is a bad thing, though - many processes running in the same address space can easily abuse it. Threads should be chosen judiciously, not thrown into a program willy-nilly.
Someone who programs in ErlangLanguage might disagree. Threads are cheap; why not have plenty? Just write your programs in an almost-functional style and you hardly ever need any locking. -- StephanHouben
That is almost correct. In Erlang, threads are not only cheap, they are very simple and safe because they are CommunicatingSequentialProcesses, and are completely isolated from each other. So much so that we use them not only when we need concurrency for performance reasons, but simply as a way of modeling the domain. Threads (confusingly called processes in Erlang) are Erlang's modeling equivalent to objects. What is incorrect in your statement is that the functional style is what avoids the need for locking. In fact, Erlang concurrency and asynchronous messaging is the part of the language that is not functional. Only the sequential parts of the code are in a FunctionalProgramming paradigm. The thing that makes locking unnecessary is that Erlang processes (i.e. threads) are isolated, and cannot share data -- DominicWilliams.
That's called LockFreeSynchronization. It is a cousin of AsynchronousProgramming. -- GuillermoSchwarz
No, it's MessagePassingConcurrency. LockFreeSynchronization is something entirely different.
The criticism may be specific to C/++ threads, in that they share an address space.
So do Erlang threads. I believe the commonly accepted definition of "thread" (as opposed to "process") includes the notion that address space is shared. -- sh
All right - now I'm confused. At erlang.org, the FAQ, the tutorial on concurrent programming, and the 'Programming Rules and Conventions' depict 'lightweight processes' which seem to converse through message passing; no obvious mention is made of shared memory or threads. My second comment, above, assumes these processes are what you mean by 'threads,' but the definitions you assert contradict this. What are Erlang threads? Are they the same as Erlang processes? Do they duplicate the C/++ hazards of memory corruption and of one segfaulting thread taking down an entire process?
Well, in Erlang the difference between threads and processes is (intentionally) fuzzy, since Erlang doesn't have assignment. This means that there is no way for the programmer to find out if the address space is shared or not. The system can thus figure out itself whether it should use threads or processes. And of course, Erlang is a safe language, so there aren't any segfaults. -- sh
No, the above statement is incorrect. Erlang processes are extremely lightweight threads managed by the Erlang runtime (virtual machine). A single Erlang runtime is called a node, and is seen by the host operating system as a single process. In that sense, all Erlang process share an address space. However, from a programming perspective, an Erlang process is completely isolated: it can only communicate with other processes by sending and receiving asynchronous messages in which data can only be passed by value. -- DominicWilliams.
At this time, given the current lack of maturity of our tools, techniques and libraries, threads are difficult and dangerous to use.
But I think we, as the computing industry, need to find a way to make threads >not< harmful. To better serve the needs of our users, I think our applications should work more like browsers: Display the information you have now, and allow the user to interact with it. Don't lock up the user interface waiting for every last bit of data and graphics to trickle down through the network interface; offer the maximum possible functionality at each moment of the loading process.
Unfortunately, current tools and libraries don't seem to offer the abstractions necessary to make this practical.
[I explore this idea in more detail in MultiThreadedGuiWouldBeGreat.] -- JeffGrigg
Threads are only harmful if they actual share data/resources. If you reduce the amount of data you need to share between your threads, perhaps by making a copy for each thread, you can go a long way towards making thread safe programs. Once you've done that you can isolate the bits of your program that do need to share data between threads and make sure they're correctly synchronized.
Try hard to avoid statics and singletons.
Threads wreck performance. They context switch, which ruins the cache and the pipelining. On the very same multiple processor machines they were supposedly made to take advantage of, they cause cache coherency chatter beyond control in many cases. You wait for locks. so what you're saying is that, for programs where threading reduces complexity, serialization (removing threads) is a PrematureOptimization?
You lock too much. You don't lock enough. The program crashes. You try to debug the program. Debugging a threaded program has a lot in common with medieval torture methods, IMHO.
I work in HPC (high performance computing), with very large vector supercomputing facilities, and I will tell you the ugly truth: only Wu-Tang and Shaolin monks can do thread in all confidence, if they have the Tao in them. Isn't Shaolin actually Ch'an (Zen)?
JohnCarmack added multithreading to Quake III: Arena (Q3A), a recent OpenGL first-person shooter game. He's widely recognized as one of the best programmer in this industry, without equal at squeezing features and speed out of his graphics engine. When he added a second thread to Q3A to take advantage of his dual processor machine, performance dropped by 50%!!! This is from memory, but the number is around that. Wasn't it supposed to double, or at least increase in some manner? Yes, of course. He worked at it a lot, and while he managed to get some kind of improvement out of this (not close to twice as fast), the things to remember was:
Not sharing data between threads is very hard to achieve, because you "share" data without really knowing, because two unrelated things are in the same memory page, which cause a cache flush for coherency, and messages to go from one CPU to the others. Maybe a specialized allocator may help there.
Here is a link to more information about multithreading: http://www3.sympatico.ca/pphaneuf/thread.html.
But all is not lost! Non-preemptive threads (also known as "fibers" in the Win32 API, see FibersExplanation?) can give you the same "feel". "Fibers" sound a lot like CoRoutines - same thing? Not at all - Win32 Fibers are just threads scheduled in user space, that don't get preempted by kernel code to give running time to other fibers in the same process. They're meant only for porting apps from Unix systems that have similar threading models.
Also, JeffGrigg comment that as an industry, we needed to make threads not harmful, as to better serve the needs of our users. What does threads have to do with the needs of our users? On a single CPU, threads only give the illusion of things happening at the same time, an illusion that good olde event-driven programming can do just as well (as always, you have to KnowWhatYoureDoing?). Good event-driven programming is hard? Well, debug a complex multithreaded program and come back to tell us about it, okay?
Now, many libraries don't make it easy (i.e., they often don't have asynchronous interfaces). A famous example is the gethostbyname() call. As much of a Unix fan that I am, I find the Winsock asynchronous interface to the resolver very nice. But they have an "enforced" event loop to maintain (the Win32 message pump), so it's easier to implement everywhere. It would be quite feasible to make a resolver object for libXt for example, where you can hook up sockets and timers to the event loop (it has probably already been done).
So write an async version of gethostbyname(). I'm not being facetious, that's what I did last time this came up some years back. I've also fiddled with the X11 socket, but of course that's inherently non-portable (not all X implementations use sockets; shared memory is also common).
I would reply to GlenStampoultzis with a question of my own: why use threads at all if you isolate the parts of your program properly? Processes with message passing could do just as well, no?
(If you had a software application where you needed to upload FTP files while you were still working in the application..you'd want to utilize a thread so that your FTP uploading wouldn't rob all your CPU power and halt the application for a few seconds/minutes. Of course, you could fire up a separate FTP process but
Don't mistake me for someone who uses threads everywhere. Best to avoid them if possible. Try writing a scalable servlet that avoids threads however. Message passing to separate processes can do the job. It has the advantage of forcing you to tightly define the communication channels you're using. Managing processes and message passing involves a little more work to also. I could also see problems if the amount of data shared between processes needs to be large. Perhaps we need a new threading model that can give us the benefits of both worlds. -- GlenStampoultzis
A RelationalWeenie solution is to spawn off a separate process that communicates with other processes only through the database. Databases already take care of most semaphore-like and/or multi-user transaction issues. It is like client-server where you spin off another temporary client-let. (I can't say this works for embedded systems or systems in need of predictable timing, however, due to the unpredictability of garbage collection schemes.)
One way to use pre-emptive threads to advantage and without pain is a MultiCaster. Another way is OrganicThreads. -- PeterMerel
As a bit of an aside, is there any particular good reason a VirtualMachine should croak on a couple hundred threads, or would it be something that generally just isn't done? I happen to like the determinableness/testability of code written using blocks and safe threading objects which invoke them for me: but apparently I got a bit trigger happy or something... I seem to fall off a cliff of some variety at around 500 threads or so, but I can't seem to track any particular resources spiking other than cpu time... ah, is it the context switches getting me? Or perhaps some too-smart implementation deep within the thread code with O(n^2) complexity or something... Either way, too much blogging, not enough testing/experimenting. (And sleep, as the case may be.) -- cwillu
What is the failure mode? Does the VM crash, slow down or lock up? My experience is that context switches can be expensive.
500 threads seems a bit small, but benchmarks comparing ErlangLanguage with other languages have shown that a lot of systems die given a few thousand threads. But (here comes that SmugErlangWeenie), Erlang does fine with tens of thousands of threads (OK, processes), and maybe more. -- BillTrost
In real-time systems context switches are not expensive. Only in heavy OSs are they expensive.
After using threads with both C (scary) and Java (still problematic) the only real way I could see myself coding threads is by having completely separate contexts and by message passing. For instance, the model used for threading with the ToolCommandLanguage. You can always only have one thread per interpreter. Messages can be read when the interpreter enters its own event loop. Effectively they are kind of light-weight processes with message passing abilities. Thus there is very little chance to get stuck with the usual thread nonsense. Even dynamically loadable extensions tend to work out of the box, assuming they attach any shared information to the interpreter instead of globals (which they should be doing anyway). -- Setok
This page confuses me, since I use threads all the time and don't know any alternatives. How do you write a server that handles concurrent requests?
See the apache 1 source code for unix like operating systems. They use processes.
How do you keep the UI active while doing something that takes a few seconds? How can you time out a remote function call if it doesn't respond after a specified duration? Creating a separate process and dealing with interprocess communication is much more of a kludge than writing multi-threaded code, at least in Java.
Traditionally you use asynchronous logic. Apps decompose themselves into state machines that are driven by async requests and responses that drive the state machine forward. As you usually have only have one CPU there's really no doing things at the same time anyway. You can have as many simultaneous activities has you can have overlapping state machines. An app must take no more time than the greatest desired latency. This is a form of cooperative multitasking. Though in a low-latency priority based system you always have cooperative multi-tasking. You can have code that shouldn't take a long time or shouldn't take a lock for a long time.
I've done that. I called it "writing my own thread management". It still uses threads, they just have to cooperate.
An approach like that would be plausible if threads didn't exist. But in a language like Java where threads are built-in, why not use them?
"As you usually have only have one CPU there's really no doing things at the same time anyway." Wow. That's dated. Even phones have multiple cores now.
I don't see why you couldn't remove the word "thread" in the original proposition and replace it with "C++". The argument is the same, or at least, very similar.
Those in favour will argue that threads/C++ make coding for mere mortals near impossible.
Those against will claim that people need to know what they are using in order to do effectively.
At the end of the day, I would argue it's about picking the right tool for the job and the team, and most of all, about the software being GoodEnough.
I have found that people don't understand even very simple threading code, largely because they can't keep it all in their heads at once. The idea that in just one instruction a different thread can be executing doesn't register. This is the problem behind the entire java synchronization approach. The proposition that you can replace "thread" with "C++" is false, though because you can write C++ code that is understandable, and though C++ is difficult, it is easier than threading.
Threads wreck performance.
Scheduling latency is very OS specific. It doesn't have to be large. If it is it is the OSs fault. Yet still we must consider this latency for the target system when creating an application. Lots of locks is an application issue. Java makes lots of locks all too easy though. Then there's the length of locks. People probably see low performance from threads because their locking and app architecture aren't designed towards parallelism.
As always with performance issues, until there's evidence of performance problems, DontOptimizeYet?. Even once there is, consider simpler architectural fixes before moving to an entirely single-threaded solution. You should use threads if it makes your code simpler, and optimize when your code is too slow.
I somewhat agree. It's part of engineering to validate your assumptions and requirements. If your requirements aren't ambitious for your platform then don't worry about it. But, if you think your requirements could not be met and the penalty for that is high enough, then as an engineer you must do some up front work. If you must handle Z requests a second and you know a context switch averages X usecs and your architecture requires Y number of context switches per request then and that turns out not to meet your requirements then you need to do something about it. It's not premature to know what you are doing.
Actually, one of the problems I often see with using threads is that people don't abstract the objects. They end up scattering POSIX thread calls across their application code. They then can't use idioms like stack-objects to get mutex locks, and hence spend loads of time trying to track down obscure interference issues. Bit of a whinge in general really... lack of ability to close the gap between the language used and the application domain.
The only real grief I've ever had with exception handling is that the recent versions of GnuCpp don't like exceptions and threading. The thread which THROW's isn't necessarily the one that CATCHes... *sigh* -- KatieLucas
"having completely separate contexts and by message passing" - yes!
"the gap between the language used and the application domain" - you put your finger on a problem that I felt was a root cause of a lot of the problems we had with application development over the years. Some time ago I sort of accidentally stumbled on the concepts underlying FlowBasedProgramming (FBP), which has completely separate contexts and message passing over BoundedBuffers. It also seems to result in components that have a good match with the kind of words we use in business applications, e.g. merge, update, summarize, report, etc. These functions are hard to build as reusable components in a single-threading environment. For a very simple example, see http://www.jpaulmorrison.com/cgi-bin/wiki.pl?TelegramProblem.
I'm also perplexed by the mention above of "thousands of threads" - some extremely complex banking applications that we built using FBP software did not exceed about 60 threads/components. These apps were written about 30 years ago, and have consistently performed well (and accurately!) since that time.
If you have 5 apps talking to a 100 nodes and you have a thread per concurrent activity that's 500 threads. It can get a lot larger obviously. There's no reason not to have a lot of threads if the threading system is well built.
There was a recent article in DrDobbs? called No Free Lunch. This article points out that Moore's law has allowed us to write software now, and assume that it will continue to run faster on newer hardware for some time to come (that's the free lunch), but Moore's law has recently ceased to apply when measured in purely ClockSpeed? terms. If CPU speeds today were following the trends they were following a few years back, we should have 10GHz processors by now, and notice that we don't. Also notice how big the cooling systems have become and now much the power requirements have risen just to support the current 2.x GHz processors.
Future improvements in CPU performance will have to come by adding more cores to the die more so than by increasing the clock speed of each core, and that means that for a single program to take advantage of performance improvements in newer processors, it will have to be able to seamlessly scale up to utilize ever increasing numbers of CPU cores. Whether that means writing multi-threaded apps, or something else, I don't know, but multi-threading is the most obvious approach. I guess another way would be the FiniteStateMachine approach, like StatiCee uses. Any others?
First class processes would be another.
EditText of this page
(last edited November 12, 2012)
or FindPage with title or text search