Dual core everything

If you read high tech news, you probably have noticed that the future of high performance microprocessors has several core on the same die. Many just see yet another feature that will allow the famous Moore's law to remain true for an additional decade. I both agree and disagree with that.

If you look back at the 40 past years of computing you can see that the programing model where you have a single program (single instruction flow) running on single data set has been the most successfull one. Today you can find this form of computing everywhere from your washing machine to your car or your PC. Many embedded applications have several programs that they run on several several dedicated cores, but in the end this 1 to 1 mapping remains true.

The speed of this form of computing has increased at a speed of about 2x every 18 months (Moore). The sources of speed increase are technology improvement (frequency increase) and extraction of instruction level parallelism (ILP). In the past 10 years another form of parallelism has also contributed to this performance increase, SIMD for single instruction multiple data, marketing called these instructions "Multimedia". The invariant in this evolution has been the existence of only 1 flow of instructions to be executed by one processor. If you follow this idea, the generalisation of multicore/multithreaded microprocessor is a revolution as it would be the first time in the history of microprocessor that such a paradygm shift occurs.

If this was the case, neither AMD nor Intel would have taken such a risk. There are multiprocessor systems for decades in the server and high performance computing world and the next logical step was to integrate several of those processors on the same die. Getting performance our of this type of systems is a known problem and threading techniques and parallel algorithmics are well known... by a very small set of developpers.

So what does it change ? To keep providing the increase of performance the world asks to microprocessors, going to multiprocessors for everybody from the mobile market to the server market through every house and office seems pretty good. The difference between this improvement and most of the previous ones (except the SIMD one) is that it will not happen right away. It will take years to adapt the softwares, to form the developpers to use efficient threading techniques, and during this period outside of a small set of specifically tuned applications, the average speed increase of computer applications will slow down significantly.

How is this different from SIMD instructions ? It took years for them to be used efficiently, and only a small set of widely used applications were ever written to benefit them. It is now a pretty good success, and close to no programer will ever use these and this will never be a problem. So why being more concerned with this dualcore thing and multithreaded applications ? Why wouldn't it become the same kind of success, most programers ignoring it and most users benefiting it ?

The main difference between those 2 improvements to microprocessors is the cost in term of area on the chip. Microprocessor performance has increased due to frequency and miniaturisation, minuaturisation allowing to put increasingly complex and efficient algorithms in the same area. The initial area cost of SIMD was roughly 5% of the chip, the initial cost of dual core is somewhere around 50%. What this mean is that for SIMD about 5% of the area were sacrificed from general purpose performance in order to increase 1 class of important applications. Nowadays with 5% more area you can't do that much to improve general purpose performance (and anyway it will be much less than a 5% gain average) so it had a negligible impact for in the end a pretty nice benefit. The 50% that are spent on the second core however are definitely not negligible and could have been used to improve general purpose performance by a non negligible amount.

The area used by this second core is about equal to the gain in minuaturisation achieved through a process change (usually around 2x increase in density) which means that traditional non threaded applications will see a smaller performance increase than they usually saw for this type of evolution (frequency gain is still there).

So why everybody's moving to multicore if that isn't that great ? Well, the short answer is that the other options were worse... RIP Tejas (cancelled successor to the Intel P4 line of processors) is probably a very good symbol of the other options, one single big, huge, complex, fast, HOT core.

I see 2 reasons for the needed change to multi core, the first one well admitted everywhere is the power dissipated by those increasingly complex microprocessors. It is easier to manage the power for 2 simpler cores than a big fat one, especially when you know that in many situations the second one will not be used. You can play tricks like having one core down and the other at high frequency (and voltage) when you run a single threaded application, and have both at medium frequency when you run several threads. For the big cores the added transistors are usually supposed to contribute to some complex algorithm so it is harder to shut them down. Even if a lot of transistors could be micro managed to save power it ended up not being efficient enough. I will not spend time justifying the need for lower power processing it has been done a lot before.

The second reason that I haven't heard many people talking about is the failure of the microprocessor architects (I am one of them) to provide new algorithms that were efficient enough in term of power/area/performance to be a better choice over multicore. Maybe it is just because there isn't much left ILP to extract from traditional applications, and the techniques we have found require way to much masturbation to be efficient. Anyway, wether it is a failure from the architects or just an existing limit, this marks the change on relying on architecture for performance to relying on software. Remember all those mid-90s talks about ia-64 and VLIW saying that ? It seems they weren't that wrong on the root problem, it just started to appear 10 years later than they initially thought (very easy to look back and say they weren't that wrong, i know :).

I wrote a small article for Onversity about the need to increase the emphasis put on the teaching of parallelism to future programmers because of the multicore trend. The only reactions I had were disagrement, saying that it wasn' t needed, that the basics of parallel programming were already taught and that anyway only very few applications will be benefiting it and the average programmer will probably never have to deal with it. I obviously disagreed with that, having taught that subject a few years ago In France and being passionated about it but ... they are probably right in the short term (only specific applications will care about it). And in the long term assuming that multicore are a success, there probably will be software tools to provide a simple abstract model so programmers will be able to ignore the structure of what they are programming, especially multiprocessors systems(today we can just hope for these, I haven't seen much forward motion in that area in the past decade, Ok I've not been looking hard). However, for the transition period I believe that the rare architecture aware programmers will be a very sought after ressource. How long will this transition periode be ?