Multicore Processors will pwn

The current issue of ACM Queue is all about multicore processors. (Members of the ACM get a couple of magazines, Communications of the ACM, and Queue, which is more IT-related. I suppose they’re trying to reach both the Research and the Development sides of the ACM membership. Anyway, I bring this up because I seem to be the only person I know who ever opens these magazines. And indeed, they are often fairly boring, especially Queue.) But the newest issue of Queue has a lot of valuable analysis of multicore processing and its real-world impact. I would recommend that anyone even vaguely interested in multicore have a look at it.

Until now, even though I understood the broad outlines of multicore hardware, and some of its ramifications for software, I had not stopped to consider it as much more than a waystation on the path to more of the same of CPU evolution we’ve seen during my lifetieme: deeper chip pipelines, more pipelines, higher clock speeds, out-of-order execution and ever more clever optimization of such – basically, single-threaded Von Neumann computational destiny as we understand it. We all know there are serious hurdles along this path (not just problems, but equations, beady-eyed equations with ominously few terms.) But new materials come out, people talk about optical chips, diamond substrates . . . something good will come along. And why should multicore present a real shift in computing, when symmetric multiprocessing has been around for years?

Short answer: memory is still slow, so shared cache matters. But as a medium answer, I’ll briefly note a couple of the items that surprised me. Surprise #1 was an analysis by a Google engineer showing that electricity is now 40% of the total cost of ownership of a low-end server, which consumes 200-300 watts. This assumes electricity costs stay the same – if they go up, electricity may become more than half the TCO. The reason that power requirements have continued to grow is the way Intel has spent its transistor budget for the last five or ten years. Performance has grown, but performance per watt has stayed flat, so that each increase in processing speed has come with an increase in power consumption, making power a bigger and bigger factor in the cost of large-scale computing.

And this brings up Surprise #2: the notion that perhaps Intel has been _wasting_its_time_. The deeper and deeper pipelines, the prediction logic, all the things that they have spent their engineering budget on – and their power budget, their transistor budget, their chip fab budget – has been simply prolonging the megahertz race because that race was easy to market. A Google query – or any memory-intensive computing thread – would run almost as fast on a 500 Mhz chip as on a 2 GHz chip (!). The chips end up stalling waiting for memory anyway. But with multicore, four such 500 MHz chips can fit on the same size die, and for LOWER power costs, as the 2GHz chip. And they can share cache lines, possibly benfitting from each other’s memory fetches. So if the software is written smartly, the multicore setup will give four times the throughput as the 2Ghz chip. And I imagine it would be significantly cheaper as well in engineering and manufacturing terms.

So perhaps multicore is not a waystation – it might instead be the sign of having to go BACKWARDS (to lower megahertz and fewer pipelines, for example), in order to get out of a dead end.

But of course the software has to be written to take advantage of it, and apparently there's an amazing amount of work to be done. Something I won’t go into here is Surprise #3: Windows and even Linux have very coarse concurrency, though the move the multicore has been understood in the research community for years. To get anywhere with multicore, almost every line in an OS has to be reexamined for “serial” sections and OS-wide locking situations, and serious rewrites are in order. Which OS has already undergone this years-long process? Solaris. Will Sun be suddenly relevant?

(And might fine-grained concurrency be part of the reason we hear rumors that Google has its own version of Linux? Can anyone enlighten me?)