Steve Tally of Purdue University has written a wonderful overview of the critical issue facing high-performance computing today: performance gains are no longer tied to transistor counts but to new concurrent hardware architectures, and the programmer base is lagging behind the necessary skills to drive those architectures.

The fastest of the fastest computers — supercomputers used at national research centers, research universities and major corporations — will soon gain even more performance by taking advantage of multicore computing.

Despite the promise of almost unimagined computing power, however, even computing experts wonder whether this time the hardware developers have raced too far ahead of many programmers’ ability to create software.

From ACM TechNews.

My father has been in direct marketing for years now and in sales for even more years before that. He’s picked up a number of colorful phrases along the way (some so colorful that they can’t be repeated here), but he often shares this fine nugget:

When value exceeds price, a deal is made.

I’ve always found that to be a Zen koan-like phrase, especially given that neither value nor price are necessarily fixed, and you can manipulate either to make a deal. When you can’t budge the price, bump the value. And when real value isn’t elastic, you do as great salesmen do and create perceived value. An Ariel Atom, strictly speaking, is just a car, a mode of transportation of comparable value to a Dodge Neon. Four wheels, brakes, moderate gas mileage. So why does an Ariel Atom trade at a price at four times the price of a Neon? Because, for some, the self-image of open-wheeled racing and thrilling performance has a value. And if that value (plus the real value of owning a car itself minus the premium demanded to avoid destitution) exceeds invoice, somebody’s pulling out a checkbook.

Anyway, I was reading Larry O’Brien‘s post, “Map, Everything’s An Object, and Inline”, and I kept thinking about this relationship. The changes in modern hardware, the palpable lean of our tools toward parallelism — these factors are creating a considerable value proposition for pervasive concurrency. Multithreading is suddenly starting to look more “fat free” than “free fat”. Erlang is coming back from the dead. PlayStation 3 consoles are topping the teraflop charts at Folding@Home.

But when a coder introduces threading to solution, are they creating real value or perceived? Is it faster? Is is simpler? Is it better? Is it appropriate?

I love functional languages and their neat mapping into parallel constructs. I love the idea that a program may be paired someday with a virtual machine capable of wise and balanced decisions to parallelize or distribute based on an accurate assessment of real costs and benefits. But I wonder if, like Larry suggests, the intermediate solutions of compiler switches, pragma statements, and metadata attributes might just be nothing but enough rope to hang ourselves with.

Last week my family and I returned from a visit with the in-laws and boarded a fully-booked Continental Airlines flight at Sacramento Airport bound for New York-La Guardia. Typical of air travel these days, it took nearly a half-hour to shuffle the nearly 140 passengers on board. There were crying kids, wimpering animals in pet carriers, amorphous lines, bustling mobs, and a cavalcade of renegades who believe boarding by row number can’t possible apply to them.

From a civilian’s perspective, this was just one more example of the headache and disorganization we come to expect while being carted into passenger carriers like Guernseys onto a cattle car. From an engineer’s perspective, though, it’s actually an interesting and tantalizingly thorny optimization problem.

Read the rest of this entry »

Nope, it’s not from Microsoft Research. It doesn’t hail from Cambridge or Mountainview, nor is it some underbelly technology in Microsoft Vista that only Mark Russinovich and the responsible SDE is aware of. Rather, it’s a new service-oriented application model built on two overlapping technologies: Decentralized Software Services (DSS) and the Concurrency and Coordination Runtime (CCR). It is currently shipping as part of the Microsoft Robotics Studio (more on that later) and is poised to disrupt the way we think about the Windows Communication Framework (WCF) and the way we design, architect, and implement distributed applications. It is a work of genius.

Read the rest of this entry »

From ACM Queue:

PeakStream Founder and CTO Matthew Papakipos explains how the PeakStream Virtual Machine provides automatic parallelization of programs written in C/C++ so that developers can focus on their application logic — and not the intricate details of parallelizing the application — and ultimately improve the performance of HPC applications when running on multi-core processors.

Matt is a friend of a friend, and I spoke with him several months ago about his company and their premier product, PeakStream Platform. PeakStream provides enabling technologies that allow developers to more quickly take advantage of general-purpose computation on graphical hardware (also known as GPGPU). Current NVIDIA and ATI graphics cards have incredible power to perform certain types of mathematically intensive calculations in a streaming, data-parallel fashion; however, taking advantage of that power requires some deep programmer sophistication. While efforts such as HLSL, Cg, CUDA, and CTM have made GPGPU programming more accessible, a cursory look at the documentation for these technologies proves that it still isn’t anywhere near easy.

Read the rest of this entry »

There’s an old saying, “When you’re a hammer, everything looks like a nail.” Nowhere does that seem more relevant than when watching a developer, newly proselytized in the gospel of concurrency, his dual-monitor kit propped up on hardbound copies of Threads Primer: A Guide to Multithreaded Programming and Patterns for Parallel Programming, while he listens to his backlog of Merlin Mann podcasts, IM’ing three friends and coding up TDD test suites for a multithreaded way to collate TPS reports.

Now, I’m not saying that that developer is you or that your TPS reports aren’t worth collating. I’m simply saying that there is a time and place for everything, and threads are no exception.

Read the rest of this entry »