STREET#GRID 2007: Developers Are A Lazy, Superfluous Lot

17 Apr 2007

I figure it’s even money that you read the title and said to yourself, “Well, duh,” or, “Dude, we are soooo not superfluous.” Either way, I think you might have raised an eyebrow at the first annual STREET#GRID 2007 Conference at the posh W Hotel in Union Square this morning. Lab49 Director Ross Hamilton and I attended a day of presentations and panel discussions and came away with the impression that developers were corporate waiters shuttling grid apps from IT service kitchen to table for tips. Who knew service-oriented architecture was about food service?

From the conference website:

Hosted by DWT and Waters Magazine, the inaugural STREET#GRID conference provides CIOs, CTOs, Chief Architects and senior technologists with a comprehensive understanding of how next-generation grid computing, virtualization, and SOA technologies can be applied to ensure optimal use of existing IT resources and increased revenues for business lines.

I tend to enjoy conferences like these, not for new information, but for the comforting sense of familiarity they breed: same faces, same themes, same vendors, same food. [Chicken or fish, sir?] It’s like sinking into a cushy leather chair with a smoking jacket and a hard-bound copy of Emerson. The whole High-Performance Rat Pack were in town: Platform, DataSynapse, Gigaspaces, Tangosol, HP, IBM, Microsoft. IBM came as close to Sinatra as one could get, hitting the high notes and talking like it’s all been there, done that. Microsoft was a teenage Wayne Newton, the Rat Pack-wannabe singing Danke Schoen just for the privilege to hang.

It was at a conference just like this back in January that I first got the idea about writing a book about distributed programming. When you get a sufficient din of vendors and customers together, you cannot help but hear the parallel conversations, the lost messages, and the pleas for a set of technologies that work together and simply do what the customers need them to do. People want guidance. People want the stuff to just work. People want standards and interoperability. So far, that’s not what the vendors give them. It’s the same old song at these conferences, but I get something new each time from hearing it.

Today’s rock in my shoe, though, was the perceived impotence of developers to advance the distributed computing cause. Bob Hintze, Vice President of Utility Computing at Wachovia Corporate and Investment Bank, fired the first salvo with the following quote:

We [at Wachovia] place little emphasis on SDKs and developer consciousness in making progress [with grid computing]. They’re simply less of a priority. Global monitoring and global management is a greater priority.

True, the SDKs and APIs for most of the current distributed computing frameworks are passably mature, while the management and deployment features of those frameworks are in diapers. If you’ve got a ten thousand node grid across multiple physical locations, it’s still pretty much up to you to figure out if any of these nodes has a bad power supply, is overheating, or is disk-bound due to a unscheduled backup. As Mr Hintze said, “Management was and still is the largest obstacle to overcome [in the adoption of grid computing.]”

Developer consciousness, though, can’t be overlooked as a key factor in distributed computing adoption. Distributed applications don’t just write themselves or become distributed with a simple recompilation (all hopes on Cluster OpenMP aside). All of the elastic computing power in the world is nothing but a ball of rubber bands if developers don’t know or can’t see how to take advantage of it.

So why don’t developers write more distributed applications? Maybe it’s hard? Kevin Pleiter, IBM Emerging Business Solutions Executive for the Financial Services Sector of IBM, had a different perspective:

So far, institutions have just done the low-hanging fruit to a large degree. The principal reason for that is programmer laziness.

Ack. Sure, mainstream developers may not yet know much about concurrency or grid computing, and their lack of expertise isn’t exactly accelerating adoption. But it’s a ghastly overstatement to suggest developers are slowing adoption because of their laziness. Developing distributed applications requires a massive integrated effort, of caches and job scheduling and algorithm decomposition and more. Some architects may grok distributed computing, but many journeymen don’t yet because the education isn’t available, the tools are all flavors-of-the-month, and many of the coders, as Herb Sutter suggests, just got to the point where they are finally effective with OOP. Laziness may be to blame, but not on the part of the developer. Vendors, educators, and book authors have all been slackers.

The only representative on stage that seemed to go all Braveheart for developers was Nati Shalom, CTO & Founder of Gigaspaces. Both Nati and his technology are very much developer-centric and seem to connect with the idea that developers are like craftsmen: they don’t choose for you whether your business builds chairs, bookshelves, or widgets, but without them to build it, you’ll have little more than wood blanks and sawdust to sell.

Advertisements

3 Responses to “STREET#GRID 2007: Developers Are A Lazy, Superfluous Lot”

  1. Nati Shalom Says:

    “The only representative on stage that seemed to go all Braveheart for developers was Nati Shalom, CEO & Founder of Gigaspaces”

    Marc thanks for the kind words.

    For accuracy my role is CTO and Founder not CEO, Our CEO is Yaron Benvenisti a good colleague and a friend. Believe me, the last thing I want is to get into his shoes :)

    Back to the developers view, I was actually very surprised to see how far the industry is lagging behind in understanding the criticality of that issue to the mass adoption of grid architecture. After years of experience with grid I still heard that the way to scale your application is to make it stateless, on the other hand people including those on the panel said that they do want to bring the business critical applications such as front-office applications into the grid. Obviously there is an implicit contradiction in that statement, those application are stateful by nature therefore you can’t scale them by making them stateless!. It is also clear that to build an application that will utilize the grid effectively it would require awareness of the application and the developers on how to write such application on a grid, for example: how do you partition the data, how do you route the transactions to node that contains that data?. How do you handle a scenario in which the resource that you depend on is not available but may become available at a later stage?. How can you verify that your application will behave correctly in a grid in your development environment?. These are only few of the things that needs to be addressed and cannot be avoided. The developer role is therefore critical since he is the one responsible for dealing with those issues.
    Creating a development environment that will simplify how you deal with all this tradeoffs will also have a direct impact on how much we can utilize our existing investment in the grid and how reliable those application will become on that environment. So rather then spending millions of dollars on building a bigger grid we need to make sure first that we can make the best out of the existing one. I’m hoping that with the work that were doing with some of the grid vendors we will be able to address that sooner rather then later.

    Nati S.
    GigaSpaces
    Write Once Scale Anywhere

  2. Marc Jacobs Says:

    Nati:

    Sorry about the mistake in your title. I’ve fixed it in the post.

    Truthfully, I think vendors like yourself are in a tough position trying to hone a single marketing message to drive adoption of distributed computing products. There are really two distinct audiences, the CIO and the coder, and they each require divergent messages that their counterpart doesn’t really understand, value, or support.

    It’s like trying to sell the government on seatbelt legislation or motorcycle helmet laws. The reasons government may be interested (e.g. control funding of medical care programs, appease insurance lobbyists, manipulate states’ rights) are orthogonal, if not contradictory, to the reasons the populace gets on board (self-preservation, controlling willful teenagers with new drivers licenses, limiting accident liability, controlling car insurance costs). The way the US Government dealt with these issues was to take a paternal, “Daddy Knows Best” stance and enact the legislation over the cries of the constituency. That seems to be what the financial community is doing as well. Of course, if it worked for Big Government, it could work for Big Finance too.

  3. kazim Says:

    As a audinece member I also heard the same information. Its always interesting when a snippet of a conversation is used to make a point. All be it a interesting one. Statements made on laziness were in the context of sticking largely with microsoft based programming tools and in the main not leveraging the power of the chipset by doing mixed precision programming (mix of DP and SP) see university of tennessee article on this and other techniques to fully exploit technology. I think many people find the extra time effort and programming knowledge to much, is that lazy I dont know but given the needs on wallstreet it is amazing how little of this work actually gets done


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: