The Most Revolutionary Microsoft Technology You’ve Never Heard Of

12 Apr 2007

Nope, it’s not from Microsoft Research. It doesn’t hail from Cambridge or Mountainview, nor is it some underbelly technology in Microsoft Vista that only Mark Russinovich and the responsible SDE is aware of. Rather, it’s a new service-oriented application model built on two overlapping technologies: Decentralized Software Services (DSS) and the Concurrency and Coordination Runtime (CCR). It is currently shipping as part of the Microsoft Robotics Studio (more on that later) and is poised to disrupt the way we think about the Windows Communication Framework (WCF) and the way we design, architect, and implement distributed applications. It is a work of genius.

CCR/DSS (which clearly needs a more catchy moniker, such as Erlang# or Microsoft Nano) is the brainchild of Henrik F Nielsen and George Chrysanthakopoulos, a Lennon-McCartney, Morrissey-Marr pairing of gearheads that isn’t often seen in software since Carmack met Romero and formed id Software. Henrik was co-author of the HTTP protocol and Microsoft’s lead envoy to the W3C SOAP committee, and George was one of the wizards that got all the bits and bare metal talking to each other in the original Xbox project. I spoke with them on a recent visit to Redmond and heard first-hand about the things they’re working on.

The story about how these two met and came to work with each other is unfortunately unsatisfying and arbitrary. Plucked by whim from Microsoft’s technologista, Henrik, a fair bespectacled rail, and George, a raucous Dale Carnegie of the software world, were thrust together in an office to have a think about how complex distributed software ought to be written. As luck would have it, they hit it off, and they have been vibrating with technological creativity ever since.

An early prototype demonstrated a bank of computers, each outfitted with mouse, keyboard, audio, and graphics. In their demo, they mixed and matched the peripherals in real-time through a web-browser, routing the keyboard of one to the other, the graphics from one to the other, and so forth for all of the peripherals. It was a perfect, seamless, high-performance Frankenstein, and it challenged many ingrained assumptions at Microsoft about how operating systems should be written. They have since generalized their work and developed something that should scare the bejeezus out of the .NET Framework guys, if only they were paying enough attention to it.

Imagine an application model composed of ubiquitous state-driven services. Each service has a little bulletin board of richly-typed key/value pairs and can trigger events whenever a value is changed. Services can be instantiated anywhere where there is a lightweight service host (such as a PC, microprocessor, browser, or smartphone) and can register themselves as visible to other services, either by kind or identity. Services easily detect, address, and send messages to each other, requesting changes to each other’s state values and registering for another’s change notifications. Running services can migrate from machine to machine. Service state is completely versioned and serializable and can be backed up, restored, and rolled forward and back in time. Rich exception information flows between deep trees of services along lines of causality, regardless of whether the services are on the same machine or across the world. Each service and its state can be trivially viewed and modified from a web browser with no extra work, but can also be locked down using several different layers of security. Now imagine that that application model supported a level of high-performance concurrency usually relegated to specialized language systems like Erlang.

If you were to imagine all that as a fully-managed .NET system, you’ve got a decent picture of what CCR/DSS is all about. Working with the framework offers many “duh/aha/of course” moments; it quickly becomes clear that the way they did it is exactly the way it should be done. The idea of a constellation of communicating, state-driven services isn’t necessarily revolutionary, but their conception and execution of that idea is. The framework was conceived not just as a library, but as a full-blown application model. It’s an elegant work, esoteric to a point, but transformative and crystalline in purpose.

It is also yet another reminder that the top one percent of engineers are of an entirely different ilk than the rest of us. It’s not that the quality or quantity of their work is untouchable (though it comes close); rather, it comes down to the completeness of their work and the ease with which they do it, like they were jumping over a puddle. Capable of glimpsing a sublime solution to a problem and sketching it up without syntax errors in the back pages of a Moleskine notebook, the top one percent of engineers are the technological equivalent of a three-year old solving a Rubik’s cube or a twelve-year old scoring symphonies for the New Haven Symphony Orchestra. They can bind creativity to purpose and realize it with great fidelity.

Henrik and George are in that top one percent.

So why then, as previously mentioned, is this incredible Leninist technology squirreled away in the oddball Microsoft Robotics Studio? It’s not that CCR/DSS has any deep relationship to robots or robotics.

There are at least two answers to that question.

First, it is true that robotics marry well with a state-driven application model. Servos have speed and position values, on/off flags, and service records. Sensors have sensor data. Robots can contain an arbitrary number of services that need to route information and events to and from each other. In a sense, a robot is kind of distributed system on a chassis. Though robotics aren’t the fat part of the market for a technology like CCR/DSS, it certainly is an interesting playground for it.

Second, Microsoft’s internal version of Pin the Tail on the Donkey is called Pin the Product on the Ship Vehicle. For various reasons, Microsoft can’t simply ship a couple assemblies and a lone executable all by itself. It didn’t fit temporally or politically in the Microsoft .NET Framework 3.0 effort, and it wasn’t large enough to stand on its own. So, for better or worse, it became the backbone of Microsoft Robotics Studio (along with the very interesting Visual Programming Language or VPL).

I’ll write some more about CCR/DSS in the near future, but in the meantime, check out the wiki, watch a video or two and install Microsoft Robotics Studio. Right now it’s a bit like a 1966 VW Bug fitted with a turbocharged Porsche engine. And, yes, the documentation is meager. But that’s okay. We know how to pop the hood.

Advertisements

29 Responses to “The Most Revolutionary Microsoft Technology You’ve Never Heard Of”

  1. Frank Says:

    You’re a little slow to the uptake. It was published in either DDJ or MSDN Mag (who can tell the difference these days?) some months back and their Channel 9 video is quite old by now.

    But yes, the work is quite good. Not as elegant as Erlang, but, within the confines of C# and .NET, it is pretty nice.


  2. […] Microsoft technology squirreled away in Robotics Studio The Most Revolutionary Microsoft Technology You’ve Never Heard Of talks about a generalized event-driven framework that puts the CLR’s and Vista’s […]


  3. […] The Most Revolutionary Microsoft Technology You’ve Never Heard Of Nope, it’s not from Microsoft Research. It doesn’t hail from Cambridge or Mountainview, nor is it some […] […]


  4. […] a look at this blog post from Marc Jacobs – what do you think, do you agree […]


  5. […] The Most Revolutionary Microsoft Technology You’re Never Heard Of. Includes my new favorite phrase “Pin the Product on the Ship Vehicle.” [Via Mike Hall] […]


  6. […] The Most Revolutionary Microsoft Technology You’re Never Heard Of. Includes my new favorite phrase “Pin the Product on the Ship Vehicle.” [Via Mike Hall] […]

  7. Nauman Says:

    Agree, definately something that required more attention. It probably didn’t get that attention due to the fact that it is bundled in MS Robotic Studio and most people don’t know that it can be used without that. At the same time, Microsoft is also working on PLINQ and there must be some overlaps between the two technologies.

    As of current state, it is also not possible to use CCR assemblies in any commercial work without the license. However, you are right in saying that it is the best thing that MS has to offer in PP.

  8. damien morton Says:

    The CCR presents a very awkward way of programming, mixing the use of iterators with explicit continuation passing. Im guessing that it went into the robotics kit because 99% of programmers wouldnt know how to work with it.

    In my opinion, this kind of thing requires explict language support, unfortunately.

  9. pomonapost Says:

    I find it very interesting even though I don’t understand it at all.

  10. Marc Jacobs Says:

    Nauman: The licensing for CCR/DSS is definitely a little unclear for non-robotics applications, but I’d encourage anyone interested to contact the team. They’re eager to have the platform used outside of robotics. You can find more information about licensing, as well as contact the team, through the Microsoft Robotics Studio Blog [http://blogs.msdn.com/msroboticsstudio/archive/2007/03/09/coordination-and-concurrency-runtime-can-be-used-for-anything.aspx].

  11. Marc Jacobs Says:

    Damien: I entirely agree that explicit language support would make it much simpler to use. However, many of the concurrency researchers at Microsoft have given up on language extensions, at least for the moment, because it’s harder to experiment and harder to get beta testers. Claudio Russo wrote an interesting paper on the Microsoft Joins Concurrency Library, an offshoot of Comega. In the paper, he discusses some of the obstacles of trying to develop new concurrency paradigms as language extensions. Worth a read. [http://research.microsoft.com/research/pubs/view.aspx?type=inproceedings&id=2005]

  12. Marc Jacobs Says:

    Frank: The Jeffrey Richter article in the Sep 06 issue of MSDN Magazine [http://msdn.microsoft.com/msdnmag/issues/06/09/concurrentaffairs/default.aspx] is definitely worth a read, but it only covers CCR (and only a portion of it at that). DSS, which I believe to be the bigger story, was not covered.


  13. […] Matthews highlighted this interesting article by Marc Jacobs tonight in an email to Readify’s internal tech mailing list. It […]

  14. myal Says:

    c001 :cool: tech

  15. damien morton Says:

    C# is in desperate need of a moderately powerful metalanguage.

  16. alok Says:

    Check out Jini: http://www.sun.com/software/jini/jini_technology.xml

    Not a new concept, people have been doing this for years.

  17. FrankG Says:

    I suggest you google for “jini javaspaces” to see where Microsoft *borrowed* these ideas…

    Aren’t there any original thinkers at Microsoft?

  18. Christian Vest Hansen Says:

    Python developers have had similar concurrent programming options with the Twisted framework for some time.

    By the way, I’m not really into their naming convention. For instance, why is it called Arbitor when it enques work for execution? Why not WorkEnquer? And why is it called Port rather than queue? Maybe I’m just nit picking, but I find it confusing.


  19. […] really aren’t the best way to do it. Well now, some guys in an odd corner at Microsoft have figured that out. And the best part is, they’ve released a library that you can use in your C# 2.0 […]

  20. Marc Jacobs Says:

    Christian: I totally agree with you about their esoteric choice of terminology. When I met with them, I told them that their naming conventions like Port, Arbiter, Causality, et al, were too esoteric and would limit the audience for their technology. They certainly seemed open to suggestions, though.

  21. rektide Says:

    Thanks so much, I picked this up via reddit and would have had no idea about this project except for this blog.

    It should be fascinating to see how MS tries to bring this into product form, see what pieces they try to capitalize on most.

    Linux has avahi and dbus, but theres no service oriented infrastructure to make use of the platform. Gnome and KDE are happy playing with themselves as end to end desktop environments, not pieces of an information ecosystem.


  22. […] Other Stuff Does Microsoft Have Hidden? The Most Revolutionary Microsoft Technology You’ve Never Heard Of It is also yet another reminder that the top one percent of engineers are of an entirely different […]


  23. […] Microsoft Technology You’ve Never Heard Of Marc Jacobs has an interesting post titled The Most Revolutionary Microsoft Technology You’ve Never Heard Of He talks about the new application model based on the DSS/CSS components shipped with the Microsoft […]


  24. […] passing work I have done in the past and which I strongly believe in. More recently I read this link which I agreed with and made me think again about […]


  25. […] a new idea, despite how it may be touted by vendors. Regardless of the hype, CEP isn’t the most revolutionary technology you’ve never heard of. What’s fascinating is that out from a decades-old, primordial soup of ideas, research, […]

  26. Ankur Mittal Says:

    really cool info I love writing about MS and finally found this article

  27. George Giles Says:

    I really like this product, but it really is nothing new the Deneb IGRIP, and Silma Cimstation product had this in the late 80’s running on SGI’s. Tecnomatix RobCAD followed a few years later. They had better kinematics engines and robot path planners along with primitive b-rep modelers. What is novel is the Microsoft support and leverage the Windows the technology provides. The functionality is old hat. Sorry.

  28. George Giles Says:

    As a post script you can see it all at the above website, and yes that is a 33Mhz IRIX Indigo that ran all of the above in distributed mode in 32MB of RAM with room to spare.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: