STREET#GRID 2007: I’m Sorry, I Have a Headache
17 Apr 2007
One of the more interesting, semi-futurist ideas floated by the morning panel discussion at STREET#GRID 2007 yesterday was the idea that job schedulers would begin to use the hardware monitoring capabilities of modern blade computers to influence task assignments. Kevin Pleiter, IBM Emerging Business Solutions Executive for the Financial Services Sector of IBM, imagined a toolset that allowed job schedulers to take into account whether a particular blade or rack was running too hot, was disk-bound or drawing too much power, etc.
While today you can get to a lot of interesting system information through technologies like SNMP and WMI, without coupling this information with more accurate models of how your distributed applications use compute resources such as CPU, bandwidth, and disk, it’s nearly impossible for job schedulers to make better decisions about task assignments. For example, if you have a task that is CPU-only, what does it matter if the target resource is disk-bound? What if your task pounds the network interface for its first five seconds but is quiet after that?
Creating accurate models of task resource usage is so far well outside the capability of any distributed computing product on the market. It would be nice, though.