Gigabit, We Hardly Knew You
20 Mar 2008
It seems just yesterday that 10Mbit 10BASE-T Ethernet networks were the norm, and the workstation wonks I worked with years ago at US Navy CINPACFLT in Pearl Harbor, Hawaii jockeyed to have high-speed ATM fiber run to their offices. Sure, this was the age when dual bonded ISDN lines represented the state of the art in home Internet connectivity, but who really needed that much bandwidth? What did we have to transfer? Email? Usenet posts? Gopher pages?
Then, slowly over the course of the next 5-10 years, network vendors upgraded their parts at negligible marginal cost, and 100Mbit began to pervade enterprise networks of all sizes, democratizing fast file transfers and streaming multimedia. 100Mbit seemed pretty fast. The Next Big Thing, Gigabit Ethernet, seemed a pipe so big it couldn’t be saturated and certainly not worth the exorbitant prices the hardware was going for at the time.
This week, I’ve been helping out one of our Lab49 teams working at a big investment bank to run performance tests on a large distributed cache deployed a 96-node farm all connected on Infiniband. Interestingly, when we ran these brutal performance tests on a homegrown Gigabit blade setup with eight nodes, it was pretty trivial to saturate the network while the CPU idled along at about 3% utilization. Running them on the 96-node Infiniband system, though, the same tests pegged the CPUs while the network drank mint juleps on the veranda during the first warm day of spring.
The sad thing about this situation is that Gigabit is only now getting sufficient uptake in enterprise NOCs that it is worthwhile to upgrade the NICs out at the clients. The last mile, just like with 100Mbit, is taking a while. Despite the fact that Gigabit just hasn’t been with us that long, it’s pretty clear to me that, with snowballing of HPC, CEP, real-time messaging, and P2P network services (not to mention HD audio and video), Gigabit will be led out to pasture before it ever really gets a chance to race.
We may all still be using Gigabit at the desktop for years to come (or not, if 802.11n and offspring steal the show), but between Infiniband and all the activity we’ve been seeing now in 10Gbit (as evident during the SC07 conference in Reno, NV last November), Gigabit just isn’t “it” anymore. Just like pet rocks were in the late 70′s, Gigabit is a temporary salve for a social ill that ultimately requires a more vital solution. I may not yet know clearly what it is, but I know it ain’t Gigabit.