The Ripple Effect: Part 2

Recently, a company called Precision I/O announced their 1^st round of funding and their high level technology strategy.  (http://www.precisionio.com/news/pr-03-29-2004.html).

Precision I/O is birthed from Packet Design, a technology incubation company started by Judy Estrin and Bill Carrico, who have a string of both successful start-up and large company (e.g. Cisco) experiences to their credit.  As I understand it, cool technical ideas are conceived and nourished with-in Packet Design.  When an idea gets enough legs to be proven feasible from both technical and business perspectives, a company is spun off to focus on the commercialization of the idea.  The latest (and last for now, as Packet Design focuses on making the companies they have sired successful) of such companies is Precision I/O.

In this case, the Packet Design brain trust seeks to put to rest the age-old questions:

·        Why are servers so much slower than networks?

·        What can be done about it?

Basically, the answer to the first question is that today servers use the Operating System’s TCP stack to act as an intermediary to move bits from the application to the network and back.   This causes a lot of inefficiencies as bits need to be moved in and out of different memory spaces to get them from the network to the operating system to the application.  Things would be far more efficient if the applications could just “pull” bits off the network directly into it.   Precision I/O has engineered a software solution for a server that sits partly in the network interface driver and partly in user space. The result achieved is to get the operating system out of the network-to-application-path (and vice-versa), eliminating multiple memory copy and other kernel inefficiencies associated with passing data from user space to the network in traditional software architectures.  The payoff is much higher I/O bandwidth, greatly reduced latency, and lots of CPU cycles returned to the server for application processing.  All of this is achieved in a standard Ethernet/IP/TCP environment.  This sounds very RDMA/TOE-like in approach and value, and it is. But Precision I/Os approach is different in some important ways (that you’ll need to talk to them about) that result in some major advantages relative to existing RDMA/TOE approaches:

·        It provides server efficiency for all traffic load types – short transfers (e.g. OLTP, RPC, and possibly even web), streaming media, and large file transfers (RDMA is optimized for the latter).

·        At Gigabit and multi-Gigabit speeds, the solution is purely software – no special TOE or RNIC hardware is required.

·        No modifications to applications are needed.

·        Each server configured with the Precision I/O software gets faster and more efficient – it doesn’t require similar technology to be implemented on the other end of the connection (e.g server it is talking to or client).

What this means is that all of the promises of TOE and RDMA can be enjoyed by all servers - not just selected ones performing selected functions - immediately, with no disruption to server hardware, OS, or applications.  Like magic, any server that is engaged in any significant network I/O at all will get a lot more efficient.  

This almost sounds too good to be true.  We’ve been waiting for TOEs and RNICs to make servers more efficient for a long time and still haven’t really seen the payoff.  What’s different here?  Well, I’m not here to explain the technology -- I think it’s mostly due to thinking about the problem differently in a manner that they explained to me and made sense to my somewhat addled Marketing brain.    To get a better explanation of what they are doing, you’ll need to talk to Precision I/O people directly.  But my gut tells me it will work for a couple of reasons:

1.      The principal brains behind the project are Bob Felderman (co-founder of Myricom, the leading vendor of solutions for making clustered servers more efficient for high performance computing) and Van Jacobson (leading TCP researcher and co-founder of All That Matters Today).   These two guys are serious dudes who have the brainpower and experience to think about the problem a little differently and the staying power to grind for 2 ½ years to make it work.  They also have a lot of integrity and wouldn’t associate their name with something that wasn’t based on a solid technical foundation.

2.      Generally when we think about accelerating things, we think about hardware assist as a brute force way to make protocol implementations we are familiar with faster.  But the Precision I/O team has identified that in this case, the big payoff comes fro re-thinking how the functions are architected and implemented in software.  Often there is a bigger payoff in improving software architecture than in brute forcing a poor architecture.  I’ve experienced this recently myself with a client, Redline Networks, where they took a fresh look at web server proxy software architecture and came up with a vastly superior approach to managing clients and client connections.  Their customers leverage this advantage by front-ending web servers and basically mask all of the I/O inefficiencies inherent in all web servers today.  I’ve heard several of their customers say “I don’t know how it works.  It’s magic.  But it makes everything in and around our web servers run much faster and efficiently.”   And it’s all just software running on standard Intel hardware and a standard OS (which has been largely designed out of the critical path).   This represents existing proof of the power of re-thinking I/O software architecture. So I’m not totally shocked that the bright people at Precision I/O might have achieved similar breakthroughs by focusing on functional implementation rather than brute force hardware assist or off-load. 

But whether Precision I/O works or not isn’t my primary concern here.  Precision I/O is not a client. I don’t have any ownership directly or even indirectly through their VCs.  What I’m really interested in is the Ripple Effect potential of Precision I/O so that I can project the resultant industry dynamics and perhaps position a client company or two to benefit from them.  So…I’m hoping and expecting that the Precision I/O technology works as advertised.  If it does, I expect it to be adopted rather quickly because Precision I/O has eliminated most of the reasons to not deploy it (no impact on applications, no new hardware required, enhances all servers with any significant amount of I/O). 

Anyway, we won’t have to pontificate whether the Precision I/O stuff will work or not for very long. It’s supposed to be shipping pretty soon.  The 2^nd half of 2004 will be the acid test period for the technology. Since it is software and since it is so transparent to other system elements, I expect a lot of people to trial the capability very quickly.  If the technology makes it through the landmines of early adoption, I could easily see it being deployed pervasively in 2005.  The more rapidly a new technology gets adopted and deployed, the greater the Ripple Effect potential. 

So the question is: If all of a sudden, for a small incremental cost, commodity servers got 5x more efficient in terms of I/O – if they could actually run at wire speed at a fraction of the latency of present Ethernet/IP/TCP systems and at the same time return almost all of the CPU cycles presently consumed servicing network I/O to the server for application processing - what would change in the industry and what are the resultant opportunities created?

That will be the focus of the next two Musings of this series.