After my usual walk across the street to Starbucks to start my morning I was browsing Gmail and saw the daily AWS forum digest email. I’ve been reading it for the last three years, in 2008-09 it was a pretty cool way to see what people were up to in an emerging space, and it was almost always worth the time to read every post every day. Lately, for the last few months I’d stopped reading them as well as the EC2 search tag on Twitter–the posts and problems have just gotten too repetitive. Randy Bias had a Twitter quote a few months back about the massive technical debt inherent in EC2, and since I read it that tweet has been circulating in my mind. The usual meme has @jeffbarr busy on blog posts, signaling massive new innovations coming to our greedy little cloud community hands.
Maybe it was the calming aroma of my Sbux chai, or nostalgia, but this morning I decided to open the AWS forum mail.
Same old, same old. Rampant EBS IO issues, strange instance state behavior and problems connecting to instances (these are often newbie user errors). I threw out a tweet lamenting the dull consistency of these problems over the years, Randy’s ‘technical debt’ on my mind. These disappearing instances, network and load balancing limitations, zombie instances, and shaky, inconsistent disk throughput compared to traditional data center solutions are some of the main reasons why the Sr. Architect of Netflix’s movie streaming solution honestly recommend existing enterprise applications should not be deployed to EC2 without a scaling and availability refactoring. He said this while being a big fan of the platform. He was willing to re-write his app and got a good ROI for his content distribution and web interactivity use case.
Still nobody can argue with AWS’s 0-60 momentum in the online content, batch analytics/rendering and start-up market. A VC would laugh at you these days if you told them you weren’t building your consumer web application on the ‘cloud’. AWS’s innovation has been in two primary areas: operational and business efficiency, and in turning everything they do into a thoughtful URI. As the great Simon Wardley says “servers on the internet, innit?” S3 represents the summit of this efficient and URI focused innovation model.
But are these two areas of innovation enough to shift core business user behaviors on core data-center servers and networking? In July I last blogged that in order to win even complex analytical and web applications they would significantly need to improve their network topology and performance. The zealots told me I was crazy, the world was shifting, and that applications would be re factored to fit the EC2 model. A week later EC2 released cluster compute instances–but with a completely different network control model (direct switch connection configuration) and with a new AMI/hypervisor layer. This crude, but expedient method of meeting obvious market demand for higher performance stuck in my head–it looked like a hack.
So in the predictable twitter backlash to my ‘same old problems’ post, I leaked the thought that had been ruminating in my mind the last few months since Randy’s comment..what if technical debt was preventing them from innovating in areas core and vital to enterprise buyers? What if they never cross the chasm to the enterprise mainstream (like say HP blades + RHEL have, utterly devastating UNIX systems), what if like Netscape or Yahoo……
Boom. A hush fell across the room and all eyes were staring–did that dude just say AWS wasn’t innovating? (I didn’t, just that they weren’t in some areas) Wow he must be paid to say that, or just a dumb @ss right?
In 2004 I was responsible for crafting some of the early response to Red Hat’s incursion into Sun Solaris. RHEL became deadly when one thing happened–when they started becoming highly compatible with traditional data-center software. Yes, they started in web stacks early, but Matt Szulik was laser focused on being a cheaper version of UNIX, with higher availability and higher performance. Even before it was called RHEL all of the engineers I worked with in 1998 at Level 3 considered Linux more resilient and faster. It was a diamond in the rough, waiting to explode. Szulik knew this, packaged it as RHEL and hammered relentlessly on compatibility, tier 1 workloads, and the core of enterprise budgets.
AWS innovations aren’t doing this, and are not focused on single node performance and availability, they regard ‘the enterprise’ as wrong headed around architecture and are out to convert them to their pure distributed system design paradigm. They will win some converts, they have won Neflix and new start ups and many like workloads. They are also famous for a ‘take it or leave it’ negotiating stance with enterprises. They don’t believe in salespeople en-mass, and obviously think this article was written by a dumb @ss.
They are innovative as heck, they are fashionable as heck–but to what end. Where is their end game? How big is the batch processing and re-factored web application market? Its not even clear that developers care about all of this IaaS fuss. What does their somewhat hack like cluster offering tell us about their commitment to the hardest infrastructure innovations?There is no central conflict with enterprise vendors as there was with Linux. No technology is good at everything, R&D managers have to decide where to innovate and fight the universal physics of software development.
And let’s not forget when it comes to web applications, the PaaS monster is out there, growing, lurking and ready to remove the complexity of managing a distributed system for developers.