I wanted to make some comments about Douglas Gourlay’s engaging but somewhat flawed recent article, A discussion and thesis on what it takes to build profitable public clouds, which was published in his Network World Blog. Overall, it’s a decent article and I agree with him on a couple of basic points. For instance, he is right when he says to commoditize everything to get where you need to be. I do not agree, however, with his main points that going open source and buying really crappy gear are the only ways to compete in public cloud computing.
To be fair to Gourlay, he’s really talking to people who are competing against Web hosting companies and low-end cloud providers that focus on development, QA, and demo environments. These environments are solely focused on cost and have little reason to care that much about uptime, security, supportability, performance, and redundancy. It’s quite a different matter when you’re dealing with companies that want to run their entire business using cloud technology. At that point, you are not competing on price against a cloud provider that offers Web hosting. Rather, you should be comparing costs with a company that has a data center (small or large) or somebody who has done outsourcing in the past where the total cost of ownership (TCO) is 5-10x more expensive when you add in the real “enterprise requirements.” In essence, Gourlay is taking a perspective on a single market, so understanding the validity of what he discusses gets down to “know your customer.”
Open Source, Open Source….
I hear this continually within IT groups and also from people who stand on top of large organizations. All they talk about is how they will drive efficiency with open source. I completely agree that any organization should use free tools when available and when there is no long term supportability challenges or, even worse, ROI issues. The bottom line is that open source does not matter for most businesses until the company supporting the “free” software can afford the long term investment and risk associated with using such tools. This isn’t even about a Microsoft vs. Linux. This is more about you get what you pay for.
Many times, I have seen companies use open source and talk to me how they created a way to do better database replication or how they made their LDAP authentication work better and be more redundant. When I ask them how long it takes to maintain this setup or what their real ROI is – once maintenance and support have been factored in – the answer is often, “I don’t know.” This is quite common. Plus, when you are talking to people who make $80-100k per year, it really starts to not add up for the company not to buy software that is tested and works perfectly instead of “creating their own wheel” or attempting to support something that has no facility for support.
In certain respects, this mindset can explain the unfortunate recent outage at Amazon Web Services (AWS) in their elastic block storage. This was an internal project built on commodity gear with open source technologies that they put together. AWS makes sense for doing open source and building their own custom system as they have an ARMY of IT people dedicated to the development, testing, and support of such critical software. Yet, the outage occurred because the technology did not know when to stop replicating and basically killed itself. I am not saying that AWS would have avoided problems if they bought enterprise storage replication products. But, for most companies it makes more sense to take this route as they don’t have the army of engineers to develop, test, and support this kind of technology. The reason you buy software or hardware is because the vendor company has done the development, testing, and supportability for you and will help you and be the expert when it is above your IT department’s head.
One last comment about open source. The true hard core open source shops that I know have taught me some interesting lessons:
- All of these companies have larger engineering staffs than shops running Windows environments. There are often good business reasons for making this choice, but it’s important think about adding up the additional investment in personnel if you are small. Windows averages $1000 per server and about $8000 for an enterprise database system. If you have 4 engineers on your team to support Windows but you might need 8 for open source – an increase in expense of around $500,000 per year at today’s salaries.
- Most of these companies spend over 20% of their time attempting to integrate and patch their systems with open source technologies. Out of that time it is really them trying to make things work. This choice should be a business decision if what they are trying to make work is the technology that runs their actual business.
- Other than the really big Web sites such as Google, Netflix, and Facebook, very few of these companies have the expertise to create a large scale Hadoop cluster and maintain it. Basically they read about it and try to implement one – this is just plain stupid and very risky when you have limited resources. The only way this works is if you are committed to that system and willing to put in the time and effort to be become experts in that software.
Hardware Choices
Gourlay is right when he says that enterprise hardware is very expensive and can cost so much that it makes perfect sense for a public cloud/low cost service/Web hosting/development/QA/demo type of business to go as generic as possible. That said, my feeling about going generic on your networking or storage layer is that it is about as crazy walking down the street with a half a cantaloupe on your head saying “I’m a hamster, I’m a hamster.” (And don’t think I haven’t thought about doing this from time to time in my line of work…) The reason it is crazy to go generic on your network or storage layer is that everything on your network relies on this technology. In most cases, when there is an issue in these layers, it will affect a large majority of your customers. In contrast, servers have been shown to have good uptime and supportability. Yes, you want to work toward better commoditization, but the choice of using enterprise quality gear should depend on your market. If your market is enterprise, you need the values that go along with it then you will end up using enterprise equipment.
Here’s how I view hardware. Hardware is a commodity and it is your company’s job to architect for the commoditization of enterprise equipment in your environment. This means that no matter what hardware comes out you always have an abstraction layer that allows you to automatically use other hardware that is enterprise quality but gives a better price point for your organization. Optimization is the key in this. As you optimize your technology stack, the enterprise hardware will give you better ROI if that is what you need and your customers expect. When Gourlay talks about power, I had to chuckle as by the time most companies get to that size, there will always be more efficiencies in the data center that can be promoted. Already, companies such as Google, Microsoft, and Facebook have talked openly on how they are saving large amounts of money by building their own data centers using shipping crates which you can buy for $2MM from SGI (http://www.sgi.com/products/data_center/ice_cube/) if you don’t want to engineer it yourself.
For Tier 3 we believe that hardware is just a means to the enterprise cloud but is not the reason customers will be with us as long term. This stuff will only get cheaper and cheaper. At Tier 3, our focus is on bringing on the intelligence and optimizations in the cloud that companies could never achieve on their own or with another provider. People like to think that the cloud is about hardware, but it has a lot to do with the software that’s used to manage it. That’s our approach and we will continue on that path as there is always more to do…