IDG Contributor Network: Why measure the cloud


Let’s see if this scenario sounds familiar. One of the business units has a new application. Management is really excited because it’s been developed to run “in the cloud,” and you’ve been tasked with finding out how IT is going to take over administration, which is going to be required for security, volume purchasing, and demand-management reasons. The business is also going to want to integrate it with several other corporate apps down the road, and your team will end up being responsible for a lot of that work. When you look into the background you find out that no real evaluation was done before an IaaS provider was selected. The development team simply chose the service they “thought would be best,” which really means the one that they were most interested in using.

You discover that this is going to give you a variety of headaches to deal with, including the following:

  • Your security standards call for on-site audits of any provider datacenter facilities to make sure that your internal security requirements are being met, along with periodic penetration testing, and the provider doesn’t allow either. Nor is it clear that their standard procedures, which are the same for all customers, meet your requirements. The application will need a special exception to the policy before it can even move forward.
  • None of the standard change management windows the provider offers fully meet the needs of the application’s users all year round. Custom change windows are available, but only for additional cost.
  • SLAs are not as aggressive as what you are accustomed to for other outsourced services, and significant service outages have recently occurred. To guarantee the uninterrupted performance that the business requires, you may have to fail the service over to a service from another provider, which will be costly.
  • The cost of 24/7 support from the provider is high and has not been planned for in the business case. Your team in Brazil is much more comfortable receiving support in Portuguese than English, but that is not available.
  • The infrastructure required for the application is expected to fluctuate very rapidly. The service can provision new resources in less than a minute, which is good enough for your needs, but releasing resources takes longer, and charges continue for a full day after a resource is released. That will raise costs dramatically compared to a service that stops billing immediately.
  • The primary datacenter where the service is provided has almost reached a limit on the power it can provide for infrastructure, and no guarantee of scalability exists without requiring downtime to rehome the application to another facility. That new facility would be further away, and its performance characteristics are unknown.
  • Disaster Recovery has not been addressed at all, and addressing it will require additional coding.
  • The only cost analysis in the business case is a straight comparison of published prices for compute instances and storage. Differences in configuration, support and included features and functionality have not been taken into account. The volume discount that you’re currently receiving from a competing provider was also not considered.
  • The service does not currently support all of the protocols necessary to integrate it effectively with your back-office systems, and management of the service will have to be via an independent console.
  • Response times have not been tested with real-world user counts, database sizes or network locations.
  • The provider’s tools are numerous, poorly documented and difficult to use. Substantial training will be required to master them, again increasing your costs.

Recoding the application to work with another service isn’t a good option for a number of reasons. It would take time that you don’t have, and it would demotivate the development team. The application wasn’t written to be portable, and moving it would make IT the “bad guys” once again, putting roadblocks in the way of getting things done instead of facilitating them. So, what is there to do? Sadly, there are no good options.

Back in the old days of IT outsourcing you could simply specify the service you needed down to the last detail, and the providers would be happy to comply to get your business. In the new world of standard cloud services, you don’t get to do that. Oh, the menus you get to choose from are extensive to be sure, but they are still fixed. That means you need to do a thorough evaluation of services before you choose one—far more thorough than you would have done with the managed services of the past. If you don’t, then things like security, interoperability and all the other items on the list above could come back to bite you. Not only that, a service that works best for one application may not be best for another, so these thorough evaluations need to be done frequently. Companies that skip evaluations are losing money—a lot of money, possibly consuming every dollar they planned to save by moving to the cloud in the first place. The few that do rigorous evaluations up front, before they make a final selection, are the ones saving the most money with cloud.