This post continues the discussion in my “The Cloud Mystery Machine” post.

Private cloud is a key 2010 objective for many Burton Group clients. However, our clients are consistently frustrated by the difficulties of on-demand service and mobility in virtualized environments. Until we get closer to having a virtual infrastructure center of the universe, these problems will persist. For example, suppose you setup soft security zoning using VMware’s vShield Zones. Does your third party orchestration product consider zoning restrictions prior to moving a VM to a particular server? Considering that VMware’s own Distributed Resource Scheduler (DRS) service doesn’t have such capabilities, it’s unlikely that any third party tool will either. In fairness, VMware hasn’t exposed such features through their SDK, so its unfair to ask vendors to support something in which they have little control.

When it comes to orchestration, everything falls apart without a central metadata store. Call it an infrastructure authority (IA), or whatever you like. The bottom line is that if a tool wants to place an object somewhere within a cloud infrastructure, there needs to be a central place where it can check to make sure the physical location offers the necessary resources (compute, memory, networking and storage I/O) and security policy isn’t violated in the process, among other concerns. We don’t need to re-invent the wheel. Instead, we need to take existing virtual infrastructure management databases and evolve them so that they can act as the central authority for all infrastructure decisions. Microsoft’s System Center suite of products already supports some extensibility and third party integration. VMware’s Virtual Center (VC) supports third party plug-in integration, but extensibility is taboo. If the VC database was extensible, issues such as downstream storage I/O would factor into VM placement decisions today. Virtual Instruments, for example, has the technology to do it, but their hands are tied. I’m hopeful that the infrastructure authority is something that VMware and Microsoft can lead in 2010. No vendor can own the universe. How many have to try and fail to prove it doesn’t work? Server hardware vendors acting as though each enterprise infrastructure should be homogeneous is a perfect example. VMware, Microsoft, Citrix, and other members of the virtualization community need to get serious about the complexities of managing an increasingly agile infrastructure, and give their software partners the APIs and meta database extensibility they need to fuel innovation.

In the end, the IA may not comprise just a single vendor solution, but involve collaboration from multiple vendors on what may emerge as one or more de facto standards. Many technical (e.g., CPU, memory, network, and storage requirements) and non-technical (e.g., security, location, organizational policy, and SLA) requirements determine the feasibility for VM mobility and placement. In my opinion, enterprises will continue to lack confidence in true virtual infrastructure/private cloud self-service and automation until we have some type of centralized infrastructure authority. What do you think?