What To Do When Your “Core” Infrastructure Services Aren’t In Your “Core?”
Okay. I am teh lam3r. I'd be intellectually dishonest if I didn't post this, and it's likely I'll revise it once I get to think about it more, but I've got to get it down. Thanks to an innocent tweet from @botchagalupe I had an
aneurysm epiphany. Sort of 😉
A little light went on in my head this morning regarding how the cloud, or more specifically layers of clouds and the functions they provide (a-la SOA,) dramatically impact the changing landscape of what we consider "core infrastructure services," our choices on architecture, service provisioning, and how and from whence they are provided.
Specifically, the synapse fired on the connection between Infrastructure 2.0 as is usually talked about from the perspective of the evolution from the enterprise inside to out versus the deployment of services constructed from scratch to play in the cloud.
You've no doubt seen discussions from Greg Ness (InfoBlox) and Lori Mac Vittie (f5) regarding their interpretation of Infrastructure 2.0 and the notion that by decoupling infrastructure services from their physical affinity we can actually "…enable greater levels of integration between the disparate layers of infrastructure: network, application, the endpoint, and IP address management, necessary to achieve interconnectedness."
Totally agree. Been there, done that, bought the T-Shirt, but something wasn't clicking as it relates to what this means relative to cloud.
I was slurping down some java this morning and three things popped into my head as I was flipping between Twitter and Google Reader wondering about how I might consider launching a cloud-based service architecture and what impact it would have on my choices for infrastructure and providers.
Here are the three things that I started to think about in regards to what "infrastructure 2.0" might mean to me in this process, beyond the normal criteria related to management, security, scalability, etc…
- I always looked at these discussions of Infrastructure 2.0 as ideation/marketing by vendors on how to take products that used to function in the "Infratructure 1.0" dominion, add a service control plane/channel and adapt them for the inside-out version of the new world order that is cloud. This is the same sort of thing we've dealt with for decades and was highlighted when one day we all discovered the Internet and had to connect to it — although in that case we had standards!
- Clouds are often discussed in either microcosmic vacuum or lofty, fluffy immensity and it makes it hard to see the stratosphere for the cirrocumulus. Our "non-cloud" internal enterprises today are conglomerates of technology integration with pockets of core services which provide the underpinnings for much of what keeps the machinery running. Cloud computing is similar in approach, but in this regard, it brings home again the point that there is no such thing as "THE Cloud" but rather that the overarching integration challenge lays in the notion of overlays or mash-ups of multiple clouds, their functions, and their associated platforms and API's.
- Further, and as to my last blog post on private clouds and location independence, I really do believe that the notion of internal versus external clouds is moot, but that the definitional nuance of public versus private clouds — and their requisite control requirements — are quite important. Where, why, how and by whom services are provided becomes challenging because the distinction between inside and out can be really, really fuzzy, even more so if you're entirely cloud based in the first place.
For some reason, my thinking never really coalesced on how what relevance these three points have as it relates to the delivery of a service (and thus layers of applications) in a purely cloud based architecture built from scratch without the encumbrance of legacy infrastructure solutions.
I found this awesome blog post from Mike Brittain via a tweet from @botchagalupe titled "How we built a web hosting infrastructure on EC2" and even though the article is a fascinating read, the single diagram in the post hit me like a hammer in the head…and I don't know why it did, because it's not THAT profound, but it jiggled something loose that is probably obvious to everyone else already:
Do you see the first three layers? Besides the "Internet," as the transport, you'll see two of the most important service delivery functions staring back at you: Akamai's "Site Accelerator Proxy" CDN/Caching/Optimization offering and Neustar's "UltraDNS" distributed, topologically intelligent DNS services
Both of these critical services (one might say "core infrastructure 2.0" services) are, themselves, cloud-based. Of course, the entire EC2/S3 environment which hosts the web services is cloud-based, too.
The reason the light bulb went on for me is that I found that I was still caught in the old school infrastructure-as-a-box line of thought when it came to how I might provide the CDN/Caching and distributed DNS capabilities of my imaginary service.
It's likely I would have dropped right to the weeds and started thinking about which geographic load balancers (boxes) and/or proxies I might deploy somewhere and how (or if) they might integrate with the cloud "hosting/platform provider" to give me the resiliency and dynamic capabilities I wanted, let alone firewalls, IDP, etc.
Do I pick a provider that offers as part of the infrastructure a specific hardware-based load-balancing platform? Do I pick on that can accommodate the integration of a software-based virtual appliances. Should I care? With the cloud I'm not supposed to, but I find that I still, for many reasons — good and bad — do.
I never really thought about simply using a cloud-based service as a component in a mash-up of services that already does these things in ways that would be much cheaper, simpler, resilient and scalable than I could construct with "infrastructure 1.0" thinking. Heck, I could pick 2 or 3 of them, perhaps.
That being said, I've used outsourced "cloud-based" email filtering, vulnerability management, intrusion detection & prevention services, etc., but there are still some functions that for some reason appear to sacrosanct in the recesses of my mind?
I think I always just assumed that the stacking of outsourced (commoditized) services across multiple providers would be too complex but in reality, it's not very different from my internal enterprise that has taken decades to mature many of these functions (and consolidate them.)
Despite the relative immaturity of the cloud, it's instantly benefited from this evolution. Now, we're not quite all the way there yet. We still are lacking standards and that service control plane shared amongst service layers doesn't really exist.
I think it's a huge step to recognize that it's time to get over the bias of applying so called "infrastructure 1.0" requirements to the rules of engagement in the cloud by recognizing that many of these capabilities don't exist in the enterprise, either.
Now, it's highly likely that the two players above (Neustar and Akamai) may very well use the same boxes that *I* might have chosen anyway, but it's irrelevant. It's all about the service and engineering enough resiliency into the design (and choices of providers) such that I mitigate the risk of perhaps not having that "best of breed" name plate on a fancy set of equipment somewhere.
I can't believe the trap I fell into in terms of my first knee-jerk reaction regarding architecture, especially since I've spent so much of the last 5 years helping architect and implement "cloud" or "cloud-like" security services for outsourced capabilities.
So anyway, you're probably sitting here saying "hey, idiot, this is rather obvious and is the entire underlying premise of this cloud thing you supposedly understand inside and out." That comment would be well deserved, but I had to be honest and tell you that it never really clicked until I saw this fantastic example from Mike.