Rational Survivability

It’s very interesting to see that now that infrastructure-as-a-service (IaaS) players like Amazon Web Services are clawing their way “up the stack” and adding more platform-as-a-service (PaaS) capabilities, that Microsoft is going “down stack” and providing IaaS capabilities by way of adding RDP and VM capabilities to Azure.

From Carl Brooks’ (@eekygeeky) article today:

Microsoft is expected to add support for Remote Desktops and virtual machines (VMs) to Windows Azure by the end of March, and the company also says that prices for Azure, now a baseline $0.12 per hour, will be subject to change every so often.

Prashant Ketkar, marketing director for Azure, said that the service would be adding Remote Desktop capabilities as soon as possible, as well as the ability to load and run virtual machine images directly on the platform. Ketkar did not give a date for the new features, but said they were the two most requested items.
…

This move begins a definite trend away from the original concept for Azure in design and execution. It was originally thought of as a programming platform only: developers would write code directly into Azure, creating applications without even being aware of the underlying operating system or virtual instances. It will now become much closer in spirit to Amazon Web Services, where users control their machines directly. Microsoft still expects Azure customers to code for the platform and not always want hands on control, but it is bowing to pressure to cede control to users at deeper and deeper levels.

One major reason for the shift is that there are vast arrays of legacy Windows applications users expect to be able to run on a Windows platform, and Microsoft doesn’t want to lose potential customers because they can’t run applications they’ve already invested in on Azure. While some users will want to start fresh, most see cloud as a way to extend what they have, not discard it.

This sets the path to allow those enterprise customers running HyperV internally to take those VMs and run them on (or in conjunction with) Azure.

Besides the obvious competition with AWS in the public cloud space, there’s also a private cloud element. As it stands now, one of the primary differentiators for VMware from the private-to-public cloud migration/portability/interoperability perspective is the concept that if you run vSphere in your enterprise, you can take the same VMs without modification and move them to a service provider who runs vCloud (based on vSphere.)

This is a very interesting and smart move by Microsoft.

/Hoff

Related articles by Zemanta

• Silent Lucidity: IaaS – Already A Dinosaur? The Evolution of PaaSasaurus Rex… (rationalsurvivability.com)
• Incomplete Thought: Virtual Machines Are the Problem, Not the Solution… (rationalsurvivability.com)
• Just A Reflective Bookmark: Microsoft’s Azure…The Dark Horse Emergeth… (rationalsurvivability.com)
• Cloud Pricing War Begins (cloudave.com)
• Windows Azure starts bringin’ in the cash (seattlepi.com)

Allan Leinwand from GigaOm wrote a great article asking “Where are the network virtual appliances?” This was followed up by another excellent post by Rich Miller.

Allan sets up the discussion describing how we’ve typically plumbed disparate physical appliances into our network infrastructure to provide discrete network and security capabilities such as load balancers, VPNs, SSL termination, firewalls, etc.  He then goes on to describe the stunted evolution of virtual appliances:

To be sure, some networking devices and appliances are now available in virtual form.  Switches and routers have begun to move toward virtualization with VMware’s vSwitch, Cisco’s Nexus 1000v, the open source Open vSwitch and routers and firewalls running in various VMs from the company I helped found, Vyatta.  For load balancers, Citrix has released a version of its Netscaler VPX software that runs on top of its virtual machine, XenServer; and Zeus Systems has an application traffic controller that can be deployed as a virtual appliance on Amazon EC2, Joyent and other public clouds.

Ultimately I think it prudent for discussion’s sake to separate routing, switching and load balancing (connectivity) from functions such as DLP, firewalls, and IDS/IPS (security) as lumping them together actually abstracts the problem which is that the latter is completely dependent upon the capabilities and functionality of the former.  This is what Allan almost gets to when describing his lament with the virtual appliance ecosystem today:

Yet the fundamental problem remains: Most networking appliances are still stuck in physical hardware — hardware that may or may not be deployed where the applications need them, which means those applications and their associated VMs can be left with major gaps in their infrastructure needs. Without a full-featured and stateful firewall to protect an application, it’s susceptible to various Internet attacks.  A missing load balancer that operates at layers three through seven leaves a gap in the need to distribute load between multiple application servers. Meanwhile, the lack of an SSL accelerator to offload processing may lead to performance issues and without an IDS device present, malicious activities may occur.  Without some (or all) of these networking appliances available in a virtual environment, a VM may find itself constrained, unable to take full advantage of the possible economic benefits.

I’ve written about this many, many times. In fact almost three years ago I created a presentation called  “The Four Horsemen of the Virtualization Security Apocalypse” which described in excruciating detail how network virtual appliances were a big ball of fail and would be for some time. I further suggested that much of the “best-of-breed” products would ultimately become “good enough” features in virtualization vendor’s hypervisor platforms.

Why?  Because there are some very real problems with virtualization (and Cloud) as it relates to connectivity and security:

1. Most of the virtual network appliances, especially those “ported” from the versions that usually run on dedicated physical hardware (COTS or proprietary) do not provide feature, performance, scale or high-availability parity; most are hobbled or require per-platform customization or re-engineering in order to function.
2. The resilience and high availability options from today’s off-the-shelf virtual connectivity does not pair well with the mobility and dynamism of de-coupled virtual machines; VMs are ultimately temporal and networks don’t like topological instability due to key components moving or disappearing
3. The performance and scale of virtual appliances still suffer when competing for I/O and resources on the same physical hosts as the guests they attempt to protect
4. Virtual connectivity is a generally a function of the VMM (or a loadable module/domain therein.) The architecture of the VMM has dramatic impact upon the architecture of the software designed to provide the connectivity and vice versa.
5. Security solutions are incredibly topology sensitive.  Given the scenario in #1 when a VM moves or is distributed across the pooled infrastructure, unless the security capabilities are already present on the physical host or the connectivity and security layers share a control plane (or at least can exchange telemetry,) things will simply break
6. Many virtualization (and especially cloud) platforms do not support protocols or topologies that many connectivity and security virtual appliances require to function (such as multicast for load balancing)
7. It’s very difficult to mimic the in-line path requirements in virtual networking environments that would otherwise force traffic passing through the connectivity layers (layers 2 through 7) up through various policy-driven security layers (virtual appliances)
8. There is no common methodology to express what security requirements the connectivity fabrics should ensure are available prior to allowing a VM to spool up let alone move
9. Virtualization vendors who provide solutions for the enterprise have rich networking capabilities natively as well as with third party connectivity partners, including VM and VMM introspection capabilities. As I wrote about here, mass-market Cloud providers such as Amazon Web Services or Rackspace Cloud have severely crippled networking.
10. Virtualization and cloud vendors generally force many security vs. performance tradeoffs when implementing introspection capabilities in their platforms: third party code running in the kernel, scheduler prioritization issues, I/O limitations, etc.
11. Much of the basic networking capabilities are being pushed lower into silicon (into the CPUs themselves) which makes virtual appliances even further removed from the guts that enable them
12. Physical appliances (in the enterprise) exist en-mass.  Many of them provide highly scalable solutions to the specific functions that Alan refers to.  The need exists, given the limitations I describe above, to provide for integration/interaction between them, the VMM and any virtual appliances in order to offload certain functions as well as provide coverage between the physical and the logical.

What does this mean?  It means that ultimately to ensure their own survival, virtualization and cloud providers will depend less upon virtual appliances and add more of the basic connectivity AND security capabilities into the VMMs themselves as its the only way to guarantee performance, scalability, resilience and satisfy the security requirements of customers. There will be new generations of protocols, APIs and control planes that will emerge to provide for this capability, but this will drive the same old integration battles we’re supposed to be absolved from with virtualization and Cloud.

Connectivity and security vendors will offer virtual replicas of their physical appliances in order to gain a foothold in virtualized/cloud environments in order to intercept traffic (think basic traps/ACL’s) and then interact with higher-performing physical appliance security service overlays or embedded line cards in service chassis.  This is especially true in enterprises but poses many challenges in software-only, mass-market cloud environments where what you’ll continue to get is simply basic connectivity and security with limited networking functionality.  This implies more and more security will be pushed into the guest and application logic layers to deal with this disconnect.

This is exactly where we are today with Cloud providers like Amazon Web Services: basic ingress-only filtering with a very simplistic, limited and abstracted set of both connectivity and security capability.  See “Dear Public Cloud Providers: Please Make Your Networking Capabilities Suck Less. Kthxbye“  Will they add more functionality?  Perhaps. The question is whether they can afford to in order to limit the impact that connecitivity and security variability/instability can bring to an environment.

That said, it’s certainly achievable, if you are willing and able to do so, to construct a completely software-based networking environment, but these environments require a complete approach and stack re-write with an operational expertise that will be hard to support for those who have spent the last 20 years working in a different paradigm and that’s a huge piece of this problem.

The connectivity layer — however integrated into the virtualized and cloud environments they seem — continues to limit how and what the security layers can do and will for some time, thus limiting the uptake of virtual network and security appliances.

Situation normal.

/Hoff

Related articles by Zemanta

• The Forthcoming Citrix/Xen/KVM Virtual Networking Stack…What Does This Mean to VMware/Cisco 1000v? (rationalsurvivability.com)
• Cloud Providers and Security “Edge” Services – Where’s The Beef? (rational survivability)
• Oh Great Security Spirit In the Cloud: Have You Seen My WAF, IPS, IDS, Firewall…
• Dear Public Cloud Providers: Please Make Your Networking Capabilities Suck Less. Kthxbye (rational survivability)
• The Four Horsemen Of the Virtualization Security Apocalypse (rational survivability)

• Xen packages build-your-own-cloud kit (theregister.co.uk)
• I Found the Missing Piece of the Virtualization Puzzle (devcentral.f5.com)
• CohesiveFT Rocks With VPNCubed For vCloud (cloudave.com)

In the Information Security community, many of us have long come to the conclusion that we are caught in what I call my “Security Hamster Sine Wave Of Pain.”  Those of us who have been doing this awhile recognize that InfoSec is a zero-sum game; it’s about staving off the inevitable and trying to ensure we can deal with the residual impact in the face of being “survivable” versus being “secure.”

While we can (and do) make incremental progress in certain areas, the collision of disruptive innovation, massive consumerization of technology along with the slow churn of security vendor roadmaps, dissolving budgets, natural marketspace commoditzation and the unfortunate velocity of attacker innovation yields the constant realization that we’re not motivated or incentivized to do the right thing or manage risk.

Instead, we’re poked in the side and haunted by the four letter word of our industry: compliance.

Compliance is often dismissed as irrelevant in the consumer space and associated instead with government or large enterprise, but as privacy continues to erode and breaches make the news, the fact that we’re putting more and more of our information — of all sorts — in the hands of others to manage is again beginning to stoke an upsurge in efforts to somehow measure and manage visibility against a standardized baseline of general, common sense and minimal efforts to guard against badness.

Ultimately, it doesn’t matter how “secure” Cloud providers suggest they are.  It doesn’t matter what breakthroughs in technology sprout up in the face of this new model of compute. The only measure that counts in the long run is how compliant you are.  That’s what will determine the success of Cloud.  Don’t believe me? Look at how the leading vendors in Cloud are responding today to their biggest (potential) customers — taking the “one size fits all” model of mass-market Cloud and beginning to chop it up and create one-off’s in order to satisfy…compliance.

Why?  Because it’s easier to deal with the vagaries of trust and isolation and multi-tenant environments by eliminating the latter to increase the former. If an auditor/examiner doesn’t understand or cannot measure your compliance to those things he/she is tasked to evaluate you against, you’re sunk.

The only thing that will budge the needle on this issue is how agile those who craft the regulatory guidelines are or how you can clearly demonstrate why your compensating controls mitigate the risk of the provider of service if they cannot. Given the nature and behavior of those involved in this space and where we are with putting our eggs in a vaporous basket, I wouldn’t hold my breath.  Movement in this area is glacial at best and in many cases out of touch with the realities of just how disruptive Cloud Computing is.  All it will take is one monumental cock-up due to a true Cloudtastrophe and the Cloud will hit the fan.

As I have oft suggested, the core issue we need to tackle in Cloud is trust, since the graceful surrender of such is at the heart of what Cloud requires.  Trust is comprised of Security, Control, Service Levels and Compliance.  It’s relatively easy to establish where we are today with the first three, but the last one is MIA.  We’re just *now* seeing movement in the form of SIGs to deal with virtualization.  Cloud?

When the best you have is a SAS-70, it’s time to weep.  Conversely, wishing for more regulation will simply extend the cycle.

What can you do?  Simple. Help educate your auditors and examiners. Read the Cloud Security Alliance’s guidelines. Participate in making the Automated Audit, Assertion, Assessment, and Assurance API (A6) a success so we can at least gain back some visibility and transparency which helps demonstrate compliance, since that’s how we’re measured.  Ultimately, if you’re able, focus on risk assessment in helping to advise your constituent business customers on how to migrate to Cloud Computing safely.

There are TONS of things one can do in order to make up for the shortcomings of Cloud security today.  The problem is, most of them erode the benefits of Cloud: agility, flexibility, cost savings, and dynamism.  We need to make the business aware of these tradeoffs as well as our auditors because we’re stuck.  We need the regulators and examiners to keep pace with technology — as painful as that might be in the short term — to guarantee our success in the long term.

Manage compliance, don’t let it manage you because a Cloud is a terrible thing to waste.

/Hoff

Related articles by Zemanta

• ENISA launches Cloud Computing Security Risk Assessment Document (rationalsurvivability.com)
• Security and the Cloud – What Does That Even Mean? (rationalsurvivability.com)
• These Apocalyptic Assessments Of Cloud Security Readiness Are Irrelevant… (rationalsurvivability.com)
• The Cloud & eHarmony’s 29 Dimensions Of Compatability… (rationalsurvivability.com)
• Cloud Computing [Security] Architectural Framework (rationalsurvivability.com)
• Jericho Forum and Cloud Security Alliance Join Forces to Address Cloud Computing Security (rationalsurvivability.com)
• From the X-Files – The Cloud in Context: Evolution from Gadgetry to Popular Culture (rationalsurvivability.com)
• Extending the Concept: A Security API for Cloud Stacks (rationalsurvivability.com)

• EuroCloud UK members making sense of Cloud standards and security (cloudave.com)
• The Cloud Computing Compliance Conundrum (datacenterknowledge.com)
• Can regulators keep up with Cloud Computing? (cloudave.com)

Sitting in an impressive room at the Google campus in Mountain View last month, I asked the collective group of brainpower a slightly rhetorical question:

How much longer do you feel pure-play Infrastructure-As-A-Service will be a relevant service model within the spectrum of cloud services?

I couched the question with previous “incomplete thoughts*” relating to the move “up-stack” by IaaS providers — providing value-added, at-cost services to both differentiate and soften the market for what I call the “PaaSification” of the consumer.  I also highlighted the move “down-stack” by SaaS vendors building out platforms to support a broader ecosystem and value proposition.

In the long term, I think ultimately the trichotomy of the SPI model will dissolve thanks to commoditization and the need for providers to differentiate — even at mass scale.  We’ll ultimately just talk about service delivery and the platform(s) used to deliver them.  Infrastructure will enable these services, of course, but that’s not where the money will come from.

Just look at the approach of providers such as Amazon, Terremark and Savvis and how they are already clawing their way up the PaaS stack, adding more features and functions that either equalize public cloud capabilities with those of the enterprise or even differentiate from it.  Look at Microsoft’s Azure.  How about Heroku, Engine Yard, Joyent?  How about VMware and Springsource?  All platform plays. Develop, click, deploy.

As I mention in my Cloudifornication presentation, I think that from a security perspective, PaaS offers the potential of eliminating entire classes of vulnerabilities in the application development lifecycle by enforcing sanitary programmatic practices across the derivate works built upon them.  I look forward also to APIs and standards that allow for consistency across providers. I think PaaS has the greatest potential to deliver this.

There are clearly trade-offs here, but as we start to move toward the two key differentiators (at least for public clouds) — management and security — I think the value of PaaS will really start to shine.

Probably just another bout of obviousness, but if I were placing bets, this is where I’d sink my nickels.

You?

/Hoff

* The most relevant “incomplete thought” is the one titled “Incomplete Thought: Virtual Machines Are the Problem, Not the Solution…” in which I kicked around the notion that virtualization-enabled IaaS and the VM containers they enable are simply an ugly solution to an uglier problem…

It’s that time again.  I am compelled after witnessing certain behaviors to play anthropologist and softly whisper my observations in your ear.

You may be familiar with Beckett’s “Waiting For Godot”*:

Waiting for Godot follows two days in the lives of a pair of men who divert themselves while they wait expectantly and unsuccessfully for someone named Godot to arrive. They claim him as an acquaintance but in fact hardly know him, admitting that they would not recognise him were they to see him. To occupy themselves, they eat, sleep, converse, argue, sing, play games, exercise, swap hats, and contemplate suicide — anything “to hold the terrible silence at bay”

Referencing my prior post about the state of Cloud security, I’m reminded of the fact that as a community of providers and consumers, we continue to wait for the security equivalent of Godot to arrive and solve all of our attendant Cloud security challenges with the offer of some mythical silver bullet.  We wait and wait for our security Godot as I mix metaphors and butcher Beckett’s opus to pass the time.

Here’s a classic illustration of hoping our way to Cloud security from a ComputerWeekly post titled “Cryptography breakthrough paves way to secure cloud services:”

A research student who had a summer job at IBM, has cracked a cryptography problem that has baffled experts for over 30 years. The breakthrough may pave the way to secure cloud computing services.

This sounds fantastic and much has been written about this “homomorphic encryption,” with many people espousing how encryption will “solve our Cloud security problems.”

It’s a very interesting concept, but as to paving the “…path to secure cloud computing,” the reality is that it won’t.  At least not in isolation and not without some serious scale in ancillary support mechanisms including non-trivial issues like federated identity.

Bruce Schneier wades in with his assessment:

Unfortunately — you knew that was coming, right? — Gentry’s scheme is completely impractical…Despite this, IBM’s PR machine has been in overdrive about the discovery. Its press release makes it sound like this new homomorphic scheme is going to rewrite the business of computing: not just cloud computing, but “enabling filters to identify spam, even in encrypted email, or protection information contained in electronic medical records.” Maybe someday, but not in my lifetime.

The reality is that in addition to utilizing encryption — both existing and new approaches — we still continue to need all the usual suspects as they deal with the fact that fundamentally we’re still in a cycle of constructing insecure code in infostructure sitting atop infrastructure and metastructure that has its own fair share of growing up to do.

As a security architect, engineer, or manager, you need to continue to invest in understanding how what you have does or does not work within the context of Cloud.

You will likely find that you will need to continue to invest in threat and trust models analysis, risk management, vulnerability assessment, (id)entity management, compensating controls implemented as hardware and software technology solutions such as firewalls, IDP, DLP, and policy instantiation, etc. as well as host of modified and new approaches to dealing with Cloud-specific implementation challenges, especially those based on virtualization and massive scale with multitenancy.

These problems don’t solve themselves and we are simply not changing our behavior.  We wait and wait for our Godot.

So here’s the obligatory grumpy statement of the obvious as providers of solutions and services churn to deliver more capable solutions to put in your hands:

There is no silver bullet, just a lot of silver buckshot.  Use it all.  You’re going to have to deal with the cards we are dealt for the foreseeable future whilst we retool our approach in the longer term and technology equalizes some of our shortfalls.

Godot is not coming and you likely wouldn’t recognize him if he showed up anyway because he’d be dressed in homomorphic invisible hotpants…

Get on with it.  Treat security as the enterprise architecture element it is and use Cloud as the excuse to make things better by working on the things that matter.

If Godot does happen to show up, tell him I want my weed whacker back that he borrowed last summer.

/Hoff

* Wikipedia

By now you’ve undoubtedly heard that Google had a little operational hiccup.  I particularly enjoyed Craig Labovitz’s (arbor) account of “The Great GoogleLapse”

When a suite of services that account for a projected 5% of the entire Intertube’s traffic shits the bed, people pay attention.

Sometimes for the wrong reasons.

Conspiracy theories, rumors of the end of days and chants of “don’t trust the Cloud!” start to fly when operational issues such as the routing boo-boo that hit Google turn up.

The reality is that in the grand scheme of things, we should take the three salient points from this experience and move on:

1. Cloud services — even those with the scale, maturity and operational track-record of Google — still depend on fundamentally weak, insecure and unstable infrastructure that is easy to screw up.
   This is the premise for my upcoming Black Hat talk titled “Cloudifornication: Indiscriminate Information Intercourse Involving Internet Infrastructure.”
2. You ought to have a Plan B. That maybe difficult as it relates to Cloud-based SaaS application offerings and service which, by definition, tend to tie you to the platform/provider offering them.
3. This isn’t going to stop anyone from moving to the Cloud.  It may give people pause and they may spend a few more cycles evaluating what Plan B might mean, but it also pushes the agendas of hybrid architectures like Google’s NaCl and client-side hypervisors for “off-line” Cloud goodness.  All in all, it’s a nice reminder, but Cloud goes on.

The economic lubricant provided by the Astro Glide that is Cloud is just too compelling. If someone hasn’t factored potential widespread outages from single-sourced providers, shame on them; that’s poor risk assessment.

Yes, we’ve got lots of attendant issues to solve when it comes to Cloud.  Many of them, I have so soapboxed, are the same ones we’ve had for a long while.  To those of us who recognize the Internet Cloud for what it is, Google’s outage was simply an opportunity to order another Hoffachino.

What doesn’t kill us makes us…just as insecure and potentially unavailable due to some monkey pushing the wrong button as we’ve always been.

Besides, now we know that outsourcing your traffic to China is the sux0r.

So chill.  Learn from this.  Use it to form rational arguments about how to deal with this sort of thing when it does happen — because it’s going to again, just like it always has.  Remember?

Worse comes to worse, may I suggest one of these — it is the cure for all your woes anyway, right?

/Hoff