Rational Survivability

I’ve tripped over it a couple of times.

I’ve done things to it and with it that perhaps I shouldn’t have.

I’ve even rebooted it once or twice.

On Thursday, I tried — unsuccessfully — to once and for all take down the Internet.

It’s he’s just too damned resilient for his own good. 😉

One of my heroes…and an awesome person. Thank you, Vint.

You can read about the exploits of the Infrastructure 2.0 Working Group at SRI from Greg Ness’ blog here.

/Hoff

Update: Ooops.  I forgot to announce that I'm once again putting on my Devil's Advocacy cap. It fits nicely and the contrasting color makes my eyes pop.;)

It should be noted that obviously I recognize that dedicated
hardware offers performance and scale capabilities that in many cases
are difficult (if not impossible) to replicate in virtualized software
instantiations of the same functionality. 

However, despite spending the best part of two years raising
awareness as to the issues surrounding scalability, resiliency,
performance, etc. of security software solutions in virtualized
environments via my Four Horsemen of the Virtualization Security Apocalypse presentation, perception is different
than reality and many network capabilities will simply consolidate into the virtualization platforms until the next big swing of the punctuated equlibrium.

This is another classic example of "best of breed" versus "good enough" and in many cases this debate becomes a corner-case argument of speeds and
feeds and the context/location of the network topology you're talking
about. There's simply no way to sprinkle enough specialized hardware around to get the pervasive autonomics across the entire fabric/cloud without a huge chunk of it existing in the underlying virtualization platform or underlying network infrastructure.

THIS is the real scaling problem that software can address (by penetration) that specialized hardware cannot.

There will always be a need for dedicated hardware for specific needs, and if you have an infrastructure service issue that requires massive hardware to support traffic loads until the sophistication and technology within the virtualization layer catches up, by all means use it!  In fact, just today after writing this piece Joyent announced they use f5 BigIP's to power their IaaS cloud service…

In the longer term, however, application delivery control (ADC) will ultimately become a feature of the virtual networking stack provided by software as part of a larger provisioning/governance/autonomics challenge provided by the virtualization layer.  If you're going to get as close to this new atomic unit of measurement in the VM, you're going to have to decide where the network ends and the virtualization layer begins…across every cloud you expect to host your apps and those they may transit.

I've been reading Lori McVittie's f5 DevCentral blog for quite some time.  She and Greg Ness have been feeding off one another's commentary in their discussion on "Infrastructure 2.0" and the unique set of challenges that the dynamic nature of virtualization and cloud computing place on "the network" and the corresponding service layers that tie applications and infrastructure together.

The interesting thing to me is that why I do not disagree that that the infrastructure must adapt to the liquidity, agility and flexibility enabled by virtualization and become more instrumented as to the things running atop it, much of the functionality Greg and Lori allude to will ultimately become a function of the virtualization and cloud layers themselves*.

One of the more interesting memes is the one Lori summarized this morning in her post titled "Managing Virtual Infrastructure Requires an Application Centric Approach," wherein the she lays the case for the needs of infrastructure becoming "application" centric based upon the "highly dynamic" nature of virtualized and cloud computing environments:

…and yet the applications themselves, despite how integrated they may be, suffer from the same horizontal management problem as the network today does.  So I'm not so sure about the finality of the "it's all about the application" because we haven't even solved the "virtual infrastructure management" issues yet.

Bridging the gap between where we are today and the infrastructure 2.0/application-centric focus of tomorrow is illustrated nicely by Billy Marshall from rPath in his post titled "The Virtual Machine Tsunami," in which he describes how we're really still stuck being VM-centric as the unit measure of application management:

So we still have all the problems of managing the applications atomically, but I think there's some general agreement between these two depictions.

However, where it gets interesting is where Lori essentially paints the case that "the network" today is unable to properly provide for the delivery of applications:

This is where I start to squint a little because Lori's really taking the notion of "application intelligence" and painting what amounts to a router/switch in an appliction delivery controller as a "platform" as she attempts to drive wedge between an ADC and "the network."

Besides the fact that "the network" is also rapidly evolving to adapt to this more loosely-coupled model and the virtualization layer, the traditional networking functions and the infrastructure service layers are becoming more integrated and aware thanks to the homgenizing effect of the hypervisor, I'll ask the question I asked Lori on Twitter this morning:

Why won't this ADC functionality simply show up in the hypervisor?  If you ask me, that's exactly the goal.  vCloud, anyone?  Amazon EC2?  Azure?

If we take the example of Cisco and VMware, the coupled vision of the networking and virtualization 800 lb gorillas is exactly the same as she pens above; but it goes further because it addresses the end-to-end orchestration of infrastructure across the network, compute and storage fabrics.

So, why do we need yet another layer of network routers/switches called "application delivery controllers" as opposed to having this capability baked into the virtualization layer or ultimately the network itself?

So, can you really make the case for deploying more "application-centric" routers/switches (which is what an application delivery controller is) regardless of how aware it may be?

/Hoff

I had a  briefing with Patrick Kerpan and Craig Heimark of CohesiveFT last week in response to some comments that Craig had left on my blog post regarding PCI compliance in the Cloud, here. 

I was intrigued, albeit skeptically, with how CohesiveFT was positioning the use of VPNs within the cloud and differentiating their solution from the very well established IPSec and SSL VPN capabilities we all know and love.  What's so special about tunnels across the Intertubes?  Been there, done that, bought the T-Shirt, right?

So I asked the questions…

I have to say that unlike many other companies rushing to associate their brand by rubber cementing the word "cloud" and "security" onto their product names and marketing materials, CohesiveFT spent considerable time upfront describing what they do not do so as to effectively establish what they are and what they do.  I'll let one of their slides do the talking:

Okay, so they're not a cloud, but they provide cloud and virtualization services…and VPN-Cubed provides some layer of security along with their products and services…check. But…

Digging a little deeper, I still sought to understand why I couldn't just stand up an IPSec tunnel from my corporate datacenter to one or more cloud providers where my assets and data were hosted.  I asked for two things: (1) A couple of sentences summarizing their elevator pitch for VPN-Cubed and (2) a visual representation of what this might look like.

Here's what I got as an answer to question (1):

Here are two great visuals to address question (2):

 

So the differences between a typical VPN and VPN-Cubed comes down to being able to securely extend your "internal clouds infrastructure" in your datacenters (gasp! I said it) in a highly-available manner to include your externally hosted assets which in turn run on infrastructure you don't own.  You can't stand up an ASA or Neoteris box on a network you can't get to.  The VPN-Cubed Managers are VM's/VA's that run as a complement to your virtual servers/machines hosted by your choice of one or multiple cloud providers.

They become the highly-available, multiprotocol aribters of access via standardized IPSec protocols but do so in a way that addresses the dynamic capabilities of the cloud which includes service governance, load, and "cloudbursting" failover between clouds — in a transparent manner.

Essentially you get secure access to your critical assets utilizing an infrastructure independent solution, extending the VLAN segmentation, isolation and security capabilities your providers may put in place while also controlling your address spaces within the cloudspaces encompassing your VM's "behind" the VPN Managers.

VPN-Cubed is really a prime example of the collision space of dynamic application/service delivery, virtualization, security and cloud capacity governance.  It speaks a lot to re-perimeterization that I've been yapping about for quite some time and hints at Greg Ness' Infrastructure 2.0 meme.

Currently VPN-Cubed is bundled as a package which includes both product and services and supports the majority of market-leading virtualization formats, operating systems and cloud providers such as Amazon EC2, Flexiscale, GoGrid, Mosso, etc.

It's a very slick offering.

/Hoff

In my previous post titled "Cloud Computing: Invented By Criminals, Secured By ???" I described the need for a new security model, methodology and set of technologies in the virtualized and cloud computing realms built to deal with the dynamic and distributed nature of evolving computing:

This
basically means that we should distribute the sampling, detection and
prevention functions across the entire networked ecosystem, not just to
dedicated security appliances; each of the end nodes should communicate
using a standard signaling and telemetry protocol so that common
threat, vulnerability and effective disposition can be communicated up
and downstream to one another and one or more management facilities.

Greg Ness from Infoblox reminded me in the comments of that post of something I was very excited about when it
became news at InterOp this last April: the Trusted Computing Group's (TCG) extension to the Trusted Network Connect (TNC) architecture called IF-MAP.

IF-MAP is a standardized real-time publish/subscribe/search mechanism which utilizies a client/server, XML-based SOAP protocol to provide information about network security objects and events including their state and activity:

While the TNC was initially designed to support NAC solutions, extending the capabilities to any security product to subscribe to a common telemetry and information exchange/integration protocol is a fantastic idea.

I'm really interested in how many vendors outside of the NAC space are including IF-MAP in their roadmaps. While IF-MAP has potential in convential non-virtualized infrastructure, I see a tremendous need for it in our move to Infrastructure 2.0 with virtualization and Cloud Computing. 

Integrating, for example, IF-MAP with VM-Introspection capabilities (in VMsafe, XenAccess, etc.) would be fantastic as you could tie the control planes of the hypervisors, management infrastructure, and provisioning/governance engines with that of security and compliance in near-time.

You can read more about the TCG's TNC IF-MAP specification here.

/Hoff

* Update:  A few hours after writing this last night, AWS announced they had achieved Level 1 PCI DSS Compliance.* If you pay attention to how the announcement is worded, you’ll find a reasonable treatment of what PCI compliance means to an IaaS cloud provider – it’s actually the first time I’ve seen this honestly described:

Merchants and other service providers can now run their applications on AWS PCI-compliant technology infrastructure to store, process and transmit credit card information in the cloud. Customers can use AWS cloud infrastructure, which has been validated at the highest level (Level 1) of PCI compliance, to build their cardholder environment and achieve PCI certification for their applications.

Note how they phrased this, then read my original post below.

However, pay no attention to the fact that they chose to make this announcement on Pearl Harbor Day 😉

—

Here’s the thing…

A cloud provider can achieve compliance (such as PCI — yes v2.0 even) such that the in-scope elements of that provider which are audited and assessed can ultimately contribute to the compliance of a customer operating atop that environment.  We’ve seen a number of providers assert compliance across many fronts, but they marketed their way into a yellow card by over-reaching…

It should be clear already, but for a service to be considered compliant, it clearly means that the customer’s in-scope elements running atop a cloud provider must also undergo and achieve compliance.

That means compliance is elementally additive the same way “security” is when someone else has direct operational control over elements in the stack you don’t.

In the case of an IaaS cloud provider who may achieve compliance from the “concrete to the hypervisor,” (let’s use PCI again,) the customer in turn must have the contents of the virtual machine (OS, Applications, operations, controls, etc.) independently assessed and meet PCI compliance in order that the entire stack of in-scope elements can be described as compliant.

Thus security — and more specifically compliance — in IaaS (and PaaS) is a shared responsibility.

I’ve spent many a blog battling marketing dragons from cloud providers that assert or imply that by only using said provider’s network which has undergone and passed one or more audits against a compliance framework, that any of its customers magically inherit certification by default. I trust this is recognized as completely false.

As compliance frameworks catch up to the unique use-cases that multi-tenancy and technologies such as virtualization bring, we’ll see more “compliant cloud” offerings spring up, easing customer pain related to the underlying moving parts.  This is, for example, what FedRAMP is aiming to provide with “pre-approved” cloud offerings.  We’ve got visibility and transparency issues to solve , as well as temporal issues such as the frequency and period of compliance audits, but there’s progress.

We’re going to see more and more of this as infrastructure- and platform-as-a-service vendors look to mutually accelerate compliance to achieve that which software-as-a-service can more organically deliver as a function of stack control.

/Hoff

* Note: It’s still a little unclear to me how some of the PCI requirements are met in an environment like an IaaS Cloud provider where “applications” that we typically think of that traffic in PCI in-scope data don’t exist (but the infrastructure does,) but I would assume that AWS leverages other certifications such as SAS and ISO as a cumulative to petition the QSA for consideration during certification.  I’ll ask this question of AWS and see what I get back.

Related articles

• Navigating PCI DSS (2.0) – Related to Virtualization/Cloud, May the Schwartz Be With You! (rationalsurvivability.com)
• Seeing Through The Clouds: Understanding the Options and Issues (thesecuritysamurai.com)
• I’ll Say It Again: Security Is NOT the Biggest Barrier To Cloud… (rationalsurvivability.com)
• PCI 2.0 Allows For Compliant Clouds, Make Sure Your Service Providers Are PCI Compliant (securecloudreview.com)
• What’s The Problem With Cloud Security? There’s Too Much Of It… (rationalsurvivability.com)
• FedRAMP. My First Impression? We’re Gonna Need A Bigger Boat… (rationalsurvivability.com)
• Mixed Mode and PCI DSS 2.0 (brandenwilliams.com)
• The Future Of Audit & Compliance Is…Facebook? (rationalsurvivability.com)
• Incomplete Thought: Compliance – The Autotune Of The Security Industry (rationalsurvivability.com)

After my initial post yesterday (How To Wield the New vShield (Edge, App & Endpoint) remarking on the general sessions I sat through on vShield, I thought I’d add some additional color given my hands-on experience in the labs today.

I will reserve more extensive technical analysis of vShield Edge and App (I didn’t get to play with endpoint as there is not a lab for that) once I spend some additional quality-time with the products as they emerge.

Because people always desire for me to pop out of the cake quickly, here you go:

You should walk away from this post understanding that I think the approach holds promise within the scope of what VMware is trying to deliver. I think it can and will offer customers choice and flexibility in their security architecture and I think it addresses some serious segmentation, security and compliance gaps. It is a dramatically impactful set of solutions that is disruptive to the security and networking ecosystem. It should drive some interesting change. The proof, as they say, will be in the vPudding.

Let me first say that from VMware’s perspective I think vShield “2.0” (which logically represents many technologies and adjusted roadmaps both old and new) is clearly an important and integral part of both vSphere and vCloud Director’s future implementation strategies. It’s clear that VMware took a good, hard look at their security solution strategy and made some important and strategically-differentiated investments in this regard.

All things told, I think it’s a very good strategy for them and ultimately their customers. However, there will be some very interesting side-effects from these new features.

vShield Edge is as disruptive to the networking space (it provides L3+ networking, VPN, DHCP and NAT capabilities at the vDC edge) as it is to the security arena. When coupled with vShield App (and ultimately endpoint) you can expect VMware’s aggressive activity in retooling their offers here to cause further hastened organic development, investment, and consolidation via M&A in the security space as other vendors seek to play and complement the reabsorption of critical security capabilities back into the platform itself.

Now all of the goodness that this renewed security strategy brings also has some warts. I’ll get into some of them as I gain more hands-on experience and get some questions answered, but here’s the Cliff Note version with THREE really important points:

1. The vShield suite is the more refined/retooled/repaired approach toward what VMware promised in delivery three years ago when I wrote about it in 2007 (Opening VMM/HyperVisors to Third Parties via API’s – Goodness or the Apocalypse?) and later in 2008 (VMware’s VMsafe: The Good, the Bad, and the Bubbly…“) and from 2009, lest we forget The Cart Before the Virtual Horse: VMware’s vShield/Zones vs. VMsafe API’s…
   _
   Specifically, as the virtualization platform has matured, so has the Company’s realization that security is something they are going to have to take seriously and productize themselves as depending upon an ecosystem wasn’t working — mostly because doing so meant that the ecosystem had to uproot entire product roadmaps to deliver solutions and it was a game of “supply vs. demand chicken.”
   _
   However, much of this new capability isn’t fully baked yet, especially from the perspective of integration and usability and even feature set capabilities such as IPv6 support. Endpoint is basically the more streamlined application of APIs and libraries for anti-malware offloading so as to relieve a third party ISV from having to write fastpath drivers that sit in the kernel/VMM and disrupt their roadmaps. vShield App is the Zones solution polished to provide inter-VM firewalling capabilities.
   _
   Edge is really the new piece here and represents a new function to represent vDC perimeterized security capabilities.Many of these features are billed — quite openly — as relieving a customer from needing to use/deploy physical networking or security products. In fact, in some cases even virtual networking products such as the Cisco Nexus 1000v are not usable/supportable. This is and example of a reasonably closed, software-driven world of Cloud where the underlying infrastructure below the hypervisor doesn’t matter…until it does.
   _
2. vShield Edge and App are, in the way they are currently configured and managed, very complex and unwieldy and the performance, resiliency and scale described in some of the sessions is yet unproven and in some cases represents serious architectural deficiencies at first blush. There are some nasty single points of failure in the engineering (as described) and it’s unclear how many reference architectures for large enterprise and service provider scale Cloud use have really been thought through given some of these issues.
   _
   As an example, only being able to instantiate a single (but required) vShield App virtual appliance per ESX host brings into focus serious scale, security architecture and resilience issues. Being able to deploy numerous Edge appliances brings into focus manageability and policy sprawl concerns.There are so many knobs and levers leveraged across the stack that it’s going to be very difficult in large environments to reconcile policy spread over the three (I only interacted with two) components and that says nothing about then integrating/interoperating with third party vSwitches, physical switches, virtual and physical security appliances. If you think it was challenging before, you ain’t seen nothin’ yet.
   _
3. The current deployment methodology reignites the battle that started to rage when security teams lost visibility into the security and networking layers and the virtual administrators controlled the infrastructure from the pNIC up. This takes the gap-filler virtual security solutions from small third parties such as Altor which played nicely with vCenter but allowed the security teams to manage policy and blows that model up. Now, security enforcement is a commodity feature delivered via the virtualization platform but requires too complex a set of knowledge and expertise of the underlying virtualization platform to be rendered effective by role-driven security teams.

While I’ll cover items #1 and #2 in a follow-on post, here’s what VMware can do in the short term to remedy what I think is a huges issue going forward with item #3, usability and management.

Specifically, in the same way vCloud Director sits above vCenter and abstracts away much of the “unnecessary internals” to present a simplified service catalog of resources/services to a consumer, VMware needs to provide a dedicated security administrator’s “portal” or management plane which unites the creation, management and deployment of policy from a SECURITY perspective of the various disparate functions offered by vShield App, Edge and Endpoint. [ED: This looks as though this might be what vShield Manager will address. There were no labs covering this and no session I saw gave any details on this offering (UI or API)]

If you expect a security administrator to have the in-depth knowledge of how to administer the entire (complex) virtualization platform in order to manage security, this model will break and cause tremendous friction. A security administrator shouldn’t have access to vCenter directly or even the vCloud Director interfaces.

Since much of the capability for automation and configuration is made available via API, the notion of building a purposed security interface to do so shouldn’t be that big of a deal. Some people might say that VMware should focus on building API capabilities and allow the ecosystem to fill the void with solutions that take advantage of the interfaces. The problem is that this strategy has not produced solutions that have enjoyed traction today and it’s quite clear that VMware is interested in controlling their own destiny in terms of Edge and App while allowing the rest of the world to play with Endpoint.

I’m sure I’m missing things and that given the exposure I’ve had (without any in-depth briefings) there may be material issues associated with where the products are given their early status, but I think it important to get these thoughts out of my head so I can chart their accuracy and it gives me a good reference point to direct the product managers to when they want to scalp me for heresy.

There’s an enormous amount of detail that I want to/can get into. The last time I did that it ended up in a 150 slide presentation I delivered at Black Hat…

Allow me to reiterate what I said in the beginning:

You should walk away from this post understanding that I think the approach holds promised within the scope of what VMware is trying to deliver. I think it can and will offer customers choice and flexibility in their security architecture and I think it addresses some serious segmentation, security and compliance gaps. It is a dramatically impactful set of solutions that is disruptive to the security and networking ecosystem. It should drive some interesting change. The proof, as they say, will be in the vPudding.

…and we all love vPudding.

/Hoff

Related articles by Zemanta

• How To Wield the New vShield (Edge, App & Endpoint) (rationalsurvivability.com)
• VMWare launches six new vCloud products (zdnet.com)
• VMworld: New cloud security emphasis from VMware (v3.co.uk)
• VMware acquires Integrien, TriCipher for IT-as-a-Service Era (zdnet.com)
• Trend Micro Announces Trend Micro(TM) Deep Security 7.5: New Agentless Anti-Malware Module for VMware Environments Provides Unprecedented Security, Manageability and Performance for Dynamic Datacenters (newswire.ca)

You’ll forgive my impertinence, but the last time I saw a similar claim of a PCI compliant Cloud offering, it turned out rather anti-climatically for RackSpace/Mosso, so I just want to make sure I understand what is really being said.  I may be mixing things up in asking my questions, so hopefully someone can shed some light.

This press release announces that:

“…Verizon’s On-Demand Cloud Computing Solution First to Achieve PCI Compliance” and the company’s cloud computing solution called Computing as a Service (CaaS) which is “…delivered from Verizon cloud centers in the U.S. and Europe, is the first cloud-based solution to successfully complete the Payment Card Industry Data Security Standard (PCI DSS) audit for storing, processing and transmitting credit card information.”

It’s unclear to me (at least) what’s considered in scope and what level/type of PCI certification we’re talking about here since it doesn’t appear that the underlying offering itself is merchant or transactional in nature, but rather Verizon is operating as a service provider that stores, processes, and transmits cardholder data on behalf of another entity.

Here’s what the article says about what Verizon undertook for DSS validation:

To become PCI DSS-validated, Verizon CaaS underwent a comprehensive third-party examination of its policies, procedures and technical systems, as well as an on-site assessment and systemwide vulnerability scan.

I’m interested in the underlying mechanicals of the CaaS offering.  Specifically, it would appear that the platform – compute, network, and storage — are virtualized.  What is unclear is if the [physical] resources allocated to a customer are dedicated or shared (multi-tenant,) regardless of virtualization.

According to this article in The Register (dated 2009,) the infrastructure is composed like this:

The CaaS offering from Verizon takes x64 server from Hewlett-Packard and slaps VMware’s ESX Server hypervisor and Red Hat Enterprise Linux instances atop it, allowing customers to set up and manage virtualized RHEL partitions and their applications. Based on the customer portal screen shots, the CaaS service also supports Microsoft’s Windows Server 2003 operating system.

Some details emerge from the Verizon website that describes the environment more:

Every virtual farm comes securely bundled with a virtual load balancer, a virtual firewall, and defined network space. Once the farm is designed, built, and named – all in a matter of minutes through the CaaS Customer Management Portal – you can then choose whether you want to manage the servers in-house or have us manage them for you.

If the customer chooses to manage the “servers…in-house (sic)” is the customer’s network, staff and practices now in-scope as part of Verizon’s CaaS validation? Where does the line start/stop?

I’m very interested in the virtual load balancer (Zeus ZXTM perhaps?) and the virtual firewall (vShield? Altor? Reflex? VMsafe-API enabled Virtual Appliance?)  What about other controls (preventitive or detective such as IDS, IPS, AV, etc.)

The reason for my interest is how, if these resources are indeed shared, they are partitioned/configured and kept isolated especially in light of the fact that:

Customers have the flexibility to connect to their CaaS environment through our global IP backbone or by leveraging the Verizon Private IP network (our Layer 3 MPLS VPN) for secure communication with mission critical and back office systems.

It’s clear that Verizon has no dominion over what’s contained in the VM’s atop the hypervisor, but what about the network to which these virtualized compute resources are connected?

So for me, all this all comes down to scope. I’m trying to figure out what is actually included in this certification, what components in the stack were audited and how.  It’s not clear I’m going to get answers, but I thought I’d ask any way.

Oh, by the way, transparency and auditability would be swell for an environment such as this. How about CloudAudit? We even have a PCI DSS CompliancePack 😉

Question for my QSA peeps: Are service providers required to also adhere to sections like 6.6 (WAF/Binary analysis) of their offerings even if they are not acting as a merchant?

/Hoff

Related articles by Zemanta

• PCI DSS Compliance and IT Security: Harmony or Discord? (prweb.com)
• Brief PCI Council Interview in Regards to PCI DSS 2.0 (chuvakin.blogspot.com)
• Revisions to Credit Card Security Standard on the Way (pcworld.com)
• Data Encryption for PCI 101: Introduction (securosis.com)
• Why your QSA should not be your Security Partner (brandenwilliams.com)
• Ask HN: Are you PCI DSS compliant? (pcisecuritystandards.org)
• Can You Have a PCI Compliant Virtualized Web Site? (securecloudreview.com)

“Cloud Security Will  Supplant Patching…”

What a sexy-sounding claim in this Network World piece which is titled with the opposite suggestion from the title of my blog post.  We will still need patching.  I agree, however, that how it’s delivered needs to change.

Before we get to the issues I have, I do want to point out that the article — despite it’s title —  is focused on the newest release of Qualys’ Laws of Vulnerability 2.0 report (pdf,) which is the latest version of the Half Lives of Vulnerability study that my friend Gerhardt Eschelbeck started some years ago.

In the report, the new author, Qualys’ current CTO Wolfgang Kandek, delivers a really disappointing statistic:

In five years, the average time taken by companies to patch vulnerabilities had decreased by only one day, from 60 days to 59 days, at a time when the number of flaws and the speed at which they are being exploited has accelerated from weeks to, in some cases, days. During the same period, the number of IP scanned on an anonymous basis by the company from its customer base had increased from 3 million to a statistically significant 80 million, with the number of vulnerabilities uncovered rocketing from 3 million to 680 million. Of the latter, 72 million were rated by Qualys as being of ‘critical’ severity.

That lack of progress is sobering, right? So far I’m intrigued, but then that article goes off the reservation by quoting Wolfgang as saying:

Taken together, the statistics suggested that a new solution would be needed in order to make further improvement with the only likely candidate on the horizon being cloud computing. “We believe that cloud security providers can be held to a higher standard in terms of security,” said Kandek. “Cloud vendors can come in and do a much better job.”  Unlike corporate admins for whom patching was a sometimes complex burden, in a cloud environment, patching applications would be more technically predictable – the small risk of ‘breaking’ an application after patching it would be nearly removed, he said.

Qualys has its head up its SaaS.  I mean that in the most polite of ways… 😉

Let me make a couple of important observations on the heels of those I’ve already made and an excellent one Lori MacVittie made today in here post titled “The Real Meaning Of Cloud Security Revealed:”

1. I’d like a better definition of the context of “patching applications.”  I don’t know whether Kandek mean applications in an enterprise or those hosted by a Cloud Provider or both?
2. There’s a difference between providing security services via the Cloud versus securing Cloud and its application/data.  The quotes above mix the issues.  A “Cloud Security” provider like Qualys can absolutely provide excellent solutions to many of the problems we have today associated with point product deployments of security functions across the enterprise. Anti-spam and vulnerability management are excellent examples.  What that does not mean is that the applications that run in an enterprise can be delivered and deployed more “securely” thanks to the efforts of the same providers.
3. To that point, the Cloud is not all SaaS-based.  Not every application is going to be or can be moved to a SaaS.  Patching legacy applications (or hosting them for that matter) can be extremely difficult.  Virtualization certainly comes into play here, but by definition, that’s an IaaS/PaaS opportunity, not a SaaS one.
4. While SaaS providers who do “own the entire stack” are in a better position through consolidated multi-tenancy to transfer the responsibility of patching “their” infrastructure and application(s) on your behalf, it doesn’t really mean they do it any better on an application-by-application basis.  If a SaaS provider only has 1-2 apps to manage (with lots of customers) versus an enterprise with hundreds (and lost of customers,) the “quality” measurements as it relates to management of defect (from any perspective) would likely look better were you the competent SaaS vendor mentioned in this article.  You can see my point here.
5. If you add in PaaS and IaaS as opposed to simply SaaS (as managed by a third party.) then the statement that “…patching applications would be more technically predictable – the small risk of ‘breaking’ an application after patching it would be nearly removed” is false.

It’s really, really important to compare apples to apples here. Qualys is a fantastic company with a visionary leader in Phillipe Courtot.  I was an early adopter of his SaaS service.  I was on his Customer Advisory Board.  However, as I pointed out to him at the Jericho event where I was a panelist, delivering a security function via the Cloud is not the same thing as securing it and SaaS is merely one piece of the puzzle.

I wrote a couple of other blogs about this topic:

• Patching the Cloud
• What People Really Mean When They Say The Cloud Is More Secure
• Cloud Providers Are Better At Securing Your Data Than You Are…
• Reprise: On-Demand SaaS Vendors Able to Secure Assets Better than Customers?

/Hoff

It finally dawned on me that if I have a few hundred to a thousand people sitting in front of me at one of my presentations, I should take advantage of that collective intelligence to perform a little selfish information gathering.

I’ve had an opinion for quite some time that the rampant squawking and generalizations regarding hyper-mobility suggesting VM sprawl and uncontrolled instance spawning was nothing more than FUD given where we are today with the technology and platforms that supposedly enable it.

We constantly hear how organizations big and small are suffering (or will) from the evils of virtualization by way of VM’s and information turning up everywhere, putting your data and assets at risk. It gets worse with the multi-tenancy issues surrounding moving to “The Cloud,” they say.

So in a couple of my panels at RSA, I asked for some sanity and fact checking.

Informally, 95% of those in attendance at the two RSA panels I engaged run VMware in production. I asked that in cases OTHER than failure, how many of those in the audience take advantage of VM mobility (such as VMotion) or some other technological capability to provide autonomic mobility of VM’s in their enterprises.

About 5 people (in crowds of 100+ and 500+ respectively) raised their hands.  Given that I asked this question the second time in front of a huge audience at RSA sitting next to the CTO’s of Citrix and VMware, I’m sure they were pretty surprised by the answer, too.

The reality is that in these environments — even extremely complex and large examples — there simply isn’t that much mobility and customers are more interested in resilience than they are agility in terms of what this feature brings. That’s a really interesting and important point.

The reason for this is pretty simple; the capability to provide for integrated networking and virtualization coupled with governance and autonomics simply isn’t mature at this point. Most people are simply replicating existing zoned/perimertized non-virtualized network topologies in their consolidated virtualized environments and waiting for the platforms to catch up. We’re really still seeing the effects of what virtualization is doing to the classical core/distribution/access design methodology as it relates to how shackled much of this mobility is to critical components like DNS and IP addressing and layer 2 VLANs.  See Greg Ness and Lori Macvittie’s scribblings.

Furthermore, Workload distribution is simply impractical for anything other than monolithic stacks because the virtualization platforms, the applications and the networks aren’t at a point where from a policy or intelligence perspective they can easily and reliably self-orchestrate.

Don’t get me wrong, autonomics and business process/governance feedback loops are most definitely coming — and are absolutely required for Cloud — but they’re not here and not used much today.  This is the hard stuff we’ve skipped over because it’s really freaking hard.  Don’t believe me?  See how long folks like HP have been at their “Adaptive Enterprise” solutions.  That’s why unified fabrics make so much sense; you can get your arms around automating much, much more with a consistent set of enforceable policies and SLAs.

So the next time someone brings up this epidemic of runaway VM’s, ask them to kindly provide you with empirical data demonstrating such as just because it *might* happen, doesn’t mean it *does* happen.

So much of the purported risks associated with virtualization and Cloud are things based on what might happen. There’s a huge difference between possibility and probability. One of them is used for prudent analysis and risk assessment, the other for selling you something. I’ll let you figure out which is which.

The management, visibility and security tools and capabilities are arriving on our doorsteps. When and if this sort of problem actually becomes a problem, it’s quite likely we’ll have a good set of solutions to deal with it.

Until then, challenge these assertions and fears, and ask for proof not pandering to panic.

 

 

 

 

You'll ask "How?" Everytime...

Enter the dawn of the Cloudtastrophe…

I read a story today penned by Maureen O’Gara titled “Carbonite Loses Cloud-Based Data, Sues Storage Vendor.”

I thought this was going to be another story regarding a data breach (loss) of customer data by a Cloud Computing service vendor.

What I found, however, was another hyperbolic illustration of how the messaging of the Cloud by vendors has set expectations for service and reliability that are out of alignment with reality when you take a lack of: sound enterprise architecture, proper contingency planning, solid engineering and common sense and add the economic lubricant of the Cloud.

Stir in a little journalistic sensationalism, and you’ve got CloudWow!

Carbonite, the online backup vendor, says it lost data belonging to over 7,500 customers in a number of separate incidents in a suit filed in Massachusetts charging Promise Technology Inc with supplying it with $3 million worth of defective storage, according to a story in Saturday’s Boston Globe.

The catastrophe is the latest in a series of cloud failures.

…

The widgetry was supposed to detect disk failures and transfer the data to a working drive. It allegedly didn’t.

The story says Promise couldn’t fix the errors and “Carbonite’s senior engineers, senior management and senior operations personnel…spent enormous amounts of time dealing with the problems.”

Carbonite claims the data losses caused “serious damage” to its business and reputation for reliability. It’s demanding unspecified damages. Promise told the Globe there was “no merit to the allegations.”

…

Carbonite, which sells largely to consumers and small businesses and competes with EMC’s Mozy, tells its customers: “never worry about losing your files again.”

The abstraction of infrastructure and democratization of applications and data that Cloud Computing services can bring does not mean that all services are created equal.  It does not make our services or information more secure (or less for that matter.)  Just because a vendor brands themselves as a “Cloud” provider does not mean that “their” infrastructure is any more implicitly reliable, stable or resilient than traditional infrastructure or that proper enterprise architecture as it relates to people, process and technology is in place.  How the infrastructure is built and maintained is just as important as ever.

If you take away the notion of Carbonite being a “Cloud” vendor, would this story read any differently?

We’ve seen a few failures recently of Cloud-based services, most of them sensationally lumped into the Cloud pile: Google, Microsoft, and even Amazon; most of the stories about them relate the impending doom of the Cloud…

Want another example of how Cloud branding, the Web2.0 experience and blind faith makes for another FUDtastic “catastrophe in the cloud?”  How about the well-known service Ma.gnolia?

There was a meltdown at bookmark sharing website Ma.gnolia Friday morning. The service lost both its primary store of user data, as well as its backup. The site has been taken offline while the team tries to reconstruct its databases, though some users may never see their stored bookmarks again.

The failure appears to be catastrophic. The company can’t say to what extent it will be able to restore any of its users’ data. It also says the data failure was so extensive, repairing the loss will take “days, not hours.”

So we find that a one man shop was offering a service that people liked and it died a horrible death.  Because it was branded as a Cloud offering, it “seemed” bigger than it was.  This is where perception definitely was not reality and now we’re left with a fluffy bad taste in our mouths.

Again, what this illustrates is that just because a service is “Cloud-based” does not imply it’s any more reliable or resilient as one that is not. It’s just as important that as enterprises look to move to the Cloud that they perform as much due diligence on their providers as makes sense. We’ll see a weeding out of the ankle-biters in Cloud Computing.

Nobody ever gets fired for buying IBM…

What we’ll also see is that even though we’re not supposed to care what our Cloud providers’ infrastructure is powered by and how, we absolutely will in the long term and the vendors know it.

This is where people start to freak about how standards and consolidation will kill innovation in the space but it’s also where the realities of running a business come crashing down on early adopters.

Large enterprises will move to providers who can demonstrate that their services are solid by way of co-branding with the reputation of the providers of infrastructure coupled with the compliance to “standards.”

The big players like IBM see this as an opportunity and as early as last year introduced a Cloud certification program:

IBM to Validate Resiliency of Cloud Computing Infrastructures

Will Consult With Businesses of All Sizes to Ensure Resiliency, Availability, Security; Drive Adoption of New Technology

ARMONK, NY – 24 Nov 2008: In a move that could spur the rise of the nascent computing model known as “cloud,” IBM (NYSE: IBM) today said it would introduce a program to validate the resiliency of any company delivering applications or services to clients in the cloud environment. As a result, customers can quickly and easily identify trustworthy providers that have passed a rigorous evaluation, enabling them to more quickly and confidently reap the business benefits of cloud services.

Cloud computing is a model for network-delivered services, in which the user sees only the service and does not view the implementation or infrastructure required for its delivery. The success to date of cloud services like storage, data protection and enterprise applications, has created a large influx of new providers. However, unpredictable performance and some high-profile downtime and recovery events with newer cloud services have created a challenge for customers evaluating the move to cloud.

IBM’s new “Resilient Cloud Validation” program will allow businesses who collaborate with IBM on a rigorous, consistent and proven program of benchmarking and design validation to use the IBM logo: “Resilient Cloud” when marketing their services.

Remember the “Cisco Powered Network” program?  How about a “Cisco Powered Cloud?”  See how GoGrid advertises their load balancers are f5?

In the long term, like the CapitalOne credit card commercials challenging the company providing your credit card services by asking “What’s in your wallet?” you can expect to start asking the same thing about your Cloud providers’ offerings, also.

/Hoff