Rational Survivability

As I continue to think about the opportunities that Software Defined Networking (SDN) and Network Function Virtualization (NFV) bring into focus, the capability to deliver security as a service layer is indeed exciting.

I wrote about how SDN and OpenFlow (as a functional example) and the security use cases provided by each will be a differentiating capability back in 2011: The Killer App For OpenFlow and SDN? Security, OpenFlow & SDN – Looking forward to SDNS: Software Defined Network Security, and Back To The Future: Network Segmentation & More Moaning About Zoning.

Recent activity in the space has done nothing but reinforce this opinion.  My day job isn’t exactly lacking in excitement, either

As many networking vendors begin to bring their SDN solutions to market — whether in the form of networking equipment or controllers designed to interact with them — one of the missing strategic components is security.  This isn’t a new phenomenon, unfortunately, and as such, predictably there are also now startups entering this space and/or retooling from the virtualization space and stealthily advertising themselves as “SDN Security” companies

Like we’ve seen many times before, security is often described (confused?) as a “simple” or “atomic” service and so SDN networking solutions are designed with the thought that security will simply be “bolted on” after the fact and deployed not unlike a network service such as “load balancing.”  The old “we’ll just fire up some VMs and TAMO (Then a Miracle Occurs) we’ve got security!” scenario.  Or worse yet, we’ll develop some proprietary protocol or insertion architecture that will magically get traffic to and from physical security controls (witness the “U-TURN” or “horseshoe” L2/L3 solutions of yesteryear.)

The challenge is that much of Security today is still very topologically sensitive and depends upon classical networking constructs to be either physically or logically plumbed between the “outside” and the asset under protection, or it’s very platform dependent and lacks the ability to truly define a policy that travels with the workload regardless of the virtualization, underlay OR overlay solutions.

Depending upon the type of control, security is often operationalized across multiple layers using wildly different constructs, APIs and context in terms of policy and disposition depending upon it’s desired effect.

Virtualization has certainly evolved our thinking about how we should think differently about security mostly due to the dynamism and mobility that virtualization has introduced, but it’s still incredibly nascent in terms of exposed security capabilities in the platforms themselves.  It’s been almost 5 years since I started raging about how we need(ed) platform providers to give us capabilities that function across stacks so we’d have a fighting chance.  To date, not only do we have perhaps ONE vendor doing some of this, but we’ve seen the emergence of others who are maniacally focused on providing as little of it as possible.

If you think about what virtualization offers us today from a security perspective, we have the following general solution options:

1. Hypervisor-based security solutions which may apply policy as a function of the virtual-NIC card of the workloads it protects.
2. Extensions of virtual-networking (i.e. switching) solutions that enable traffic steering and some policy enforcement that often depend upon…
3. Virtual Appliance-based security solutions that require manual or automated provisioning, orchestration and policy application in user space that may or may not utilize APIs exposed by the virtual networking layer or hypervisor

There are tradeoffs across each of these solutions; scale, performance, manageability, statefulness, platform dependencies, etc.  There simply aren’t many platforms that natively offer security capabilities as a function of service delivery that allows arbitrary service definition with consistent and uniform ways of describing the outcome of the policies at these various layers.  I covered this back in 2008 (it’s a shame nothing has really changed) in my Four Horsemen Of the Virtual Security Apocalypse presentation.

As I’ve complained for years, we still have 20 different ways of defining how to instantiate a five-tupule ACL as a basic firewall function.

Out of the Darkness…

The promise of SDN truly realized — the ability to separate the control, forwarding, management and services planes — and deploy security as a function of available service components across overlays and underlays, means we will be able to take advantage of any of these models so long as we have a way to programmatically interface with the various strata regardless of whether we provision at the physical, virtual or overlay virtual layer.

It’s truly exciting.  We’re seeing some real effort to enable true security service delivery.

When I think about how to categorize the intersection of “SDN” and “Security,” I think about it the same way I have with virtualization and Cloud:

• Securing SDN (Securing the SDN components)
• SDN Security Services (How do I take security and use SDN to deliver security as a service)
• Security via SDN (What NEW security capabilities can be derived from SDN)

There are numerous opportunities with each of these categories to really make a difference to security in the coming years.

The notion that many of our network and security capabilities are becoming programmatic means we *really* need to focus on securing SDN solutions, especially given the potential for abuse given the separation of the various channels. (See: Software Defined Networking (In)Security: All Your Control Plane Are Belong To Us…)

Delivering security as a service via SDN holds enormous promise for reasons I’ve already articulated and gives us an amazing foundation upon which to start building solutions we can’t imagine today given the lack of dynamism in our security architecture and design patterns.

Finally, the first two elements give rise to allow us to do things we can’t even imagine with today’s traditional physical and even virtual solutions.

I’ll be starting to highlight really interesting solutions I find (and am able to talk about) over the next few months.

Security enabled by SDN is going to be huge.

More soon.

/Hoff

Related articles

• The Tyranny Of Taming (Network) Traffic: Steering, Service Insertion and Chaining…

Many people who may only casually read my blog or peer at the timeline of my tweets may come away with the opinion that I suffer from confirmation bias when I speak about security and Cloud.

That is, many conclude that I am pro Private Cloud and against Public Cloud.

I find this deliciously ironic and wildly inaccurate. However, I must also take responsibility for this, as anytime one threads the needle and attempts to present a view from both sides with regard to incendiary topics without planting a polarizing stake in the ground, it gets confusing.

Let me clear some things up.

Digging deeper into what I believe, one would actually find that my blog, tweets, presentations, talks and keynotes highlight deficiencies in current security practices and solutions on the part of providers, practitioners and users in both Public AND Private Cloud, and in my own estimation, deliver an operationally-centric perspective that is reasonably critical and yet sensitive to emergent paths as well as the well-trodden path behind us.

I’m not a developer.  I dabble in little bits of code (interpreted and compiled) for humor and to try and remain relevant.  Nor am I an application security expert for the same reason.  However, I spend a lot of time around developers of all sorts, those that write code for machines whose end goal isn’t to deliver applications directly, but rather help deliver them securely.  Which may seem odd as you read on…

The name of this blog, Rational Survivability, highlights my belief that the last two decades of security architecture and practices — while useful in foundation — requires a rather aggressive tune-up of priorities.

Our trust models, architecture, and operational silos have not kept pace with the velocity of the environments they were initially designed to support and unfortunately as defenders, we’ve been outpaced by both developers and attackers.

Since we’ve come to the conclusion that there’s no such thing as perfect security, “survivability” is a better goal.  Survivability leverages “security” and is ultimately a subset of resilience but is defined as the “…capability of a system to fulfill its mission, in a timely manner, in the presence of attacks, failures, or accidents.”  You might be interested in this little ditty from back in 2007 on the topic.

Sharp readers will immediately recognize the parallels between this definition of “survivability,” how security applies within context, and how phrases like “design for failure” align.  In fact, this is one of the calling cards of a company that has become synonymous with (IaaS) Public Cloud: Amazon Web Services (AWS.)  I’ll use them as an example going forward.

So here’s a line in the sand that I think will be polarizing enough:

I really hope that AWS continues to gain traction with the Enterprise.  I hope that AWS continues to disrupt the network and security ecosystem.  I hope that AWS continues to pressure the status quo and I hope that they do it quickly.

Why?

Almost a decade ago, the  Open Group’s Jericho Forum published their Commandments.  Designed to promote a change in thinking and operational constructs with respect to security, what they presciently released upon the world describes a point at which one might imagine taking one’s most important assets and connecting them directly to the Internet and the shifts required to understand what that would mean to “security”:

1. The scope and level of protection should be specific and appropriate to the asset at risk.
2. Security mechanisms must be pervasive, simple, scalable, and easy to manage.
3. Assume context at your peril.
4. Devices and applications must communicate using open, secure protocols.
5. All devices must be capable of maintaining their security policy on an un-trusted network.
6. All people, processes, and technology must have declared and transparent levels of trust for any transaction to take place.
7. Mutual trust assurance levels must be determinable.
8. Authentication, authorization, and accountability must interoperate/exchange outside of your locus/area of control
9. Access to data should be controlled by security attributes of the data itself
10. Data privacy (and security of any asset of sufficiently high value) requires a segregation of duties/privileges
11. By default, data must be appropriately secured when stored, in transit, and in use.

These seem harmless enough today, but were quite unsettling when paired with the notion of “de-perimieterization” which was often misconstrued to mean the immediate disposal of firewalls.  Many security professionals appreciated the commandments for what they expressed, but the the design patterns, availability of solutions and belief systems of traditionalists constrained traction.

Interestingly enough, now that the technology, platforms, and utility services have evolved to enable these sorts of capabilities, and in fact have stressed our approaches to date, these exact tenets are what Public Cloud forces us to come to terms with.

If one were to look at what public cloud services like AWS mean when aligned to traditional “enterprise” security architecture, operations and solutions, and map that against the Jericho Forum’s Commandments, it enables such a perfect rethink.

Instead of being focused on implementing “security” to protect applications and information based at the network layer — which is more often than not blind to both, contextually and semantically — public cloud computing forces us to shift our security models back to protecting the things that matter most: the information and the conduits that traffic in them (applications.)

As networks become more abstracted, it means that existing security models do also.  This means that we must think about security programatticaly and embedded as a functional delivery requirement of the application.

“Security” in complex, distributed and networked systems is NOT a tidy simple atomic service.  It is, unfortunately, represented as such because we choose to use a single noun to represent an aggregate of many sub-services, shotgunned across many layers, each with its own context, metadata, protocols and consumption models.

As the use cases for public cloud obscure and abstract these layers — flattens them — we’re left with the core of that which we should focus:

Build secure, reliable, resilient, and survivable systems of applications, comprised of secure services, atop platforms that are themselves engineered to do the same in way in which the information which transits them inherits these qualities.

So if Public Cloud forces one to think this way, how does one relate this to practices of today?

Frankly, enterprise (network) security design patterns are a crutch.  The screened-subnet DMZ patterns with perimeters is outmoded. As Gunnar Peterson eloquently described, our best attempts at “security” over time are always some variation of firewalls and SSL.  This is the sux0r.  Importantly, this is not stated to blame anyone or suggest that a bad job is being done, but rather that a better one can be.

It’s not like we don’t know *what* the problems are, we just don’t invest in solving them as long term projects.  Instead, we deploy compensation that defers what is now becoming more inevitable: the compromise of applications that are poorly engineered and defended by systems that have no knowledge or context of the things they are defending.

We all know this, but yet looking at most private cloud platforms and implementations, we gravitate toward replicating these traditional design patterns logically after we’ve gone to so much trouble to articulate our way around them.  Public clouds make us approach what, where and how we apply “security” differently because we don’t have these crutches.

Either we learn to walk without them or simply not move forward.

Now, let me be clear.  I’m not suggesting that we don’t need security controls, but I do mean that we need a different and better application of them at a different level, protecting things that aren’t tied to physical topology or addressing schemes…or operating systems (inclusive of things like hypervisors, also.)

I think we’re getting closer.  Beyond infrastructure as a service, platform as a service gets us even closer.

Interestingly, at the same time we see the evolution of computing with Public Cloud, networking is also undergoing a renaissance, and as this occurs, security is coming along for the ride.  Because it has to.

As I was writing this blog (ironically in the parking lot of VMware awaiting the start of a meeting to discuss abstraction, networking and security,) James Staten (Forrester) tweeted something from @Werner Vogels keynote at AWS re:invent:

Werner: “There’s no excuse not to use fine grained security to make your apps secure from the start.” Echoing @kindervag Zero Trust

— Staten7 (@Staten7) November 29, 2012

I couldn’t have said it better myself

So while I may have been, and will continue to be, a thorn in the side of platform providers to improve the “survivability” capabilities to help us get from there to there, I reiterate the title of this scribbling: Amazon Web Services (AWS) Is the Best Thing To Happen To Security & I Desperately Want It To Succeed.

I trust that’s clear?

/Hoff

P.S. There’s so much more I could/should write, but I’m late for the meeting

Related articles

• Should/Can/Will Virtual Firewalls Replace Physical Firewalls?
• Quick Quip: Capability, Reliability and Liability…Security Licensing

Simulação da participação de um Firewall entre uma LAN e uma WAN Français : Schéma d’un pare-feu entre un LAN et un WAN (Photo credit: Wikipedia)

“Should/Can/Will Virtual Firewalls Replace Physical Firewalls?”

The answer is, as always, “Of course, but not really, unless maybe, you need them to…”

This discussion crops up from time-to-time, usually fueled by a series of factors which often lack the context to appropriately address it.

The reality is there exists the ever-useful answer of “it depends,” and frankly it’s a reasonable answer.

Back in 2008 when I created “The Four Horsemen of the Virtualization Security Apocalypse” presentation, I highlighted the very real things we needed to be aware of as we saw the rapid adoption of server virtualization…and the recommendations from virtualization providers as to the approach we should take in terms of securing the platforms and workloads atop them.  Not much has changed in almost five years.

However, each time I’m asked this question, I inevitably sound evasive when asking for more detail when the person doing the asking references “physical” firewalls and what it is they mean.  Normally the words “air-gap” are added to the mix.

The very interesting thing about how people answer this question is that in reality, the great many firewalls that are deployed today have the following features deployed in common:

1. Heavy use of network “LAG” (link aggregation group) interface bundling/VLAN trunking and tagging
2. Heavy network virtualization used, leveraging VLANs as security boundaries, trunked across said interfaces
3. Increased use of virtualized contexts and isolated resource “virtual systems” and separate policies
4. Heavy use of ASIC/FPGA and x86 architectures which make use of shared state tables, memory and physical hardware synced across fabrics and cluster members
5. Predominant use of “stateful inspection” at layers 2-4 with the addition of protocol decoders at L5-7 for more “application-centric” enforcement
6. Increasing use of “transparent proxies” at L2 but less (if any) full circuit or application proxies in the classic sense

So before I even START to address the use cases of the “virtual firewalls” that people reference as the comparison, nine times out of ten, that supposed “air gap” with dedicated physical firewalls that they reference usually doesn’t compute.

Most of the firewall implementations that people have meet most of the criteria mentioned in items 1-6 above.

Further, most firewall architectures today aren’t running full L7 proxies across dedicated physical interfaces like in the good old days (Raptor, etc.) for some valid reasons…(read the postscript for an interesting prediction.)

Failure domains and the threat modeling that illustrates cascading impact due to complexity, outright failure or compromised controls is usually what people are interested in when asking this question, but this gets almost completely obscured by the “physical vs. virtual” concern and we often never dig deeper.

There are some amazing things that can be done in virtual constructs that we can’t do in the physical and there are some pretty important things that physical firewalls can provide that virtual versions have trouble with.  It’s all a matter of balance, perspective, need, risk and reward…oh, and operational simplicity.

I think it’s important to understand what we’re comparing when asking that question before we conflate use cases, compare and mismatch expectations, and make summary generalizations (like I just did about that which we are contrasting.

I’ll actually paint these use cases in a follow-on post shortly.

/Hoff

POSTSCRIPT:

I foresee that we will see a return of the TRUE application-level proxy firewall — especially with application identification, cheap hardware, more security and networking virtualized in hardware.  I see this being deployed both on-premise and as part of a security as a service offering (they are already, today — see CloudFlare, for example.)

If you look at the need to terminate SSL/TLS and provide for not only L4-L7 sanity, protect applications/sessions at L5-7 (web and otherwise) AND the renewed dependence upon XML, SOAP, REST, JSON, etc., it will drive even more interesting discussions in this space.  Watch as the hybrid merge of the WAF+XML security services gateway returns to vogue… (see also Cisco EOLing ACE while I simultaneously receive an email from Intel informing me I can upgrade to their Intel Expressway Service Gateway…which I believe (?) was from the Cervega Sarvega acqusition?)

Related articles

• Incomplete Thought: Virtual/Cloud Security and The Potemkin Village Syndrome
• Software Defined Networking (In)Security: All Your Control Plane Are Belong To Us…
• SiliconAngle Cube: Hoff On Security – Live At VMworld 2012

My next series of talks are focused around the emerging technology, solutions and security architectures of so-called “Software Defined Networking (SDN)”

As this space heats up, I see a huge opportunity for new and interesting ways in which security can be delivered — the killer app? — but I also am concerned that, per usual, security is a potential after thought.

At an absolute minimum example, the separation of control and data planes (much as what we saw with compute-centric virtualization) means we now have additional (or at least bifurcated) attack surfaces and threat vectors.  And not unlike compute-centric virtualization, the C&C channels for network operation represents a juicy target.

There are many more interesting elements that deserve more attention paid to them — new protocols, new hardware/software models, new operational ramifications…and I’m going to do just that.

If you’re a vendor who cares to share what you’re doing to secure your SDN offerings — and I promise I’ll be fair and balanced as I always am — please feel free to reach out to me.  If you don’t and I choose to include your solution based on access to what data I have, you run the risk of being painted inaccurately <hint>

If you have any ideas, comments or suggestions on what you’d like to see featured or excluded, let me know.  This will be along the lines of what I did with the “Four Horsemen Of the Virtualization Security Apocalypse” back in 2008.

Check out a couple of previous ramblings related to SDN (and OpenFlow) with respect to security below.

/Hoff

Related articles

• Back To The Future: Network Segmentation & More Moaning About Zoning
• Incomplete Thought: Virtual/Cloud Security and The Potemkin Village Syndrome

A “Potemkin village” is a Russian expression derived from folklore from the 1700′s.  The story goes something like this: Grigory Potemkin, a military leader and  statesman, erected attractive but completely fake settlements constructed only of facades to impress Catherine the Great (empress of Russia) during a state visit in order to gain favor and otherwise hype the value of recently subjugated territories.

I’ll get to that (and probably irate comments from actual Russians who will chide me for my hatchet job on their culture…)

Innovation over the last decade in technology in general has brought fundamental shifts in the way in which we work, live, and play. In the last 4 years, the manner in which technology products and services that enabled by this “digital supply chain,” and the manner in which they are designed, built and brought to market have also pivoted.

Virtualization and Cloud computing — the technologies and operational models — have contributed greatly to this.

Interestingly enough, the faster technology evolves, the more lethargic, fragile and fractured security seems to be.

This can be explained in a few ways.

First, the trust models, architecture and operational models surrounding how we’ve “done” security simply are not designed to absorb this much disruption so quickly.  The fact that we’ve relied on physical segregation, static policies that combine locality and service definition, mobility and the (now) highly dynamic application deployment options means that we’re simply disconnected.

Secondly, fragmentation and specialization within security means that we have no cohesive, integrated or consistent approach in terms of how we define or instantiate “security,” and so customers are left to integrate disparate solutions at multiple layers (think physical and/or virtual firewalls, IDP, DLP, WAF, AppSec, etc.)  What services and “hooks” the operating systems, networks and provisioning/orchestration layers offers largely dictates what we can do using the skills and “best practices” we already have.

Lastly, the (un)natural market consolidation behavior wherein aspiring technology startups are acquired and absorbed into larger behemoth organizations means that innovation cycles in security quickly become victims of stunted periodicity, reduced focus on solving specific problems, cultural subduction and artificially constrained scope based on P&L models which are detached from reality, customers and out of step with trends that end up driving more disruption.

I’ve talked about this process as part of the “Security Hamster Sine Wave of Pain.”  It’s not a malicious or evil plan on behalf of vendors to conspire to not solve your problems, it’s an artifact of the way in which the market functions — and is allowed to function.

What this yields is that when new threat models, evolving vulnerabilities and advanced adversarial skill sets are paired with massively disruptive approaches and technology “conquests,” the security industry  basically erects facades of solutions, obscuring the fact that in many cases, there’s not only a lacking foundation for the house of cards we’ve built, but interestingly there’s not much more to it than that.

Again, this isn’t a plan masterminded by a consortium of industry “Dr. Evils.”  Actually, it’s quite simple: It’s inertial…if you keep buying it, they’ll keep making it.

We are suffering then from the security equivalent of the Potemkin Village syndrome; our efforts are largely built to impress people who are mesmerized by pretty facades but don’t take the time to recognize that there’s really nothing there.  Those building it, while complicit, find it quite hard to change.

Until the revolution comes.

To wit, we have hardworking members of the proletariat, toiling away behind the scenes struggling to add substance and drive change in the way in which we do what we do.

Adding to this is the good news that those two aforementioned “movements” — virtualization and cloud computing — are exposing the facades for what they are and we’re now busy shining the light on unstable foundations, knocking over walls and starting to build platforms that are fundamentally better suited to support security capabilities rather than simply “patching holes.”

Most virtualization and IaaS cloud platforms are still woefully lacking the native capabilities or interfaces to build security in, but that’s the beauty of platforms (as a service,) as you can encourage more “universally” the focus on the things that matter most: building resilient and survivable systems, deploying secure applications, and identifying and protecting information across its lifecycle.

Realistically this is a long view and it is going to take a few more cycles on the Hamster Wheel to drive true results.  It’s frankly less about technology and rather largely a generational concern with the current ruling party who governs operational security awaiting deposition, retirement or beheading.

I’m looking forward to more disruption, innovation and reconstruction.  Let’s fix the foundation and deal with hanging pictures later.  Redecorating security is for the birds…or dead Russian royalty.

/Hoff

Related articles

• Is What We Need…An OpSec K/T Boundary Extinction-Level Event?
• Back To The Future: Network Segmentation & More Moaning About Zoning
• Elemental: Leveraging Virtualization Technology For More Resilient & Survivable Systems

After my keynote at Gluecon (Shit My Cloud Evangelist Says…Just Not To My CSO,) I was asked by an attendee what things he could do within his organization to repair the damage and/or mistrust between developers and security organizations in enterprises.

Here’s what I suggested based on past experience:

1. Reach out and have a bunch of “brown bag lunches” wherein you host-swap each week; devs and security folks present to one another with relevant, interesting or new solutions in their respective areas
2. Pick a project that takes a yet-to-be-solved interesting business challenge that isn’t necessarily on the high priority project list and bring the dev and security teams together as if it were an actual engagement.

Option 1 starts the flow of information.  Option 2 treats the project as if it were high priority but allows security and dev to work together to talk about platform choices, management, security, etc. and because it’s not mission critical, mistakes can be made and learned from…together.

For example, pick something like building a new app service that uses node.js and MongoDB and figure out how to build, deploy and secure it…as if you were going to deploy to public cloud from day one (and maybe you will.)

You’ll be amazed to see the trust it builds, especially in light of developers enrolling security in their problem and letting them participate from the start versus being the speed bump later.

10 minutes later it’ll be a DevOps love-fest.

/Hoff

It’s not really *that* incomplete of a thought, but I figure I’d get it down on vPaper anyway…be forewarned, it’s massively over-simplified.

Over the last five years or so, I’ve spent my time working with enterprises who are building and deploying large scale (relative to an Enterprise’s requirements, that is) virtualized data centers and private cloud environments.

For the purpose of this discussion, I am referring to VMware-based deployments given the audience and solutions I will reference.

To this day, I’m often shocked with regard to how many of these organizations that seek to provide contextualized security for intra- and inter-VM traffic seem to position an either-or decision with respect to the use of physical or virtual security solutions.

For the sake of example, I’ll reference the architectural designs which were taken verbatim from my 2008 presentation “The Four Horsemen of the Virtualization Security Apocalypse.“

If you’ve seen/read the FHOTVA, you will recollect that there are many tradeoffs involved when considering the use of virtual security appliances and their integration with physical solutions.  Notably, an all-virtual or all-physical approach will constrain you in one form or another from the perspective of efficacy, agility, and the impact architecturally, operationally, or economically.

The topic that has a bunch of hair on it is where I see many enterprises trending: obviating virtual solutions and using physical appliances only:

…the bit that’s missing in the picture is the external physical firewall connected to that physical switch.  People are still, in this day and age, ONLY relying on horseshoeing all traffic between VMs (in the same or different VLANs) out of the physical cluster machine and to an external firewall.

Now, there are many physical firewalls that allow for virtualized contexts, zoning, etc., but that’s really dependent upon dumping trunked VLAN ports from the firewall/switches into the server and then “extending” virtual network contexts, policies, etc. upstream in an attempt to flatten the physical/virtual networks in order to force traffic through a physical firewall hop — sometimes at layer 2, sometimes at layer 3.

It’s important to realize that physical firewalls DO offer benefits over the virtual appliances in terms of functionality, performance, and some capabilities that depend on hardware acceleration, etc. but from an overall architectural positioning, they’re not sufficient, especially given the visibility and access to virtual networks that the physical firewalls often do not have if segregated.

Here’s a hint, physical-only firewall solutions alone will never scale with the agility required to service the virtualized workloads that they are designed to protect.  Further, a physical-only solution won’t satisfy the needs to dynamically provision and orchestrate security as close to the workload as possible, when the workloads move the policies will generally break, and it will most certainly add latency and ultimately hamper network designs (both physical and virtual.)

Virtual security solutions — especially those which integrate with the virtualization/cloud stack (in VMware’s case, vCenter & vCloud Director) — offer the ability to do the following:

…which is to say that there exists the capability to utilize  virtual solutions for “east-west” traffic and physical solutions for “north-south” traffic, regardless of whether these VMs are in the same or different VLAN boundaries or even across distributed virtual switches which exist across hypervisors on different physical cluster members.

For east-west traffic (and even north-south models depending upon network architecture) there’s no requirement to horseshoe traffic physically. 

It’s probably important to mention that while the next slide is out-of-date from the perspective of the advancement of VMsafe APIs, there’s not only the ability to inject a slow-path (user mode) virtual appliance between vSwitches, but also utilize a set of APIs to instantiate security policies at the hypervisor layer via a fast path kernel module/filter set…this means greater performance and the ability to scale better across physical clusters and distributed virtual switching:

Interestingly, there also exists the capability to actually integrate policies and zoning from physical firewalls and have them “flow through” to the virtual appliances to provide “micro-perimeterization” within the virtual environment, preserving policy and topology.

There are at least three choices for hypervisor management-integrated solutions on the market for these solutions today:

• VMware vShield App,
• Cisco VSG+Nexus 1000v and
• Juniper vGW

Note that the solutions above can be thought of as “layer 2″ solutions — it’s a poor way of describing them, but think “inter-VM” introspection for workloads in VLAN buckets.  All three vendors above also have, or are bringing to market, complementary “layer 3″ solutions that function as virtual “edge” devices and act as a multi-function “next-hop” gateway between groups of VMs/applications (nee vDC.)  For the sake of brevity, I’m omitting those here (they are incredibly important, however.)

They (layer 2 solutions) are all reasonably mature and offer various performance, efficacy and feature set capabilities. There are also different methods for plumbing the solutions and steering traffic to them…and these have huge performance and scale implications.

It’s important to recognize that the lack of thinking about virtual solutions often seem to be based largely on ignorance of need and availability of solutions.

However, other reasons surface such as cost, operational concerns and compliance issues with security teams or assessors/auditors who don’t understand virtualized environments well enough.

From an engineering and architectural perspective, however, obviating them from design consideration is a disappointing concern.

Enterprises should consider a hybrid of the two models; virtual where you can, physical where you must.

If you’ve considered virtual solutions but chose not to deploy them, can you comment on why and share your thinking with us (even if it’s for the reasons above?)

/Hoff

Related articles

• Incomplete Thought: Will the Public Cloud Create a Generation Of Network Stupid?(rationalsurvivability.com)

  ---

• Virtual Networking is more than VMs and VLAN duct tape(ioshints.info)

  ---

• Overlays: Wasting Away Again In Abstractionville… (rationalsurvivability.com)
• Virtualized security routers for cloud security (net-security.org)

…well, it is if you listen to many of the folks who spend their time trawling about security conferences, writing blogs (like this one) or on podcasts, it is.  I don’t share that opinion, however.

Lately there’s been a noisy upswing in the security echo chamber of people who suggest that  given the visibility, scope, oft-quoted financial impact and reputational damage of recent breaches, that “security is losing.”

{…losing it’s mind, perhaps…}

What’s troubling about all this hen pecking is that with each complaint about the sorry state of the security “industry,” there’s rarely ever offered a useful solution that is appropriately adoptable within a reasonable timeframe, that satisfies a business condition, and result in an outcome that moves the needle to the “winning” side of the meter.

I was asked by Martin Mckeay (@mckeay) in a debate on Twitter, in which I framed the points above, if “…[I] don’t see all the recent breaches as evidence that we’re losing…that so many companies compromised as proof [that we're losing.]”

My answer was a succinct “no.”

What these breaches indicate is the constant innovation we see from attackers, the fact that companies are disclosing said breaches and the relative high-value targets admitting such.  We’re also seeing the better organization of advanced adversaries whose tactics and goals aren’t always aligned with the profiles of “hackers” we see in the movies.

That means our solutions aren’t aligned to the problems we think we have nor the motivation and tactics of the attackers that these solutions are designed to prevent.

The dynamic tension between “us” and “them” is always cyclical in terms of the perception of who is “winning” versus “losing.”  Always has been, always will be.  Anyone who doesn’t recognize patterns in this industry is either:

1. New
2. Ignorant
3. Selling you something
4. …or all of the above

Most importantly, it’s really, really important to recognize that the security “industry” is in business to accomplish one goal:

Make money.

It’s not a charity.  It’s not a cause.  It’s not a club.  It’s a business.

The security industry — established behemoths and startups alike — are in the business of being in business.  They may be staffed by passionate, idealistic and caring individuals, but those individuals enjoy paying their mortgages.

These companies also provide solutions that aren’t always ready from the perspective of market, economics, culture, adoptability, scope/impact of problem, etc.  This is why I show the Security Hamster Sine Wave of Pain and why security, much like bell bottoms, comes back into vogue in cycles…generally when those items above converge.

Now, if you overlay what I just said with the velocity and variety of innovation without constraint that attackers play with and you have a clearer picture of why we are where we are.

Of course, no rant like this would be complete without the anecdotal handwaving bemoaning flawed trust models and technology, insecure applications and those pesky users…sigh.

The reality is that if we (as operators) are constrained to passive defense and are expected to score progress in terms of moving the defensive line forward versus holding ground, albeit with collateral damage, then yes…we’re losing.

If, rather, we assess our ability to influence outcomes such that the business can function at an acceptable level of risk, where “winning” and “losing” aren’t measured in emotional baggage or absolutes, then perhaps more often than not, we’d be winning instead of whining.

It’s all a matter of perspective, really.

I think staring at things other than one’s bellybutton can deliver some.

Try it.  It won’t hurt.  Promise.

/Hoff

Related articles

• Bringing Sexy back (to Security): Mike’s RSAC 2012 Wrap-up (securosis.com)
• Hacking breach made us stronger says RSA (go.theregister.com)
• Building/Bolting Security In/On – A Pox On the Audit Paradox! (rationalsurvivability.com)
• PSA: Paula Deen, Sausage Pancake Egg Sandwiches & Security… (rationalsurvivability.com)

I spent yesterday at the PacketPushers/TechFieldDay OpenFlow Symposium. The event provided a good overview of what OpenFlow [currently] means, how it fits into the overall context of software-defined networking (SDN) and where it might go from here.

I’d suggest reading Ethan Banks’ (@ecbanks) overview here.

Many of us left the event, however, still wondering about what the “killer app” for OpenFlow might be.

Chatting with Ivan Pepelnjak (@ioshints) and Derrick Winkworth (@CloudToad,) I reiterated that selfishly, I’m still thrilled about the potential that OpenFlow and SDN can bring to security.  This was a topic only briefly skirted during the symposium as the ACL-like capabilities of OpenFlow were discussed, but there’s so much more here.

I wrote about this back in May (OpenFlow & SDN – Looking forward to SDNS: Software Defined Network Security):

… “security” needs to be as programmatic/programmable, agile, scaleable and flexible as the workloads (and stacks) it is designed to protect. “Security” in this context extends well beyond the network, but the network provides such a convenient way of defining templated containers against which we can construct and enforce policies across a wide variety of deployment and delivery models.

So as I watch OpenFlow (and Software Defined Networking) mature, I’m really, really excited to recognize the potential for a slew of innovative ways we can leverage and extend this approach to networking [monitoring and enforcement] in order to achieve greater visibility, scale, agility, performance, efficacy and reduced costs associated with security.  The more programmatic and instrumented the network becomes, the more capable our security options will become also.

I had to chuckle at a serendipitous tweet from a former Cisco co-worker (Stefan Avgoustakis, @savgoust) because it’s really quite apropos for this topic:

…I think he’s oddly right!

Frankly, if you look at what OpenFlow and SDN (and network programmability in general) gives an operator — the visibility and effective graph of connectivity as well as the multiple-tupule flow action capabilities, there are numerous opportunities to leverage the separation of control/data plane across both virtual and physical networks to provide better security capabilities in response to threats and at a pace/scale/radius commensurate with said threat.

To be able to leverage telemetry and flow tables in the controllers “centrally” and then “dispatch” the necessary security response on an as-needed basis to the network location ONLY that needs it, really does start to sound a lot like the old “immune system” analogy that SDN (self defending networks) promised.

The ability to distribute security capabilities more intelligently as a service layer which can be effected when needed — without the heavy shotgunned footprint of physical in-line devices or the sprawl of virtualized appliances — is truly attractive.  Automation for detection and ultimately prevention is FTW.

Bundling the capabilities delivered via programmatic interfaces and coupling that with ways of integrating the “network” and “applications” (of which security is one) produces some really neat opportunities.

Now, this isn’t just a classical “data center core” opportunity, either. How about the WAN/Edge?  Campus, branch…? Anywhere you have the need to deliver security as a service.

For non-security examples, check out Dave Ward’s (my Juniper colleague) presentation “Programmable Networks are SFW” where he details interesting use cases such as “service engineered paths,” “service appliance pooling,” “service specific topology,” “content request routing,” and “bandwidth calendaring” for example.

…think of the security ramifications and opportunities linked to those capabilities!

I’ve mocked up a couple of very interesting security prototypes using OpenFlow and some open source security components; from IDP to Anti-malware and the potential is exciting because OpenFlow — in conjunction with other protocols and solutions in the security ecosystem — could provide some of the missing glue necessary to deliver a constant,  but abstracted security command/control (nee API-like capability) across heterogenous infrastructure.

NOTE: I’m not suggesting that OpenFlow itself provide these security capabilities, but rather enable security solutions to take advantage of the control/data plane separation to provide for more agile and effective security.

If the potential exists for OpenFlow to effectively allow choice of packet forwarding engines and network virtualization across the spectrum of supporting vendors’ switches, it occurs to me that we could utilize it for firewalls, intrusion detection/prevention engines, WAFs, NAC, etc.

Thoughts?

Related articles

• What’s OpenFlow’s killer app? (infoworld.com)
• OpenFlow and SDN: Networking’s future? (networkworld.com)
• Can OpenFlow Unlock More of Virtualization’s Potential? (informationweek.com)
• OpenFlow Tutorial: Next-Gen Networking Has Much To Prove (informationweek.com)
• Network Virtualization is like a big virtual chassis (bradhedlund.com)
• A Contentious Question: The Value Proposition & Target Market Of Virtual Networking Solutions? (rationalsurvivability.com)
• VMware’s vShield – Why It’s Such A Pain In the Security Ecosystem’s *aaS… (rationalsurvivability.com)
• (Physical, Virtualized and Cloud) Security Automation – An API Example (rationalsurvivability.com)
• SecurityAutomata: A Reference For Security Automation… (rationalsurvivability.com)

I’ve become…comfortably numb…

Whilst attending VMworld 2011 last week, I attended a number of VMware presentations, hands-on labs and engaged in quite a few discussions related to VMware’s vShield and overall security strategy.

I spent a ton of time discussing vShield with customers — some who love it, some who don’t — and thought long and hard about writing this blog.  I also spent some time on SiliconAngle’s The Cube discussing such, here.

I have dedicated quite a lot of time discussing the benefits of VMware’s security initiatives, so it’s important that you understand that I’m not trying to be overtly negative, nor am I simply pointing fingers as an uneducated, uninterested or uninvolved security blogger intent on poking the bear.  I live this stuff…every day, and like many, it’s starting to become messy. (Ed: I’ve highlighted this because many seem to have missed this point. See here for example.)

It’s fair to say that I have enjoyed “up-to-the-neck” status with VMware’s various security adventures since the first marketing inception almost 4 years ago with the introduction of the VMsafe APIs.  I’ve implemented products and helped deliver some of the ecosystem’s security offerings.  My previous job at Cisco was to provide the engineering interface between the two companies, specifically around the existing and next generation security offerings, and I now enjoy a role at Juniper which also includes this featured partnership.

I’m also personal friends with many of the folks at VMware on the product and engineering teams, so I like to think I have some perspective.  Maybe it’s skewed, but I don’t think so.

There are lots of things I cannot and will not say out of respect for obvious reasons pertaining to privileged communications and NDAs, but there are general issues that need to be aired.

Geez, enough with the CYA…get on with it then…

As I stated on The Cube interview, I totally understand VMware’s need to stand-alone and provide security capacities atop their platform; they simply cannot expect to move forward and be successful if they are to depend solely on synchronizing the roadmaps of dozens of security companies with theirs.

However, the continued fumbles and mis-management of the security ecosystem and their partnerships as well as the continued competitive nature of their evolving security suite makes this difficult.  Listening to VMware espouse that they are in the business of “security ecosystem enablement” when there are so few actually successful ecosystem partners involved beyond antimalware is disingenuous…or at best, a hopeful prediction of some future state.

Here’s something I wrote on the topic back in 2009: The Cart Before the Virtual Horse: VMware’s vShield/Zones vs. VMsafe APIs that I think illustrates some of the issues related to the perceived “strategy by bumping around in the dark.”

A big point of confusion is that vShield simultaneously describes both an ecosystem program and a set of products that is actually more than just anti-malware capabilities which is where the bulk of integration today is placed.

Analysts and journalists continue to miss the fact that “vShield” is actually made up of 4 components (not counting the VMsafe APIs):

• vShield Edge
• vShield App
• vShield Endpoint
• vShield Manager

What most people often mean when they refer to “vShield” are the last two components, completely missing the point that the first two products — which are now monetized and marketed/sold as core products for vSphere and vCloud Director — basically make it very difficult for the ecosystem to partner effectively since it’s becoming more difficult to exchange vShield solutions for someone else’s.

An important reason for this is that VMware’s sales force is incentivized (and compensated) on selling VMware security products, not the ecosystem’s — unless of course it is in the way of a big deal that only a partnership can overcome.  This is the interesting juxtaposition of VMware’s “good enough” versus incumbent security vendors “best-of-breed” product positioning.

VMware is not a security or networking company and ignoring the fact that big companies with decades of security and networking products are not simply going to fade away is silly.  This is true of networking as it is security (see software-defined networking as an example.)

Technically, vShield Edge is becoming more and more a critical piece of the overall architecture for VMware’s products — it acts as the perimeter demarcation and multi-tenant boundary in their Cloud offerings and continues to become the technology integration point for acquisitions as well as networking elements such as VXLAN.

As a third party tries to “integrate” a product which is functionally competitive with vShield Edge, the problems start to become much more visible and the partnerships more and more clumsy, especially in the eyes of the most important party privy to this scenario: the customer.

Jon Oltsik wrote a story recently in which he described the state of VMware’s security efforts: “vShield, Cloud Computing, and the Security Industry”

So why aren’t more security vendors jumping on the bandwagon? Many of them look at vShield as a potentially competitive security product, not just a set of APIs.

In a recent Network World interview, Allwyn Sequeira, VMware’s chief technology officer of security and vice president of security and network solutions, admitted that the vShield program in many respects “does represent a challenge to the status quo” … (and) vShield does provide its own security services (firewall, application layer controls, etc.)

Why aren’t more vendors on-board? It’s because this positioning of VMware’s own security products which enjoy privileged and unobstructed access to the platform that ISV’s in the ecosystem do not have.  You can’t move beyond the status quo when there’s not a clear plan for doing so and the past and present are littered with the wreckage of prior attempts.

VMware has its own agenda: tightly integrate security services into vSphere and vCloud to continue to advance these platforms. Nevertheless, VMware’s role in virtualization/cloud and its massive market share can’t be ignored. So here’s a compromise I propose:

1. Security vendors should become active VMware/vShield partners, integrate their security solutions, and work with VMware to continue to bolster cloud security. Since there is plenty of non-VMware business out there, the best heterogeneous platforms will likely win.
2. VMware must make clear distinctions among APIs, platform planning, and its own security products. For example, if a large VMware shop wants to implement vShield for virtual security services but has already decided on Symantec (Vontu) or McAfee DLP, it should have the option for interoperability with no penalties (i.e., loss of functionality, pricing/support premiums, etc.).

Item #1 Sounds easy enough, right? Except it’s not.  If the way in which the architecture is designed effectively locks out the ecosystem from equal access to the platform except perhaps for a privileged few, “integrating” security solutions in a manner that makes those solutions competitive and not platform-specific is a tall order.  It also limits innovation in the marketplace.

Look how few startups still exist who orbit VMware as a platform.  You can count them on less fingers that exist on a single hand.  As an interesting side-note, Catbird — a company who used to produce their own security enforcement capabilities along with their strong management and compliance suite — has OEM’d VMware’s vShield App product instead of bothering to compete with it.

Now, item #2 above is right on the money.  That’s exactly what should happen; the customer should match his/her requirements against the available options, balance the performance, efficacy, functionality and costs and ultimately be free to choose.  However, as they say in Maine…”you can’t get there from here…” at least not unless item #1 gets solved.

In a complimentary piece to Jon’s, Ellen Messmer writes in “VMware strives to expand security partner ecosystem“:

Along with technical issues, there are political implications to the vShield approach for security vendors with a large installed base of customers as the vShield program asks for considerable investment in time and money to develop what are new types of security products under VMware’s oversight, plus sharing of threat-detection information with vShield Manager in a middleware approach.

…and…

The pressure to make vShield and its APIs a success is on VMware in some respects because VMware’s earlier security API , the VMsafe APIs, weren’t that successful. Sequiera candidly acknowledges that, saying, “we got the APIs wrong the first time,” adding that “the major security vendors have found it hard to integrate with VMsafe.”

Once bitten, twice shy…

So where’s the confidence that guarantees it will be easier this time? Basically, besides anti-malware functionality provided by integration with vShield endpoint, there’s not really a well-defined ecosystem-wide option for integration beyond that with VMware now.  Even VMware’s own roadmaps for integration are confusing.  In the case of vCloud Director, while vShield Edge is available as a bundled (and critical) component, vShield App is not!

Also, forcing integration with security products now to directly integrate with vShield Manager makes for even more challenges.

There are a handful of security products besides anti-malware in the market based on the VMsafe APIs, which are expected to be phased out eventually. VMware is reluctant to pin down an exact date, though some vendors anticipate end of next year.

That’s rather disturbing news for those companies who have invested in the roadmap and certification that VMware has put forth, isn’t it?  I can name at least one such company for whom this is a concern.

Because VMware has so far reserved the role of software-based firewalls and data-loss prevention under vShield to its own products, that has also contributed to unease among security vendors. But Sequiera says VMware is in discussions with Cisco on a firewall role in vShield.   And there could be many other changes that could perk vendor interest. VMware insists its vShield APIs are open but in the early days of vShield has taken the approach of working very closely with a few selected vendors.

Firstly, that’s not entirely accurate regarding firewall options. Cisco and Juniper both have VMware-specific “firewalls” on the market for some time; albeit they use different delivery vehicles.  Cisco uses the tightly co-engineered effort with the Nexus 1000v to provide access to their VSG offering and Juniper uses the VMsafe APIs for the vGW (nee’ Altor) firewall.  The issue is now one of VMware’s architecture for integrating moving forward.

Cisco has announced their forthcoming vASA (virtual ASA) product which will work with the existing Cisco VSG atop the Nexus 1000v, but this isn’t something that is “open” to the ecosystem as a whole, either.  To suggest that the existing APIs are “open” is inaccurate and without an API-based capability available to anyone who has the wherewithal to participate, we’ll see more native “integration” in private deals the likes of which we’re already witnessing with the inclusion of RSA’s DLP functionality in vShield/vSphere 5.

Not being able to replace vShield Edge with an ecosystem partner’s “edge” solution is really a problem.

In general, the potential for building a new generation of security products specifically designed for VMware’s virtualization software may be just beginning…

Well, it’s a pretty important step and I’d say that “beginning” still isn’t completely realized!

It’s important to note that these same vendors who have been patiently navigating VMware’s constant changes are also looking to emerging competitive platforms to hedge their bets. Many have already been burned by their experience thus far and see competitive platform offerings from vendors who do not compete with their own security solutions as much more attractive, regardless of how much marketshare they currently enjoy.  This includes community and open source initiatives.

Given their druthers, with a stable, open and well-rounded program, those in the security ecosystem would love to continue to produce top-notch solutions for their customers on what is today the dominant enterprise virtualization and cloud platform, but it’s getting more frustrating and difficult to do so.

It’s even worse at the service provider level where the architectural implications make the enterprise use cases described above look like cake.

It doesn’t have to be this way, however.

Jon finished up his piece by describing how the VMware/ecosystem partnership ought to work in a truly cooperative manner:

This seems like a worthwhile “win-win,” as that old tired business cliche goes. Heck, customers would win too as they already have non-VMware security tools in place. VMware will still sell loads of vShield product and the security industry becomes an active champion instead of a suspicious player in another idiotic industry concept, “coopitition.” The sooner that VMware and the security industry pass the peace pipe around, the better for everyone.

The only thing I disagree with is how this seems to paint the security industry as the obstructionist in this arms race.  It’s more than a peace pipe that’s needed.

Puff, puff, pass…it’s time for more than blowing smoke.

/Hoff

Related articles

• VMware’s (New) vShield: The (Almost) Bottom Line (rationalsurvivability.com)
• VMware strives to expand security partner ecosystem (infoworld.com)
• VMware preparing data loss prevention features for vShield (infoworld.com)
• Highlights from VMworld 2011 (datacenterknowledge.com)
• Clouds, WAFs, Messaging Buses and API Security… (rationalsurvivability.com)
• AWS’ New Networking Capabilities – Sucking Less  (rationalsurvivability.com)
• Sourcefire Enables Application Control Within Virtual Environments (it-sideways.com)
• Using The Cloud To Manage The Cloud (informationweek.com)
• Virtualizing Your Appliance Is Not Cloud Security (securecloudreview.com)
• OpenFlow & SDN – Looking forward to SDNS: Software Defined Network Security(rationalsurvivability.com)