Rational Survivability

Here is some of the recent press coverage on topics relevant to content on my blog:

• Information Week: Virtualization Has A Security Blind Spot
• Information Week: Securing Virtualization, or is that Virtualizing Security?
• Network World: Black Hat speakers expose virtualization, OS security gaps (**NOTE: Please see here, VERY important)
• Network World/Computerworld: Black Hat spotlights virtualization, DNS issues (**NOTE: Please see here, VERY important)
• SearchSecurity (Australia): Could securing virtualised environments destroy ROI?
• SearchSecurity: Initial virtualization costs could outweigh benefits
• Computer Zeitung: Today’s Security Products Aren’t Ready For Virtualised Data Centres
• Wall Street Journal: Hackers On the Move
• Baseline: Managing Mobility In the Enterprise
• ITWorld: Pros and Cons of VMware’s New Security Guide

Podcasts/Webcasts/Video:

• Network Security Podcast: Defcon Wrap-up (*naughty language)
• Security Wire Weekly: Virtual Security Apocalypse
• DarkReading: Four Horsemen Interview after my talk at Blackhat
• SearchSecurity "Still Nameless Podcast": Rich Mogull and associated nonsense (I wasn’t on it, just in it 😉

I am confirmed to  speak at the following upcoming events:

• Forrester’s Security Forum, Boston MA, September 4-5
• [Private] Portland, Oregon, September 8-11
• [Private], Utah, October 1-2
• Day-Con II, Dayton, Ohio, October 10-12
• RSA Europe, London UK, October 27-29
• TechTarget Information Security Decisions, Chicago IL, November 5

I will be attending the following events:

• VMworld 2008, Las Vegas NV, September 14-18

/Hoff

Bruce Schneier has artfully committed electrons to decay in an article he recently "penned" for Wired in which he has once again trumpeted the impending death of Information Security as we know it and illustrating the changing why’s, how’s, when’s and who’s that define the security industry singularity that is sure to occur.

While I thoroughly enjoyed Bruce’s opinion on the matter and will address it in a follow-on post dedicated to the meme, the real gem that sparkled for me in this article was his use of how the behemoth RSA Security conference is actually a bellweather for the security industry:

Last week was the RSA Conference, easily the largest information
security conference in the world. More than 17,000 people descended on
San Francisco’s Moscone Center to hear some of the more than 250 talks,
attend I-didn’t-try-to-count parties, and try to evade over 350
exhibitors vying to sell them stuff.

Talk to the exhibitors, though, and the most common complaint is that the attendees aren’t buying.

It’s not the quality of the wares. The show floor is filled with
new security products, new technologies, and new ideas. Many of these
are products that will make the attendees’ companies more secure in all
sorts of different ways. The problem is that most of the people
attending the RSA Conference can’t understand what the products do or
why they should buy them. So they don’t.

…

The RSA Conference won’t die, of course. Security is too important for
that. There will still be new technologies, new products and new
startups. But it will become inward-facing, slowly turning into an
industry conference. It’ll be security companies selling to the
companies who sell to corporate and home users — and will no longer be
a 17,000-person user conference.

What attracted me to the last paragraph and a rather profound point draped in subtlety that I think Bruce missed was reinforced by my recent experiences in Boston and Munich which framed RSA, which quite honestly I could almost care less about attending ever again…

Specifically, I recently attended and spoke at both SourceBoston (in Boston) and Troopers08 (in Munich, Germany.)  These are boutique security conferences with attendee counts in approximately the 200 person range.  They are intimate gatherings of a blended and balanced selection of security practitioners, academics, technologists, researchers and end-users who get together and communicate.

These events offer a glimpse into the future of what security conferences can and should provide: collaborative, open, educational, enlightening and fun events without the pretentiousness or edge of confabs trying too hard to be either too "professional" or "alternative" in their appear and nature.

Further, these events lack the marketing circle-jerk and vendor-centric detritus that Bruce alluded to.  What you get is a fantastic balance of high-level as well as in-the-weeds presentations on all manner of things security: politics, culture, technology, futurism, hacking, etc.  It’s an amazing balance with a refreshing change of pace.  People go to all the presentations because they know they are going to learn something.

These sorts of events have really been springing to life for years, yet we’ve seen them morph and become abstracted from the reason we attended them in the first place.  Some of them like BlackHat, DefCon, and ShmooCon have all "grown up" and lost that intimacy, becoming just another excuse to get together and socialize in one place with people you haven’t seen in a while. 

Some like HITB, CanSecWest, and ToorCon might appear too gritty or technical to attract a balanced crowd and the expectations for presenters is the one-upmanship associated with an overly-sensationalized exploit or the next move in the fanboy-fanned flaming game of vendor 0day whack-a-mole.  Others are simply shows that are small or regional in nature that folks just don’t know about but remain spectacular in their lineups.

My challenge to you is to discover these shows — these "Non-Cons" as I call them.  They offer fantastic networking, collaborative and learning opportunities and you’ll be absolutely blown away with some of the big names presenting at them.

Don’t turn up your nose simply because of locale and use the excuse that you’re saving your budget for RSA or InfoSec.  When is the last time you actually *learned* anything at those shows?  It costs thousands to attend RSA.  Many of the Non-Cons cost a measly couple of hundred dollars.

Take a close look at where your favorite InfoSec folks are presenting.  If five of them happen to be converging on, say, Ohio <wink, wink> for 2-3 days at a security conference you’ve never heard of, it’s probably not because of the beaches…

/Hoff

It seems I forgot to specifically call out the fact that I’ll be in San Francisco attending the RSA Conference this week.

Monday had me at the America’s Growth Capital conference with the remainder of the week spread between sessions, briefings, meeting up with old friends and making new ones.  I’m leaving back to Boston Friday morning.

I’m speaking on Wednesday (DEPL-201)

/Hoff

As previously mentioned, last week, Mogull and I presented at SOURCEBoston.  Our offering was a bit of a rough first-pass mashup at peering my talk on "Disruptive Innovation" with Rich’s excellent "Future of Security" presentation.  It went over decently well and five minutes after the preso., I bailed to the airport for a flight to New Zealand.

Upon my return, I was catching up on email and noticed all manner of really great feedback on Dan Geer’s keynote that he gave the day after I left.  I was saddened by the fact that I missed it and was really looking forward to reading the transcript of Dan’s talk given how much of a fan I am of his work and intellect.

What followed next ranged from confusion to amusement to happiness and then annoyance and disgust.  I’ve been wrestling with how to frame this so as not to imply anything at all negative about Dan as I respect him tremendously and do not in any way wish to besmudge him.

I attribute what you are about to read to serendipity and kismet with the unfortunate side-effect caused by a small but persistent group of annoying individuals who have nothing better to do than create conspiracy theories in between games of Halo3 and unrequited love via match.com.

If you read the transcript of Dan’s presentation, you will be struck when comparing presentations that a large portion of it mirrors the content and thematic representation in my presentation, down to some incredibly specific examples and references as well as a choice number of unique analogs and anecdotes.

I wasn’t particularly concerned by this, in fact I was jazzed when I realized that Dan was not only saying the same things I was but that we were interlocked on some really cool examples…all until I started getting emails and blog comments suggesting that I had ripped off Dan’s work.

So, let me just (sadly) state for the record two things:

1. The material in the presentation I gave on 3/12 was an updated version of my keynote presentation I gave at the Information Security Decisions show in Chicago in October 2007.  In fact, I posted the narrative slide-by-slide in four parts:
   • Security and Disruptive Innovation Part I: The Setup
   • Security and Disruptive Innovation Part II: Examples of Disruptive Innovation/Technology in the Security Space
   • Security and Disruptive Innovation Part III: Examples of Disruptive Innovation/Technology in the Security Space
   • Security and Disruptive Innovation Part IV: Embracing Disruptive Innovation by Mapping to a Strategic Innovation Framework
2. Rich and I presented the day before Dan did.

So, for those of you who have decided to annoy me and call into question my honor and credibility, you can take both those issues above and stuff ’em in your…it’s clear that I authored and published the bulk of my presentation almost 6 months ago and I spoke before Dan did.  This would make it difficult for me to rip him off unless I was psychic.

I know without a doubt that he didn’t take any of this from me, either, and there’s no reason to suggest otherwise.  I’ll just chalk it up to a great mind (his) and a mediocre one (mine) thinking alike.

So in closing, I’m thrilled that we both spoke of punctuated equilibrium, dampened oscillations, disruptive innovation, cyclical evolution, etc.  It means that I’m doing the same sort of thinking as someone that I truly admire.

I intend to reach out to Dan and tell him how much I really enjoyed his keynote and share with him ahead of time some of my emerging work on chaos theory, the dip and predictive economic modeling theory as applied to InfoSec…I only wish our presentation went over as well as his did 😉

I trust we can put this to bed now?

/Hoff

It’s all about awareness, and as you’ll come to read, there’s just not enough of it when we talk about the security implications of virtualizing our infrastructure.  We can fix this, however, without resorting to FUD and without vendors knee-jerking us into spin battles. 

I’m attending VMworld 2007 in San Francisco, looking specifically for security-focused information in the general sessions and hands-on labs.  I attended the following sessions and a hands-on lab yesterday:

• Security Hardening and Monitoring of VMware Infrastructure 3 (Hands-on Lab)
• Security Architecture Design and Hardening VMware Infrastructure 3 (General Session)
• Using the Secure Technical Implementation Guide (STIG) with VMware Infrastructure 3 (General Session)

I had high hopes that the content and focus of these sessions would live up to the hype surrounding the statements by VMware reps at the show.   As Dennis Fisher from  SearchSecurity wrote, there are some  powerful statements coming from the VMware camp on the security of virtualized environments and how they are safer than non-virtualized deployments.  These are a bold, and in my opinion, dangerously generalized statements to make when backed up with examples which beg for context:

To help assuage customers’ fears, VMWare executives and security
engineers are going on the offensive and touting the company’s ESX
Server as a more secure alternative to traditional computing setups.
Despite the complexity of virtualized environments, they are inherently
more secure than normal one-to-one hardware and operating system
environments because of the hypervisor’s ability to enforce isolation
among the virtual machines, Mukundi Gunti, a security engineer at
VMWare said in a session on security and virtualization Tuesday.

"It’s a much more complex architecture with a lot of moving parts.
There are a lot of misconceptions about security and virtualization,"
said Jim Weingarten, senior technical alliances manager at VMWare, who
presented with Gunti. "Virtual machines are safer."

So I undertook my mission of better understanding how these statements could be substantiated empirically and attended the sessions/labs with as much of an open mind as I could given the fact that I’m a crusty old security dweeb.

Security Hardening and Monitoring of VMware Infrastructure 3 (Hands-on Lab)
The security hardening and monitoring hands-on lab was very basic and focused on general unix hardening of the underlying RH OS as well as SWATCH log monitoring.  The labs represented the absolute minima that one would expect to see performed when placing any Unix/Linux based host into a network environment.  Very little was specific to virtualization.  This  session presented absolutely nothing new or informative.

Security Architecture Design and Hardening VMware Infrastructure 3 (General Session)
The security architecture design and hardening session was at the very end of the day and was packed with attendees.  Unfortunately, it was very much a re-cap of the hands-on lab but did touch on a couple of network-centric design elements (isolating the VMotion/VIC Management ports onto separate NICs/VLANs, etc) as well as some very interesting information regarding the security of the virtual switch (vSwitch.)  More on this below, because it’s very interesting.

The Epiphany
This is when it occurred to me, that given the general roles and constituent responsibilities of the attendees, most of whom are not dedicated network or security folks, the disconnect between the "Leviathan Force" (the network and network security admins) and the "Sysadmins" (the server/VMM administrators) was little more than the mashup of a classic turf battle and differing security mindsets combined with a lack of network and information security-focused educational campaigning on the part of VMware.

After the talk, I got to spend about 30 minutes chatting with VMware’s Kirk Larsen (Engineering Product Security Officer) and Banjot Chanana (Product Manager for Platform Security) about the lack of network-centric security information in the presentations and how we could fix that.

What I suggested was that since now we see the collapse and convergence of the network and the compute stacks into the virtualizaton platforms, the operational impact of what that means to the SysAdmins and Network/Information Security folks is huge. 

The former now own the keys to the castle whilst the latter now "enjoy" the loss of visibility and operational control.  Because the network and InfoSec folks aren’t competently trained in the operation of the VM platforms and the SysAdmins aren’t competently trained in securing (holistically — from the host through to the network) them, there’s a natural tendency for conflict.

So here’s what VMware needs to do immediately:

1. Add a series of whitepapers and sessions that speak directly to assuage the fear of the virtualization unknowns targeting the network and InfoSec security staffers. 
2. Provide more detail and solicit feedback relating to the technical roadmaps that will get the network and InfoSec staffer’s visibility and control back by including them in the process, not isolating them from it.
   
3. Assign a VMware community ombudsman to provide outreach and make his/her responsibility to make folks in our industry aware — and not by soundbites that sponsors contention — that there are really some excellent security enhancements that virtualization (and specifically VMware) bring to the table.
4. Make more security acquisitions and form more partnerships.  Determina was good, but as much as we need "prevention" we need "detection" — we’ve lost visibility, so don’t ignore the basics.
   
5. Stop fighting "FUD" with "IAFNAB" (It’s a feature, not a bug) responses
6. Give the network and InfoSec folks solid information and guidance against which we can build budgets to secure the virtualization infrastructure before it’s deployed, not scrap for it after it’s already in production and hackbait.
   
7. Address the possibility of virtualization security horizon events like Blue Pill and Skoudis’ VM Jail escapes head-on and work with us to mitigate them.
8. Don’t make the mistakes Cisco does and just show pretty security architecture and strategy slides featuring roadmaps that are 80% vapor and call it a success.
9. Leverage the talent pool in the security/network space to help build great and secure products; don’t think that the only folks you have to target are the cost-conscious CIO and the SysAdmins.
10. Rebuild and earn our trust that the virtualization gravy train isn’t an end run around the last 10 years we’ve spent trying to secure our data and assets.  Get us involved.
    

I will tell you that both Kirk and Banjot recognize the need for these activities and are very passionate about solving them.  I look forward to their efforts. 

In the meantime, a lot of gaps can be closed in a very short period by disseminating some basic network and security-focused information.  For example, from the security architecture session, did you know that the vSwitch (fabric) is positioned as being more secure than a standard L2/L3 switch because it is not susceptible to the following attacks:

• Double Encapsulation
• Spanning Tree Floods
• Random Frames
• MAC Address Flooding
• 802.1q and ISL Tagging
• Multicast Brute Forcing

Basic stuff, but very interesting.  Did you know that the vSwitch doesn’t rely on ARP caches or CAM tables for switching/forwarding decisions?  Did you know that ESX features a built-in firewall (that stomps on IPtables?)  Did you know that the vSwitch has features built-in that limit MAC address flux and provides features such as traffic shaping and promiscuous mode for sniffing?

Many in the network and InfoSec career paths do not, and they should.

I’m going to keep in touch with Kirk and Banjot and help to ensure that this outreach is given the attention it deserves.  It will be good for VMware and very good for our community.  Virtualization is here to stay and we can’t afford to maintain this stare-down much longer.

I’m off to the show this morning to investigate some of the vendor’s like Catbird and Reflex and what they are doing with their efforts.

/Hoff

Mark Wood from nCircle blogged about his recent experience at the Gartner IT Security Summit in D.C.  Alan Shimel commented on Mark’s summary and both of them make an interesting argument about how Gartner operates as the overall gauge of the security industry.  Given that I was  also there, I thought I’d add some color to Mark’s commentary:

In 2006, there were two types of solutions that seemed to dominate
the floor: network admission control and data leakage (with the old
reliable identity and access management coming in a strong third). This
year, the NAC vendors were almost all gone and there were many fewer
data leakage vendors than I had expected. Nor was there any one type of
solution that really seemed to dominate.

…that’s probably because both of those "markets" are becoming "features" (see here and here) and given how Gartner proselytizes to their clients, features and those who sell them need to spend their hype-budgets wisely and depending upon where one is on the hype cycle (and what I say below,) you’ll see less vendors participating when the $ per lead isn’t stellar.  Lots and lots of vendors in a single quadrant makes it difficult to differentiate.

The question is: What does this mean? On the one hand, I continue to
be staggered by the number of new vendors in the security space. They
seem to be like ants in the kitchen — acquire one and two more crawl
out of the cracks in the window sill. It’s madness, I tell you! There
were a good half a dozen names I had never seen before and I wonder if
the number of companies that continue to pop up is good or bad for our
industry. It’s certainly good that technological innovation continues,
but I wonder about the financial status of these companies as funding
for security startups continues to be more difficult to get. There sure
is a lot of money that’s been poured into security and I’m not sure how
investors are going to get it back.

Without waxing on philosophically on the subconscious of the security market, let me offer a far more simple and unfortunate explanation:

Booth space at the Gartner show is one of, if not the most, expensive shows on the planet when you consider how absolutely miserable the scheduling of the expo hours are for the vendors.  They open the vendor expo at lunch time and during track sessions when everyone is usually eating, checking email, or attending the conference sessions!  It’s a purely economic issue, not some great temperature taking of the industry.

I suppose one could argue that if the industry were flush with cash, everyone showing up here would indicate overall "health," but I really do think it’s not such a complex interdependency.  Gartner is a great place for a booth if you’re one of those giant, hamster wheel confab "We Do Everything" vendors like Verisign, IBM or BT.

I spoke to about 5 vendors who had people at the show but no booth.  Why?  Because they would get sucked dry on booth costs and given the exposure (unless you’re a major sponsor with speaking opportunities or a party sponsor) it’s just not worth it.  I spoke with Ted Julian prior to his guest Matasano blog summary, and we looked at each other shaking our heads.

While the quality of the folks visiting are usually decision makers, the foot traffic is limited in the highly-compressed windows of availability.  The thing you really want to do is get some face time with the analysts and key customers and stick and move. 

The best bang for the exposure buck @ Gartner is the party at the end of the second day.  Crossbeam was a platinum sponsor this year; we had a booth (facing a wall in the back,) had two speaking sessions and sponsored a party.  The booth position and visibility sucked for us (and others) while the party had folks lined out the door for food, booze and (believe it or not) temporary tattoos with grown men and women stripping off clothing to get inked.  Even Stiennon showed up to our party! 😉

On the other hand, it seemed that there was much less hysteria than
in years past. No
"we-can-make-every-one-of-your-compliance-problems-vanish-overnight" or
"confidential-data-is-seeping-through-the-cracks-in-your-network-while-you-sleep-Run!-Run!"
pitches this year. There seems to be more maturity in how the industry
is addressing its buying audience and I find this fairly encouraging.
Despite the number of companies, maybe the industry is slowing growing
up after all. It’ll be interesting to see how this plays out.

Well, given the "Security 3.0 theme" which apparently overall trends toward mitigating and managing "risk", a bunch of technology box sprinkling hype doesn’t work well in that arena.  I would also ask whether or not this really does represent maturity or the "natural" byproduct of survival of the fittest — or those with the biggest marketing budgets?  Maybe it’s the same thing?

/Hoff