Security Pros Say VirtSec Is An Operations Problem?
Mark Gaydos from Tripwire’s blog wrote an interesting article titled "Ops or Security: Who’s Responsible for Securing Virtualization?" The outcome is pretty much inline with my prior points that the biggest challenges we have in virtualization are operational and organizational rather than technical.
To wit, I quoteth from Mark’s post:
Tripwire recently performed a 25 question survey on virtualization security. Respondents broke down 78%/22% between management and administrator/staff respectively. We will be publishing a report around this survey in the next two weeks.
However, one of the interesting points that came out of the survey was that respondents feel that the operations team is responsible for securing a virtualized environment (almost two thirds of the respondents felt this way). This includes over half of the actual “security” personnel who took the survey who feel operations has this responsibility.
That’s right! Over half of the people covering security who responded to the survey said operations needs to secure virtual systems and not them.
My question is why? Does security not want to deal with virtualization? Do personnel feel that operations is closer to virtualization and they understand the issues? Does security just want to wash their hands of the issue? Or is management just leaning towards having operations handle everything around virtualization?
However, I wonder how much Mark read into the security personnel’s answers inasmuch as he suggests that they do "…not want to deal with virtualization" versus perhaps the fact that they don’t actually have the visibility or access to the tools to do so!*
Responsibility versus desire are two very different things!
Managing the "security" of virtualized environments today really centers around the deployments of virtual appliances and the configuration of the vSwitches. That means in a VMware environment, you have to have access and rights via Virtualcenter. The same is true in terms of Xen derivatives; if you don’t have access to configure and provision the networking and VM’s, you’re done.
Security in virtualized environments today is literally often thought of as a checkbox or two in a GUI somewhere. (All things considered, it would be great to be able to realize that one day…)
Just like security folks have locked server and network admins out of *their* firewalls and IPS’s, and as network folks have done the same in *their* routers and switches, virtual SysAdmins have done the same in *their* virtual server environments. If you don’t have access to the VM command and control, you can’t manage the security bits and pieces bolted onto it.
I don’t think it’s that the security folks *want* to surrender the responsibility, I think it’s that they never had it in the first place the moment the V-word entered the picture.
It ain’t rocket science. It ain’t voodoo. It ain’t a tectonic buck-passing conspiracy. It’s access, separation of duties (by force,) visibility and capability, plain and simple.
/Hoff
*Update: Per Amrit’s excellent comments, I look forward to Tripwire releasing the report to gain clarity on the question(s) asked as it begs the point as to whether the answers Mark refers to were in regards to the mechanical operationalization of security (the "doing" part) or the policy, strategy, audit and monitoring tasks. Are we talking about "security management" in general or "security operations?"
In either circumstance the "security" team is — based upon my observation from feedback — being left out of both.
In most organizations of any significant size the act of "managing" operational security is the responsibility of the operations teams, whether those teams are in desktop or server ops, or network ops, they hold the proverbial keys to the kingdom. Security teams do not generally patch, configure, deploy or modify infrastructure, although they may inform, audit, and monitor the infrastructure – this delineation of duties becomes even more abstracted in virtualized environments when multiple "functions" reside on a single physical device.
Another complicating factor is just how fluent op teams are in the dynamics of network security. This would be an important gap because the hypervisor is a kind of network appliance (hosting VMs). This is a "worlds in collision" scenario ripe for problems.
At http://www.gregness.wordpress.com I talk about virtualization-lite, which is the use of hypervisor VLANs as a short term solution that has helped VMware and VMsafe establish a beach head. If the survey was taken a year ago I think the results would be even more surprising: security wouldn't have been mentioned as a consideration or problem. I think virtsec is in a period of flux and a year from now we'll see virtsec experts who bring together knowledge of both domains.
For anyone interested in seeing the report (based on the results of the survey referenced) when it is published in two weeks, please send an email to highperformer@tripwire.com and we will be sure to send you the PDF.
Ops or Security: Whos Responsible for Securing Virtualization?
Tripwire recently performed a 25 question survey on virtualization security. Respondents broke down 78%/22% between management and administrator/staff respectively. We will be publishing a report around this survey in the next two weeks. However, on…
It starts with visibility. The security guys often have no idea what's going on inside the host environment. The auditors even less … No ACLs between virtual machines, no SoD, no key controls — there's going to be some brutal wake-up calls going out from the auditors.
"Security teams do not generally patch, configure, deploy or modify infrastructure, although they may inform, audit, and monitor the infrastructure…"
Yes, that is how the security teams would like to operate but more often than not they keep responsibility over infrastructure systems they must patch, configure, deploy and modify.
I think you might be blurring security with audit (a common issue) which invariably has no operational responsibilities. You will find security hats who also wear IT hats, but I have yet to meet a "significant size" company where the auditors have any operations role at all.
"The security guys often have no idea what's going on inside the host environment."
I think you mean the security staff have no idea what's going on inside a host environment they do not manage themselves.
Security Pros use virtual environments often and know what to look for and where to look. The problem is that management does not explicitly authorize them (yet) to extend their oversight role into others' virtual environments.
The sandbox concept has become far more pedestrian/democratic; users often used to have to work hard to be allowed space off the security team radar, but now that anyone can build a quick sandbox with virtualization the security team is faced with restating their case for oversight.
What percentage of network security pros could even say how many virtual machines they're protecting, much less what OS/APP? Yes hypervisor VLANs are a first step; but they aren't a LT solution. They limit flexibility and htey offer no inter-VM visibility or protection.
Hey Davi,
I only agree if are talking about a specific segment of the market, so perhaps we should segment the market:
0-1k
1k-10k
10k-50k
50k and above
Each of these have very different organizational characteristics. The 1k-10k organizations generally have very blended, multiple hat wearing folks, the 10k-50k tend to have domain focused individuals – security vs. ops, the 50k and above have area focused individuals – patch team, AV team, encryption team, etc…in the mid to large environments the security teams that do have operational responsibility are severely limited in what they can do to an end-point/servers (and should be) -> update AV but NOT modify configuration, registry settings, deploy applications, or patches, and in these cases the folks responsible for AV are generally not security professionals that are managing the AV infrastructure but operations folks; they do not sit in the lean-forward security role of assessing active threats.
If your organization has security professionals patching, managing AV, and performing other day to day security operations functions then something is broken.
This extends to the virtual world whether we are looking at the control plane or the communication plane.
I'd have to say part of the problem is how technical virtual security and infrastructure can be. It is also not very accessible to your everyday security guy and still relatively new. Talking intelligently about configurations and security for big virtual environments with an ops guy I imagine can be frustrating.
If A Virtualization Misconfiguration Or Security Vulnerability Exists Within An “ESX Appliance,” Does It Really Exist?
Earlier this week, I read a blog entry by the Lone Sysadmin about our recently released Tripwire ConfigCheck. He noted that it found some misconfigurations in his VMware ESX server, and then goes on to ask a very interesting question:
Now my question …
Who Will Own Virtualization And 10BASE-T Security?
Mark Gaydos wrote an article called “Ops or Security: Who’s Responsible for Securing Virtualization?” that hints at the virtualization survey results that we will be publishing next week, exploring how organizations are assigning responsibility for sec…
vRACF, anyone?
Of course the problem is organizational, but I'll also take issue with Amrit. I used to work for a communications company/ISP that had networking and security organizational silos so vertical that it was impossible to implement any new security solution or auditing package due to lack of agreement between upper managers. As a security admin I was specifically disallowed IOS and unix logins on critical servers by the network management, who was convinced IDS/IPS was impossible at Gb speeds (when I was hired I was literally told "you can work on the firewall/routers with the networking group, or you can join security to implement IDS/IPS, which is it?"), and who also failed to grasp the concept of n-tier dmz implementations, while Security was managed at the time by a former DoJ employee who felt that searching through employee's emails looking for fireable offenses was the true meaning of "security." When there was the inevitable failure on an external pen test, who's fault was it? Mine, of course, for having failed to implement an IDS/IPS system.
That kind of BS won't go away just because you put a V in front of it.