Home > General Rants & Raves > A Funny Thing Happened On My Way To Malware Removal…

A Funny Thing Happened On My Way To Malware Removal…

Update 030712: I’m going to follow this post up with yet another post mortem that includes lessons learned and more details as I can supply them.  I will point out two things:

  1. It’s pretty clear that the secondary/tertiary stages of this infestation which led to multiple alerts from my readers is related to the massive WordPress attack you can read about here.  It’s important to note, however, that the first incident (which was chalked up impoperly to a false positive) and a second started with similar symptoms back in late July.  I simply didn’t have the data to correlate.  They were different variants.
  2. The support from vendors and the security community has been outstanding.  People with no vested interest in the health of my personal blog have gone out of their way to help, even my hosting provider, Dreamhost (although we got off on a rather rocky footing ;)

I do owe both FireEye (who spotted the original attack) and Dreamhost additional data which I will attempt to retrieve.  I also owe Rich Mogull an apology/explanation regarding why I didn’t immediately take the blog down, risking further infection — I legitimately thought we’d fixed it, but because of the stealth of the malware, I was wrong.  Once I realized I couldn’t contain/isolate it, I did take it down…and then wiped the entire blog/database.

At any rate, thanks for bearing with me though this.  It’s been invaluable to me and I hope you found some value in all of this.

It certainly was interesting and gave me some unique insight into the psychology, behavior, biases and opinions of the community/industry that I didn’t fully appreciate prior.

This is an update that I originally included with the post describing the malicious infestation of malware on my WordPress site here.  I’ve split it out for clarity.

The last 12 hours or so have been fun. I’ve had many other folks join in and try to help isolate and eradicate the malware that plagued my WordPress install (read the original post below.)

I was able to determine that the Dreamhost password compromise in January (correlated against logs) was responsible for the (likely) automated injection of malicious PHP code into a plug-in directory that had poor permissions.  This code was BASE64 encoded. It was hard to find.

Further, as was alluded to in my earlier version of this post, the malware itself was adaptive and would only try (based on UA and originating IP) to drop it’s Windows-based trojan executable ONCE by way of a hidden iFrame. Hit it again and you’d never see it.

It was a variant of the Blackhole Exploit kit.

If you ran any up-to-date AV solution (as evidenced by the 6 different brands that people reported,) visiting my site immediately tripped an alert.  I run a Mac and up until today didn’t have such a tool installed. I clearly do now as a detective capability.  This was a silly thing NOT to do as it costs basically nothing to do so these days.

When I made a backup of the entire directory, my VPS hosting provider THEN decided to run a security scan on the directory (serendipity) and notified me via email that it found the malware in the directory :( Thanks.  Great timing.  The funny thing was that all the activity last night and uploaded telemetry must have set something off in Google because only late last night — 30+ days later — did Google flag the site as potentially compromised.  Sigh.

At any rate, I ended up nuking my entire WordPress and mySQL installations and doing a fresh install. I’ve rid myself of almost every plug-in and gone back to a basic theme.  I’ve installed a couple of other detective and preventative tools on the site and will likely end up finally putting the site behind CloudFlare for an additional layer of protection.

Really, I should have done this stuff LONG ago…this was my personal failure.  I owe it to the kindness and attentiveness of those who alerted me to the fact that their AV sensors tripped.

The interesting note is that most of the security pros I know who run Macs and have visited my site in the last 30 days never knew I was infected.  If this were a Mac-targeted malware, perhaps they may have been infected.  The point is that while I’m glad it didn’t/couldn’t infect Mac users, I do care that I could have harmed users with other operating systems.

Further, the “ignorance is bliss” approach is personally alarming to me; without a tool which many security pros sleight as “useless,” I would never have know I was infected.

If anything, it should make you think…

Categories: General Rants & Raves Tags:
  1. Matthew
    March 6th, 2012 at 14:31 | #1

    Some people on the DreamHost forum are adamant that this problem was solely due to user error. I’d love to know more specifics on how you traced it back to January’s hack.

  2. March 6th, 2012 at 16:31 | #2

    Hi Chris,
    So… if we are Win-browsers of your blog, what is likelihood we have been exposed?
    Which infection would it have attempted?
    How can we tell & remove?
    Thanks!
    MarkT

    • beaker
      March 7th, 2012 at 14:56 | #3

      Mark:

      You bring up an excellent question I should have (and will) address.

      There were, as far as I can tell, three different variants of a similar piece of malware dropped. I don’t have a copy of the .exe because I run a Mac and the file never reached my machine but I’m awaiting a sample from someone who collected it.

      What I do know is that 2 of the 3 variants installed what is now characterized as infecting hundreds of thousands of hosts as the “fake AV scanner” – you can read more here:

      http://www.networkworld.com/news/2012/030712-fake-av-attack-targets-wordpress-257030.html?hpg1=bn

      Note that the first iteration of the insertion that was ruled out as a false positive (clearly it wasn’t) did this back in late January.

      As far as remediation, malware removal tools seems to be marginally effective (as is often the case with these things)

      It’s truly disconcerting that I may have (unbeknownst to me) infected folks and I apologize profusely if I did. The good news is that many people notified me that their detective tools blocked the infection…

  3. March 7th, 2012 at 09:09 | #4

    Malware is getting incredibly creative, and software that exploits and targets specific OSes, even specific sites/apps (by looking up the sites you visit via history exploits) are becoming common.

    The entire chain of events is interesting… well done for documenting it.
    Let’s face it, anyone, at any time can be a victim. The response and communication is just as key for your blog (as popular as it is) as it is for a Fortune 100.

    Well done, carry on good squirrel.

  1. March 8th, 2012 at 14:03 | #1
  2. March 15th, 2012 at 08:14 | #2