This Blog is a supplement to my website. Please also visit: www.DavidCocke.com

Saturday, February 25, 2012

I don't get viruses

I'm an IT guy.  I don't get viruses.  
Well, come watch me eat some humble pie, and I'll tell you a story of how I got bit, and how I resolved it.

As is probably obvious by this blog I am an IT person.  Although I am far from perfect, I haven't had a virus or malware on my personal computer(s) in years, I mean like maybe close to a decade.  Since I'm in the business, I know what precautions to take, I observe links before I click them, I know how to kill tasks in the background that look a little suspicious, etc.  I probably watch what is in the web browser's address bar more than I watch anything else on the page so I can make sure I'm actually on the site I'm supposed to be.

Last night I got home and sat down at my computer and was about to begin a long session of upgrades for one of my clients.  I had seen a reference to a new movie being released called Act of Valor.  I knew my son had expressed an interest in seeing it and I wondered what it was rated.

So I went to fandango.com, clicked on the movie, and then suddenly I saw a browser window open with a fake antivirus.  I know these screens all to well, because I'm the one called in when one of my user's get bit.  But before my eyes could even focus to begin to read the screen and before I even moved the mouse, I watched it do a very quick download and reboot my computer, and bam! I had a virus.  Not just any virus, but a rootkit.  I've heard about these drive-by downloads, but I'd never experienced one before.

Time to Rant
I'm going to vent a little.  If you want to watch, read on.  Otherwise skip to the bottom to see how I removed it.  I am more than a little pissed off about this.  I run Windows 7 and I put up with the nagging UAC (User Access Control) prompts in the thought that this is supposed to prevent this sort of thing.  I run IE9 with the smart-screen filter on.  I use MSE (Microsoft Security Essentials).  I keep up with patches on everything, not just Windows.  And above all, I am careful.  I don't surf sites that are known for serving up malware, and I don't even download torrents or questionable software.  And yet...

When Microsoft first introduced UAC in Vista, I thought it was a dumb idea, and boy was it annoying.  Everything you clicked, the computer would essentially ask, "did you mean to click that? Are you sure?"  MS toned it down in Windows 7, and as I learned more about UAC, I began to at least appreciate what Microsoft was trying to accomplish and in fact I leave UAC on for my clients even though this has introduced a wide range of complexities with Login and Logoff scripts and general maintenance that we have to perform.

But why, to this day, with Windows 7 and UAC are rootkits still possible?  What have I missed?  Programs are supposed to not have direct access to the hardware without going through Windows, unless they are granted higher privileges, and UAC is supposed to ask you for permission to grant those higher privileges.

Thank you Sony for introducing the world to rootkits (see Wikipedia).  This is but one of several reasons I do not buy Sony products anymore.  There are other reasons, but that's for another rant.  And by God Microsoft, why after all these years can you not block these damnable things?  I could understand it if I booted from a CD or USB drive that the hard drive's MFT is fair game, but while I'm in Windows, surely you could devise a way to block it or at the very least alert the user before the deed is done.

Mixed Messages
OK, after I got bit and the computer rebooted I was facing the Blue Screen of Death on each reboot.  I was able to boot into Safe Mode and run MSCONFIG which allowed me to disable all non-Microsoft services as well as all Startup items.  This then allowed me to the boot into Windows normally without the BSOD errors.  But while I was in Safe Mode with Networking I opened up Microsoft Security Essentials and saw that my virus definitions were less than a day old.  I did a scan and it found nothing.  Then I updated the definitions and ran another scan and this time it reported that I had the alureon.a MBR rootkit virus.  It then told me that it couldn't remove the virus and suggested that I try the new Windows Defender Offline.  So from another computer I went and downloaded the 64-bit version of WDO (which supposedly has the latest definitions).

I booted from the WDO CD and it says it couldn't find any infections.  So MSE says I have a virus and the tool they suggest to remove can't find it?  So I downloaded and burned the Kaspersky Rescue CD and booted from that.  But it couldn't find anything either.  Then I ran Malwarebytes and it at least said it found an infection.  It reported that the file called C:\Windows\SvcHost.exe was infected and that a copy of that file was also running in memory.  Malwarebytes tried to remove it but after each reboot it came back, because it is a rootkit.  I had also turned off System Restore, but that didn't help either.

There is a legitimate SvcHost.exe file that lives in the C:\Windows\System32\ folder, just so you don't get them confused.

Try This, Try That
After poking around and trying some of my usual tricks, I said forget it, let me restore from last night's backup.  I use the Symantec Backup Exec System Recovery.  As I went through the restore options I noticed it didn't automatically offer to check "Restore the MBR" and I didn't choose that either.  In hindsight, I wonder if that would've corrected my issue.  But after about 45 minutes of waiting for the restore, I rebooted to find I still had the rootkit.

I then grabbed my Windows 7 install CD and booted from that, chose the Repair option, and then the Command Prompt and issued these commands:

  • bootrec /FixMbr
  • bootrec /FixBoot

This too didn't help.

Finally - A Fix
As I Googled around I began to see references to TDSSKiller.  I remember this tool.  Kaspersky made it.  In fact, I had a copy in my toolkit on my hard drive.  I ran that, it found and removed the rootkit.  Yeah!  Thanks Kaspersky.

But then I had to laugh.  Why is that my copy of TDSSKiller which is dated November of 2010 found and removed something that their Rescue CD that I just downloaded didn't find?  And why, with a virus that has been around for so long, couldn't Microsoft Security Essentials detect it with virus definitions that were only a day old, but could be detected by new ones?  Does this mean the rootkit somehow tainted MSE?  If so, that's another very serious concern.

As I was doing my research for this post (after I removed my virus) I have discovered that Kaspersky has a newer version of TDSSKiller dated Feb 7, 2012 available here:
http://support.kaspersky.com/faq/?qid=208283363

Perplexed
In looking back over this ordeal, what can I do different to prevent this from happening again?  At the moment, I'm not sure.  I feel like I am as protected as I can be without installing multiple antivirus and anti-malware programs.  I have always said, running more than one antivirus program is like wearing more than one condom.  It works, but it sure takes the fun out of it.

I do know this.  The next time one of my clients has malware, I will be a bit more empathetic, as it is entirely possible they really didn't do anything wrong by clicking something they shouldn't.  Well, maybe not "all" of my users.  Some just can't help themselves.



Friday, February 24, 2012

Android Phone Stops Syncing Contacts to Gmail

My Android phone stopped automatically syncing my contacts with Gmail.  Even if I went through the Settings menu and chose to do a manual sync, it would go through the motions but wouldn't update my contacts, and I also noticed that the date/time of the last sync wouldn't change.

I finally found the answer.  Many thanks to the user Peaser on this website:
http://androidforums.com/eris-support-troubleshooting/81579-fix-google-contacts-not-syncing.html

The solution:

In your phone, go to Settings, Applications, Manage Applications, and under the All tab, choose Contacts Storage.  Then push the "Clear Data" button.

Then resync your phone to Gmail:

In your phone, go to Settings, Accounts & Sync.  Under Managed Accounts, choose Google, and press the "Sync Now" button.

Followup (3/2/2012) 
I wanted to share a followup to this story.  A few days later I noticed my contacts weren't syncing again.  The behavior was slightly different.  Previously the date/time on the last sync didn't change.  This time it would update the date/time of the last sync but wouldn't actually sync the differences between my phone and Gmail.

So, I did this:
  • Settings
  • Accounts & Sync
  • Select your Google account
  • Uncheck "Sync Contacts"
---
  • Settings
  • Applications
  • Manage Applications
  • All (The All Applications Tab)
  • Select Contact Storage (I'm told that on some phones it is listed as "Google Contacts Sync")
  • Press the Clear Data button (which purges all contacts from your phone)
---
  • Settings
  • Accounts & Sync
  • Select your Google account
  • Check "Sync Contacts" (which should cause it to re-download your contacts from Gmail)


Monday, February 20, 2012

Unable to Browse VMware Datastores

Arg!  I can't browse my VMware vSphere Datastores.

It didn't start out that way, but let me tell you a story of how I started with one problem, only to discover another, and another, and then ultimately the fix for all of them.

I was setting up backup jobs using Trilead's VM Explorer (http://www.trilead.com/) and it kept getting stuck on a message "downloading VMX file".  When doing this from a working vSphere 5 server, this step takes half-a-second, but in my case I tried several VMs all on the same vSphere box and at one point let it set there for over an hour.

Occasionally when I tried I would eventually get a 503 http service unavailable error message.  I Googled both messages and found all sorts of suggestions for these problems and I tried most of them to no avail.  I should add I was seeing people describing this issue with various other backup products as well, like Veeam Backup, and Quest vRanger.

Then I tried using VM Explorer's File Explorer and I was able to browse the datastore and manually copy the VMX file, so I knew it wasn't locked.  I could also browse the contents of my datastores using an SSH/Putty connection.

But when I tried to browse the datastore using the vSphere Client, no matter which datastore I selected I got the "Searching Datastore...." and it would never display files or folders.  I Googled some more and found this too is a common issue with no concrete resolutions.  I also tried using the web browser to browse the datastores by going to the vSphere's IP address, but that lead to a spinning hour glass.

I tried rebooting the vSphere server several times and that didn't fix these issues.  I also tried powering down all of the VMs and placed it in Maintenance Mode, but again no luck.

Several weeks ago we setup a Buffalo Terastation NAS device and configured it to use the NFS protocol.  I had it mounted as a datastore as well.  The performance of this solution was awful and we are in the process of abandoning it.


As I sat there staring at the screen wondering what else to try and wondering if I was going to have to move these Virtual Machines to another server and completely re-install the vSphere software, I decided to remove the unused NFS datastore volume since I wasn't using it anyway.  Low and behold, after removing it I could suddenly browse both of the other datastores as quick as you please.

And my original issue with Trilead's software, the "downloading VMX file" problem also disappeared.

I would have never guessed that issues with one datastore volume would prevent you from accessing other datastores.

I hope this helps some other poor soul.


Friday, February 17, 2012

Access the vSphere 5 Console via SSH (Putty)

I just learned the neatest trick and I just had to share it.


I was unable to connect to a vSphere 5 server using the VMware vSphere Client.  


I needed to quickly reboot that server from remote.  I did have access to the server via SSH using Putty.


What I learned is that after you login via SSH just type the command: dcui
and viola! you have access to the console:




Sweet! This will be easy for me to remember, "David Cocke's User Interface".


To exit the above and return to the command line console, press CTRL-C.


Followup 


A couple of days later I found my self in a situation where I couldn't access the server via the vSphere Client and rebooting via the above GUI console didn't work either.


What did work was this command:


/sbin/shutdown.sh -r


The -r of course means reboot.  Without it, it will shutdown and not reboot.  You have to be patient after you issue this command.  In my case it took almost a minute before you actually started seeing a response.





Saturday, February 4, 2012

Unable to login with vSphere Client

This has happened to me on several occasions so I wanted to document this.  
When you use the VMware vSphere client to login to a vSphere server (in my case a vSphere 5 server) you are unable to login and you get a dialogue box that has a title of Connection Error, and a message that reads:


The server {the server you're trying to login to} could not interpret the client's request.  (The remote server returned an error: (503) Server Unavailable.)


Then below that it gives an error stack message:


Call "ServiceInstance.RetrieveContent" for object "ServiceInstance" on Server "{the server you're trying to login to}"


What appears to be happening is that the SSL encryption service on the vSphere server isn't running and the client cannot talk to the vSphere server using SSL.


What is frustrating is that you know the virtual machines are running, but since you cannot login with the client there is no graceful way to go into Maintenance Mode and/or to reboot the server.


The trick to fix it is to get the various services to restart including the one that handles SSL encryption.


So download Putty to a workstation, and connect to the server using SSH.  This assumes you have already enabled SSH on the vSphere Server.


Once logged in with Putty issue to the following command in the terminal window:


/sbin/services.sh restart


You will begin to see several services restart.  Once complete you should be able to login again using the vSphere client.  The above command works on vSphere version 5 (ESXi 5), but I have not tested this on ESXi 4.x.  


In my case this situation tends to occur when I place the server in Maintenance Mode and try to reboot it, but it doesn't reboot and you suddenly loose your connection to it with your vSphere Client and can't login again.


Followup 
July 19, 2012 - I found a faster and easier way to deal with this issue.  Please read my new post called: VmWare vSphere Client Error 503