As of tomorrow morning, VM’s running on all hosts with ESX 3.5U2 in enterprise configurations will not power on.
Apparently, there is some bug in the vmware license management code. VMware is scrambling to figure out what happened and put out a patch.
OK, while we’re all remaining calm….just imagine the implications that bugs like this can occur and get past QA testing….5 years down the road, nearly all server apps worldwide pretty much running in VM’s (pretty easy prediction)……some country decides to initiate cyberwarfare and manages to get a backdoor into whatever is the prevaling hypervisor of the day…..boom. All your VM’s belong to us.
I honestly think a lot of the new hype about products dedicated to the new industry of vm security is crap, but honestly — god protect us if the baseline code for critical hypervisors like ESX isn’t kept secure and regularly audited.
I’d love to find out what happened here. Don’t they do any regression testing on new releases to check for date based bugs? I thought that would be pretty obvious.
UPDATE: Frank Wegner has posted the following suggestions:
You can see the latest status here: http://kb.vmware.com/kb/1006716 Please check back often, because it will notify you when this issue has been fixed. Until then the best workaround I can think of is:* Do nothing
* Turn DRS off
* Avoid VMotion
* Avoid to power off VM’s
I’d council against turning DRS off as that actually deletes resource pool settings….instead, set sensitivity to 5 which should effectively disable it w/ minimal impact.
UPDATE 2: VMware Website appears to be having trouble keeping up with people requesting updates.
UPDATE 3: VMware has stated they will have fixes available in 36hrs at the earliest.
UPDATE 4: Anand Mewalal comments:
We used the following workaround to power on the VM’s.
Find the host where a VM is located
run ‘ vmware-cmd -l ‘ to list the vms.
issue the commands:
service ntpd stop
date -s 08/01/2008
service ntpd start
UPDATE 5: Apparently, there are no easily seen warnings in logs/etc or VC prior to hitting the bug. VC will continue to show the hosts as licensed and no errors will appear in vmkernel log file until you try to start up a new vm, reboot a vm, or reboot the host.
[phpbay]vmware, 5, “”, “”[/phpbay]
Related Posts: On this day...
- Kanye and Jay-Z destroy a Maybach - 2011
- Unwritten 2: A pulse-pounding graphic novel that shows the grim and selfish ways that people use stories - 2010
- Criterion Collection reissues R. Crumb documentary with special features - 2010
- HOWTO: Block brute force attacks on your SSH server - 2010
- Microsoft Rolling out Retail stores to compete directly with the Apple stores - 2009
- Showdown at the 4chan corral - 2009
- "Open Source Twitter" proposed as antidote to Twitter's DDOS vulnerability - 2009
- Not your normal road on Google Street View - 2009