Cannot open the disk and could not open change tracking file

Lately I have seen a number of VMs going down after the backup has finished and snapshots get removed. At some point when cleaning the snapshot, the ESXi host decides something is wrong and then brings down the VM. After this we’re unable to power on the VM and receive the error: “Cannot open the disk ‘/vmfs/volumes/xxxxxxxxxx/xxxxx.vmdk’ or one of the snapshot disks it depends on”. When you then check the event details of the VM, you’ll see two messages:

– “Cannot open the disk ‘/vmfs/volumes/xxxxxxxxxx/xxxxx.vmdk’ or one of the snapshot disks it depends on”.
– “Could not open/create change tracking file”.

 

cbt001

The fix to make the VM power on again is simple, however we still haven’t been able to solve the root cause. VMware Support has suggested to bring down the number of LUNs connected to a single host and to upgrade to vSphere 5.5.

[Read more…]

VMware Auto Deploy with MSCS won’t work

Our environment is fully built using VMware Auto Deploy 5.1. All the clusters have their own ruleset and we’re very content on how easily we update our hosts and deploy new hosts. However, recently we ran into a major issue with Microsoft Cluster Services inside VMs in combination with VMware Auto Deploy. In a scenario with an “old fashioned” MCSC cluster across boxes with Physical RDMs shared between the two cluster nodes (VMs), these RDMs seem to be stopping an Auto Deployed ESXi host from reconnecting to vCenter Server after a reboot.  [Read more…]

Did you know, no VMotion when HA network partitioned

During my migration from vSphere 4.1 to vSphere 5.5, I ran into an issue I had never experienced before. When your VMware HA network is partitioned, vCenter will not let you perform VMotions. At first I was surprised but after some searching I learned the reasons behind it and it makes complete sense now. [Read more…]

Changing SATP Claimrules for specific storage configurations

At my current customer I was upgrading their vSphere 4.1 environment to vSphere 5.5. When installing and configuring the first ESXi 5.5 host, I noticed a difference in the Path Selection Policy (PSP) between the ESX 4.1 hosts and the ESXi 5.5 host. Each host is connected to two Hitachi storage arrays, each a different model. In the current 4.1 config a general setting is used to have both arrays use Round Robin:

esxcli nmp satp setdefaultpsp --psp VMW_PSP_RR --satp VMW_SATP_DEFAULT_AA

Because of the above rule in the ESX 4.1 hosts, all storage that is not captured by the existing rule sets, will be set to Round Robin (VMW_PSP_RR). Although applying this rule to my ESXi 5.5 host, would solve the problem for me on short term, I could imagine running into an issue when the customer would attach a third storage that didn’t required Round Robin, but would then somehow still default to it, because of this “catch-all” rule. [Read more…]

Add Fusion IO driver to VMware Auto Deploy

A customer of mine, who was already running a vSphere environment on Cisco UCS blades, asked to expand his environment with a number of ESXi hosts that could run a VMware View environment. To be able to run as much View desktops on each host as possible, we offered them Cisco UCS Blades equipped with a Fusion IO card, the UCS 785GB MLC Fusion-io ioDrive2 to be exact.Fusion IO iodrive2

For this customer I had already deployed a vSphere environment a year ago, fully based on VMware Auto Deploy and it now was time to try and add a driver to this Auto Deploy environment, which I had never done before. It wasn’t an easy road, but I wrote down all the steps and hope it helps you should you ever have to do this too. [Read more…]