Recovering from “object does not exist or has never existed” error in VMware Infrastructure

Causes

  1. A virtual machine has been registered on more than one ESX server in a cluster.
  2. The resource groups on an ESX server in the cluster are different than the resource groups on the Virtual Center server.

Determining the Cause

To check for issue #1 above, do the following:

  • Run the command “vmware-cmd -l | grep /vmname.vmx” on each node in the cluster, if more than one node returns output from this command the VM is registered on multiple cluster nodes

To check for issue #2 above, do the following:

  • Use the virtual center client to connect to both the virtual center server that manages the cluster and the ESX server that the VM is having the issue on.
  • Compare the resource groups between the two windows, since the Virtual Center server is supposed to keep the resource configuration the same on all servers they should be identical.

Resolving the Problem

For issue #1 above

  • Shut down the VM in question
  • On all the ESX servers that showed output when finding the problem, run the command “vmware-cmd -s unregister vmxfile.vmx” the vmxfile.vmx must include the full path to the config file as it was returned from the “vmware-cmd -l | grep /vmname.vmx” above.
    • example:
      vmware-cmd -s unregister /vmfs/volumes/iscsi02/badvm/badvm.vmx

For issue #2 above

  • Place the ESX server with the issue in maint mode if possible (you do not want VMs running on it if possible since you will then have to worry about merging resource groups). Generally when this issue occurs it’s not possible to start VMs on the system in the first place so this isn’t normally much of a problem.
  • In the VI Client connected to the Virtual Center server, right click on the hostname for the problem host and select “disconnect” from the menu.
  • Wait for the server to disconnect
  • In the VI Client connected to the Virtual Center server, right click on the problem host and select “remove” from the menu.
  • Wait for the Virtual center to remove the host
  • In the VI Clent connected to the Virtual Center server, right click on the cluster and select “add host”
  • Take the default options, there are no resources on the server to merge so it is safe that way.
  • Wait for the new host to enter the cluster and configure for HA.
  • Repeat the check above to verify the resource groups are now correct.