Instance Missing on Hypervisor

Problem

  • Instance goes into ERROR state after failing to start.

  • Running openstack server show <uuid> command shows an instance fault such as the following.

    Error: Instance <UUID> could not be found on hypervisor.

Environment

Platform9 Managed OpenStack - All Versions
Nova

Causes

Instance failed to migrate.
Disk is full on hypervisor.

Resolution

Instance Failed to Migrate

  1. Determine on which host the instance resides on.

    # openstack server show <UUID>

  2. Check if the instance is running on the host obtained from the previous output.

 ` # virsh list --all | grep <UUID>`
  1. Confirm that the instance XML file still exists on the hypervisor.

    #ls -l /etc/libvirt/qemu/<UUID>.xml
    #ls -l /opt/pf9/data/instances/<UUID>/cfg-<UUID>.xml

  2. If the XML file resides on the host, redefine the instance using virsh .

    #virsh define /etc/libvirt/qemu/<UUID>.xml

  3. Reset the state of the instance.

    # openstack server set --state active <UUID>

Disk Full

  1. Identify which partition is full.

    #df -h

  2. Inspect which folder(s) or file(s) are consuming the most amount of disk space.
    # du -xh <partition-path> | sort -h | tail

  3. At your discretion, optionally backup and/or remove or truncate the offending large files which are consuming the most space.

If you have any doubts about deleting files owned by the pf9 user, please reach out to us.