Veeam Instant Recovery VM won’t boot after host failure

We were in the process of migrating VMs from one datacentre to another using Veeam Backup & Replication when our main host crashed with a CPU error. The production VMs had been running on the server for a few hours and hadn’t yet been fully replicated to the replication partner. One of the VMs is Server 2003 so was running in Instant Recovery mode as is couldn’t be migrated using replication like the other VMs.

Thankfully the server rebooted and all the VMs started without issue except the Server 2003 VM. When viewing the console it only displayed a black screen which was worrying as there was no backup of the last 6 hours of changes.

After submitting a case on the Veeam site a member of the support team called within 20 minutes and quickly got the VM up and running again. The issue in our case was that the Veeam B&R software is installed on a VM running on the host that crashed. When the VM restarted the Veeam services didn’t start in the required order. The Veeam vPower NFS service needs to start after the Veeam Backup service.

To fix the issue the only steps required were to restart the Veeam vPower NFS service on the system running Veeam B&R then start the VM.

Remove HP iLO License

In order to use the full features of HP’s Integerated Lights-Out system a license key needs to be added. The process of adding a license is as simple as connecting to the iLO web interface and following the steps in the license section. If for some reason you want to remove the license it’s not quite as simple as clicking a button.

Thankfully it’s not actually that difficult to remove the license, just a pain having to use an extra tool. The steps below show you how to complete the removal in a few minutes.

  1. Download Putty if you don’t already have it.
  2. Open putty and enter the IP of the iLO server, select SSH and open the connection.
  3. Login using the same credentials used for the iLO web interface (username is case sensitive).
  4. type the following command and hit enter. delete /map1 license
  5. The command may take 5-10 seconds to complete so wait until confirmation is shown.

Remove iLO License

Install Veeam Endpoint Backup on Hyper-V Server 2012 R2

Hyper-V Server 2012 R2 is a bare metal hypervisor which Microsoft makes available for free with the idea being that it will be managed remotely. Since it is only intended to be a host for VMs there is no GUI and the basic setup is done using the command line or Powershell. While Hyper-V Server is perfect for it’s intended use there are times when more applications like backup and monitoring software would be useful.

Thankfully Microsoft has not stripped out all the parts of the OS which support the use of applications with a GUI. There may not be a normal desktop or start menu but a large number of applications can still be used. The application I install most on Hyper-V Server is Veeam Endpoint Backup which I use as a quick and easy backup solution for test servers.

The install process is the same for most applications, just the file names and directories would be different.

  • From another computer on the network open \\192.168.0.50\C$ (where 192.168.0.50 is the IP of the Hyper-V server)
  • Enter the login details of an account with administrator privileges
  • Copy the install files to the C: drive of the server, I usually create a folder with a short name like C:\Veeam
  • Connect to the Hyper-V server using RDP
  • At the command line navigate to the folder created on the C: drive
  • Run the install by typing the full file name (I changed the install EXE name to installveeam.exe when I copied it across)
  • Follow the normal prompts to install Veeam including .NET framework

After the install has finished Veeam can be run by navigating to C:\Program Files\Veeam\Endpoint Backup\ and running the file veeam.endpoint.backup.exe

Most of the usual functions work with the exception of creating recovery media and some of file browse windows. I have successfully used recovery media created on another Server 2012 R2 system to recover a Hyper-V Server 2012 R2 using Veeam.

 

ESXi 5.5 error occurred consolidating disks

The setup is two ESXi 5.5 hosts, one is the main host with three live servers and the second is the backup host with three server replicas, Acronic Applicance and vCenter Server. There is a separate physical machine used as the backup target for the scheduled backup tasks. In addition to backing up to the physical target the three virtual servers are replicated to the secondary host each night.

Nightly backups of the three virtual machines using the Acronis appliance were failing and the Acronis appliance was powering off. According to the Acronis log it was running out of space while running the backups. Normally it is just a matter of deleting all the snapshots using vSphere Client but this time one of them just would not delete.

After reloading the vSphere client one of the replica VMs had a warning saying that it’s disks needed to be consolidated. Several attempts resulted in a error saying the disk images were locked. The Acronis appliance was turned off but it turns out that it was locking files relating to the replica VM. I had to boot and then shutdown the Acronis appliance three times before it would release the lock on the files and the consolidation worked perfectly followed by the removal of the snapshot.