Quick Fix: Upgrade to vCloud Director 9.5 fails due to a disabled ESXi host not in vCenter

While working on the upgrade of one of our vCloud Director cells from 9.1 to 9.5 the following error appeared after the binaries where installed and upgrade tool was executed to complete the DB work using
/opt/vmware/vcloud-director/bin/upgrade

The error was pointing to some ESXi hosts with disabled status. Starting with vCloud Director 9.5, disabled ESXi hosts are unsupported, so before starting the upgrade, you must enable all ESXi hosts.
For more details refer to upgrade documentation here

Our first attempt was to quickly overcome this issue by enabling the host (before realizing is not longer in vCenter!!). Attempt returned with error message below:

For some reason after decommissioning the ESXi hosts, vCloud Director didn’t fully remove it leaving the database with unwanted information. This could possible happen because the host was removed first from vCenter without disable/unprepare it from vCloud Director causing this error.
So .. in order to proceed with the successful upgrade we had to remove the Hosts no longer in the vCenter’s inventory performing the following vCloud Director database queries.
Note: In our scenario vCloud Director is running on Postgres10 and we are using PGadmin to manipulate the tables

Important! Please make sure to take a full backup of the database before executing the following steps.

1.Connect to vcloud director database (we are using PGadmin)

2.In vCD DB table go to” managed_server” table.

3.Right click the table and select query. In the query editor type and run
select * from managed_server;

This query will bring all hosts in VCD. During the database upgrade, the error the listed the ids of hosts with disabled status which are the same in column ms_id

select * from managed_server where is_enabled is false;

4. Run below query to delete hosts from vCloud Director inventory. Always replace the ID in quotes by the ones from your inventory

delete from managed_server_inv where ms_inv_id = '034cecec-b644-4a4b-9b4f-b2b8553b10bd';

Also query the child table managed_server_inv and delete the entries of the hosts in question then we need to delete.

5. Run again queries select * from managed_server where is_enabled is false; and select * from managed_server_inv where maintenance_mode is true; to validate the entries were successfully deleted.

6. Now you can re-attempt vCloud Director upgrade to 9.5!

Contact us for more information