Yesterday I had a scheduled maintenance to upgrade a vSphere environment from 6.5 to 6.7U2. This includes the upgrade from vCenter version 6.5 to 6 7U2C. Unfortunately I had a NTP error that caused the failure of the upgrade job. The deployment of the target (vCenter) appliance with the ISO file provided by VMware wasn’t the problem. The problem occurred in stage two and that’s migration phase.

Troubleshooting

The error message is very clear. The NTP server configured in the vCenter appliance is unreachable. The first test i did was trying to ping the NTP server from the vCenter appliance. i received ping replies back from the NTP server. Next step was testing the DNS name resolving with nslookup and that worked fine as well. With help from a network engineer, we found out that port 123 (NTP) was not open for the network block where the vCenter appliance resides in.

Workaround

Configure the vCenter appliance to use the Host mode in the time synchronization settings on https://vcenter-appliance:5480. This will configure the host to take the time from the physical host. Make sure to have a correct date/time config on the host.

vcenter
Click on Time –> Edit (Time synchronization) to edit the Mode to Host.
Failed to set the time via NTP Details - vCenter upgrade failed 6x Step-By-Step 2
Change the Mode to Host

As you can see, we can now successfully complete stage 2 of the upgrade process.

Failed to set the time via NTP Details - vCenter upgrade failed 6x Step-By-Step 3

Resolution

After the upgrade, we tried to configure the same NTP server without any firewall changes. The vCenter reported back that the NTP server was not reachable.

Failed to set the time via NTP Details - vCenter upgrade failed 6x Step-By-Step 4

Our network engineer have changed the firewall rule, to allow port 123 (NTP) on the VMware network block. The Edit Time Synchronization settings could be set without any downtime. More information about configuring NTP in the vCenter appliance can be found over here.