VCSA does not boot due to file system errors
Due to a power failure of the storage where the vCenter Server Appliance resides, the VCSA does not boot. Connecting to the console shows the following output:
When you see this screen, none of the services are started as the appliance does not fully start. This implies that there is no means of connecting to the H5 client nor the VAMI interface on port 5480.
Why does the VCSA not boot and where do I start troubleshooting?
There are two important things to mentioned on the screenshot above, this is where we start:
- Failed to start File System Check on /dev/log_vg/log
- journalctl -xb
First we take a look at ‘journalctl -xb’. To do this we need to supply the root password and launch the BASH:
Now that have shell access we can take a look at ‘journalctl -xb’:
1 | journalctl -xb |
Type G to go to the bottom of the log file:
1 | G |
journalctl -xb
Work upwards, the most relevant logs will be at the bottom. For the sake of this blog post, I have type -S. This will turn on/off word wrap, in this case, I turned on word wrap.
File System Check
Going up a little I find these entries:
There is a problem with a certain inode and File System Check (fsck) should be run. Let’s see how we can do that. Is it as simple as running:
1 | fsck /dev/mapper/log_vg-log |
It seems like it. Running the above command finds some errors and suggests to repair. I confirmed.
Other volumes
Let’s check the other logical volumes (lvm). First we will run ‘lsblk’ to take a look at the drive layout:
1 | lsblk |