• ESXi 5.x host gets disconnected from vCenter Server
  • Cannot connect the ESXi 5.x host to vCenter Server
  • Cannot connect to ESXi 5.x host using the vSphere Client 
  • The hostd.log file indicates that the host is out of space:

    VmkCtl Locking (/etc/vmware/esx.conf) : Unable to create or open a LOCK file. Failed with reason: No space left on device
    VmkCtl Locking (/etc/vmware/esx.conf) : Unable to create or open a LOCK file. Failed with reason: No space left on device

  • Running the vdf -h command shows that there is sufficient space in the root file system
  • The vmkernel.log file indicates that it is out of inodes:

    cpu4:1969403)WARNING: VisorFSObj: 893: Cannot create file /etc/vmware/esx.conf.LOCK for process python because the visorfs inode table is full.
    cpu11:1968837)WARNING: VisorFSObj: 893: Cannot create file /etc/vmware/esx.conf.LOCK for process hostd because the visorfs inode table is full.
    cpu5:1969403)WARNING: VisorFSObj: 893: Cannot create file /etc/vmware/esx.conf.LOCK for process python because the visorfs inode table is full.
    cpu11:1968837)WARNING: VisorFSObj: 893: Cannot create file /etc/vmware/esx.conf.LOCK for process hostd because the visorfs inode table is full.

  • The output of stat -f / indicates that inodes are low

Cause

This issue is caused by the hardware monitoring service (sfcdb) that populates the /var/run/sfcb directory with over 5000 files.

 

Resolution

To resolve this issue, you must free up inodes.
 
To free up inodes:
  1. Connect to the ESXi host using SSH.
  2. Run this command to stop the sfcbd service:

    /etc/init.d/sfcbd-watchdog stop

  3. Manually delete the files in /var/run/sfcb directory to free inodes.

    To remove these files, run this command:

    # cd /var/run/sfcb
    # rm [0-2]*
    # rm [3-6]*
    # rm [7-9]*
    # rm [a-c]*
    # rm [d-f]*

  4. Run this command to restart the sfcbd service:

    /etc/init.d/sfcbd-watchdog start

 

Source: VMware KB

Advertisements