Measuring the Infrastructure Efficiency of Virtual Environments
Top 20 VMware Performance
Metrics You Should Care About
Latest Entries »
- ESXi 5.x host gets disconnected from vCenter Server
- Cannot connect the ESXi 5.x host to vCenter Server
- Cannot connect to ESXi 5.x host using the vSphere Client
- The hostd.log file indicates that the host is out of space:
VmkCtl Locking (/etc/vmware/esx.conf) : Unable to create or open a LOCK file. Failed with reason: No space left on device
VmkCtl Locking (/etc/vmware/esx.conf) : Unable to create or open a LOCK file. Failed with reason: No space left on device - Running the vdf -h command shows that there is sufficient space in the root file system
- The vmkernel.log file indicates that it is out of inodes:
cpu4:1969403)WARNING: VisorFSObj: 893: Cannot create file /etc/vmware/esx.conf.LOCK for process python because the visorfs inode table is full.
cpu11:1968837)WARNING: VisorFSObj: 893: Cannot create file /etc/vmware/esx.conf.LOCK for process hostd because the visorfs inode table is full.
cpu5:1969403)WARNING: VisorFSObj: 893: Cannot create file /etc/vmware/esx.conf.LOCK for process python because the visorfs inode table is full.
cpu11:1968837)WARNING: VisorFSObj: 893: Cannot create file /etc/vmware/esx.conf.LOCK for process hostd because the visorfs inode table is full. - The output of stat -f / indicates that inodes are low
Cause
This issue is caused by the hardware monitoring service (sfcdb) that populates the /var/run/sfcb directory with over 5000 files.
Resolution
To resolve this issue, you must free up inodes.
To free up inodes:
- Connect to the ESXi host using SSH.
- Run this command to stop the sfcbd service:
/etc/init.d/sfcbd-watchdog stop
- Manually delete the files in /var/run/sfcb directory to free inodes.
To remove these files, run this command:
# cd /var/run/sfcb
# rm [0-2]*
# rm [3-6]*
# rm [7-9]*
# rm [a-c]*
# rm [d-f]* - Run this command to restart the sfcbd service:
/etc/init.d/sfcbd-watchdog start
Source: VMware KB
This a GUI that allows you to Backup ESXi 5.0 or ESXi 5.1 hosts to a destination of your choice on a local drive. It also allows you to restore ESXi Host configuration bundle backups taken, to other Hosts.
The main improvement introduces a function in the script which backs the GUI to check that ESX hosts are valid before attempting to backup or restore these.
Very cool tool and useful:
If you experience problems with mouse actions or enhanced graphics, there might be a VMware Tools issue in cause. There is simple fix by repairing VMware Tools installation. You can do a VMware Tools repair from the user interface of many VMware products, not only vSphere.
The thing is that you can’t invoke the “repair” option from the Add/remove windows programs if you’re working directly with your VM through vSphere Client or in WMware Workstation/player. From withing the virtual machine you can only uninstall the tools.
That’s certainly an option too but you would have to uninstall the package > reboot > re-install the package > and reboot again….. to restore the functionality.
The second way is the way you should go. It’s the one that you know, I’m sure, anyway….. You’ll be using the vSphere client, vSphere web client or VMware Workstation’s/player’s menu commands for that.
VMware Tools repair option
You should mount the appropriate ISO package, depending of which version of Windows you’re VM you’re working with. It depends of the OS template that you’re using (W2000, W2003.. etc…)
In vsphere Client:
Inventory > Virtual Machine > Guest > Install/upgrade VMware tools.
From there, if autorun is disabled, go and launch the VMware tools Installation wizard d:/setup.exe > next > and use the Repair option. This option will fix the registry settings, repair the files, if broken, on the components which are already installed.
In VMware Workstation:
VM > Install/upgrade VMware Tools.
In VMware Player :
Virtual Machine > Install/Upgrade VMware Tools
If you want, you can also use the Modify option, and specify additional components to install.
More info at vladan.fr
vCenter server 5.x running on Windows system:
Port 80 – vCenter Server requires port 80 for direct HTTP connections. Then the port 80 redirects requests to HTTPS port 443. Of course you can directly access through the HTTPs, but this redirection can be helpfull if accessing in clear through http only.
Side Note: Make sure that you don’t conflict with IIS or other webservers, which might also use port 80. You can use NETSTAT with the “–abo” switch to determine which ports are used. Or you can also use Process Explorer tool provided by Windows Sysinternals http://technet.microsoft.com/en-us/sysinternals/bb896653
Port 389 – This is the LDAP port number for the Directory Services (DS) for the vCenter Server group. If there is another service, you might be wanting to change that. The LDAP service can run different ports. Here is the range: 1025 – 65535. The vCenter Server system needs to bind to port 389, even if you are not joining this vCenter Server instance to a Linked Mode group.
Port 443 – This is the default port where vCenter Server listens. If not already, open the firewall port. This port (443) is also used for monitoring the data tranfer from SDK clients. You might want to use other port for https. In this case use https://<ip-of-your-vcenter-server>:port-number. Example : https://10.10.7.23:444
Port 636 – This port is used in case you’re using vCenter Server in Linked Mode, where this is the SSL port of the local instance. In case you’re conflicting with another port, you might want to change the other port to different one. If another service is running on this port, it might be preferable to remove it or change its port to a different port.
Port 902 – This is the default vCenter server port. vCenter server uses this port to send data to managed hosts, which also uses this port to send heartbeat (through UDP).
Port 903 – Used for diplay the VMs console. It must be opened between the vSphere Client and the hosts.
Port 8080 – This port is used for the VMware VirtualCenter Management Web Services (windows service).
Port 8443 – This port is used for the VMware VirtualCenter Management Web Services.(on SSL).
Port 60099 – Web Service change service notification port
Port 6501 – Auto Deploy Service
Port 6502 – Auto Deploy management
Port 7444 – vCenter Single Sign On HTTPS
Port 7005 – vCenter Single Sign On Base shutdown port
Port 7080 – vCenter Single Sign On HTTP port
Port 7009 – vCenter Single Sign On AJP port. For more information, see the documentation page onConfiguring VMware Tomcat Server Settings in vCenter Server 5.1
Port 9443 – vSphere Web Client HTTPS
Port 9090 – vSphere Web Client HTTP
Port 10080 – vCenter Inventory Service HTTP
Port 10443 – vCenter Inventory Service HTTPS
Port 10111 – vCenter Inventory Service Management
Port 10109 – vCenter Inventory Service Linked Mode Communication
More information at kb.vmware
The original ESXi system provided by VMware is made up of multiple software packages (currently 60 for ESXi 5.0 and 65 for ESXi 5.1) called VIBs (VMware Installation Bundles). There is e.g. one large VIB for the base system (esx-base), one for the VMware Tools (tools-light), and most of the remaining VIBs are hardware device drivers (e.g. net-e1000 or sata-ahci).
With an ESXi patch one or multiple of these VIBs are updated (or in very rare cases added). An example: The latest patch for ESXi 5.0 (ESXi-5.0.0-201209001 of Sep 2012) updates the VIBsesx-base, tools-light and misc-drivers.
ESXi patches are provided by VMware in the format of so-called Offline Bundles (in this case also called Patch Bundles) in ZIP format.
Now the most important point to understand is that these Patch Bundles do not only include the patched/updated VIBs, but all VIBs that make up an ESXi system, and in fact the latest version of them all.
Read more about this at v-front.de
In most cases you will add VMFS volumes (LUNs) to your vSphere environment, because most of the time your environment will grow. Adding a LUN is very simple: Just configure the masking correctly on your SAN, carry out a rescan on your ESXi server (or the entire cluster) and you have your LUN/volume available.
But what about removing a LUN from your environment. This is a bit more tricky, you have to think about:
- No virtual machines are using the LUN you want to remove. The LUN should not be used as a RDM;
- The LUN should not be a part of a datastorecluster and storage DRS should not be active on the LUN;
- Storage IO Control should be disabled for the LUN;
- The LUNs should not be used as a HA heartbeat datastore;
- And the LUN should not be used as persistent scratch partition.
On top of that, you cannot just remove the LUN, even when it’s not used anymore by virtual machines. It’s very important to first unmount the LUN and then detach the LUN from your ESXi server.
VMware described a clear procedure in the following two KB articles:
- Removing a LUN containing a datastore from VMware ESXi/ESX 4.x – KB 1029786
- Unpresenting a LUN in ESXi 5: KB 2004605
When a virtual machine is provisioned to the datastore cluster, Storage DRS algorithm runs to determine the best placement of the virtual machine. The interesting part of this process is the method Storage DRS determines the free space of a datastore or to be more precise the improvement made in vSphere 5.1 regarding free space calculation and the method of finding the optimal destination datastore.
vSphere 5.0 Storage DRS behavior
Storage DRS is designed to balance the utilization of the datastore cluster, it selects the datastore with the highest free space value to balance the space utilization of the datastores in the datastore cluster and avoids out-of-space situations.
During the deployment of a virtual machine, Storage DRS initiates a simulation to generate an initial placement operation. This process is an isolated process and retrieves the current datastore free space values. However, when a virtual machine is deployed, the space usage of the datastore is updated once the virtual machine deployment is completed and the virtual machine is ready to power-on. This means that the initial placement process is unaware of any ongoing initial placement recommendations and pending storage space allocations. Let’s use an example that explains this behavior.
More information about this article in frankdenneman.nl
- Download vCenter 5.1.0a ISO and mount it inside vCenter. If you splitted the components on several servers, use the ISO on the corrisponding server
- first of all, SSO. Open a command line, move into the folder “Single Sign On” of the DVD, and run:
1VMware-SSO-Server.exe/S/v" /L*v \"%temp%\vim-sso-msi.log\" /qn"You can then open %temp% folder in Windows Explorer and monitor the automatic upgrade from here. You will find the log file and a temporary sub-directory. When that directory will disappear, the installation will be completed. Check the log to be sure the upgrade was completed successfully.
- After SSO, proceed with Inventory Service. You can run the installer from the Autorun interface of the DVD. It will find out you already have Inventory Service in place and offer you to upgrade. Obviously choose YES, and wait for the upgrade to finish. There will be no interactive screen to deal with.
- Third component, Web Client. I usually like to install it even before vCenter, since it’s the only graphical interface to SSO, so if something is not working you could use it to manage SSO. As before, it will automatically discover the previous version and will upgrade it.
- Fourth, vCenter Server. Even here, it will automatically discover the previous version and will upgrade it
- Last one, Update Manager, if you have installed it.
kb4you - A Virtualization World 