Enabling Virtual Machine Interface (VMI) in a Linux kernel and in ESX 3.5

Details

VMware ESX 3.5 includes support for operating system kernels that use Virtual Machine Interface (VMI). VMI is the VMware open paravirtualization interface and is included in Linux kernel version 2.6.22 and later. To use VMI, it must be enabled in your Linux kernel and for the ESX 3.5 virtual machine in which that kernel is running.
 
Note: This is applicable only with Linux 32bit kernels.

Solution

Enabling VMI in a Linux Kernel
 
To enable VMI in your custom Linux kernel, enable the CONFIG_PARAVIRT and CONFIG_VMI options in the kernel config file, as illustrated in Figure 1.
 
Figure 1: Screen Shot Showing Kernel Configuration Options


 
A kernel compiled with VMI support can be run on native hardware or on a virtual machine. On native hardware or on a virtual machine where VMI is not enabled, the kernel functions like a regular kernel. On a virtual machine with VMI enabled, the kernel offers improved performance through its paravirtualization interface.
 
Enabling VMI in VMware ESX 3.5
 
In ESX 3.5, each virtual machine can have VMI either enabled or disabled. 
 
To enable VMI for a particular virtual machine, click the Summary tab for that virtual machine, click Edit Settings, select the Options tab, click Paravirtualization, and make sure Support VMI Paravirtualization is checked, as illustrated in Figure 2.

Figure 2: VMI Paravirtualization Option in VMware ESX 3.5
 


When VMI is enabled in the virtual machine settings, the lspci output in the guest operating system includes a new PCI device (identified as a Memory controller), as illustrated in Figure 3.

Figure 3: VMI PCI Device (identified as a Memory controller) in lspci Output


Note: The VMI PCI device, like other virtual PCI devices, takes up a virtual PCI slot.

The presence of the VMI device in the lspci output, however, does not confirm that the guest operating system is running in VMI mode. The dmesg output shown in Figure 4 confirms VMI mode, indicating that VMI is enabled in the virtual machine settings as well as in the Linux kernel.

Figure 4: Output From dmesg Confirming VMI Mode
 


Note: VMI-enabled kernels are included in some Linux distributions. Check with your Linux vendor to see if they have a VMI-enabled kernel available.

Timekeeping best practices for Linux guests

Timekeeping best practices for Linux guests

Purpose

This article presents best practices for Linux timekeeping. These recommendations include specifics on the particular kernel command line options to use for the Linux operating system of interest. There is also a description of the recommended settings and usage for NTP time sync, configuration of VMware Tools time synchronization, and Virtual Hardware Clock configuration, to achieve best timekeeping results.

Resolution

Linux Timekeeping Best Practice

Notes:
  • When both SMP and UP kernels are available, they must be applied to the appropriate SMP and UP virtual machines, otherwise a mismatch may cause time to drift.
  • The recommended kernel parameters must be added to the kernel parameters already configured by the distribution.

32bit Kernels

Linux Flavor Version Kernel Parameters Notes
RHEL



RHEL 5.4
No additional kernel parameters required.
See the note below about the optional use of divider=10.

RHEL 5.3 divider=10 clocksource=acpi_pm

RHEL 5.2 divider=10 clocksource=acpi_pm

RHEL 5.1 divider=10 clocksource=acpi_pm

RHEL 5.0 clocksource=acpi_pm

RHEL 4.8 clock=pmtmr divider=10

RHEL 4.7 clock=pmtmr divider=10

RHEL 4.6 clock=pmtmr

RHEL 4.5 clock=pmtmr

RHEL 4.4 clock=pmtmr

RHEL 4.3 clock=pmtmr

RHEL 4.2 clock=pmtmr

RHEL 4.1 clock=pmtmr

RHEL 4.0 clock=pmtmr

RHEL 3 (All updates)
No additional kernel parameters required.

RHEL 2.1
No additional kernel parameters required.
Redhat



Redhat 9.0
No additional kernel parameters required.

Redhat 8.0
No additional kernel parameters required.

Redhat 7.3
No additional kernel parameters required.

Redhat 7.2
No additional kernel parameters required.

Redhat 7.1
No additional kernel parameters required.

Redhat 7.0
No additional kernel parameters required.

Redhat 6.2
No additional kernel parameters required. 
SLES



SLES 11
No additional kernel parameters required.

SLES 10 SP2 on ESX 3.0.x and earlier clock=pmtmr

SLES 10 SP2 on ESX 3.5 and later
Use a VMI enabled kernel.

SLES 10 SP1 clock=pmtmr

SLES 10 clock=pmtmr or clock=hpet* * AMD Rev F Opteron processors

SLES 9 (All updates) clock=pmtmr

SLES 8
No additional kernel parameters required.

SLES 7
No additional kernel parameters required.
SLED



SLED 11
No additional kernel parameters required.

SLED 10 clock=pmtmr

SLED 9 clock=pmtmr
Suse Linux



Suse Linux 10.3 clocksource=acpi_pm

Suse Linux 10.2 clocksource=acpi_pm

Suse Linux 10.1 clock=pmtmr

Suse Linux 10 clock=pmtmr

Suse Linux 9.3 clock=pmtmr

Suse Linux 9.2 clock=pmtmr

Suse Linux 9.1 clock=pmtmr

Suse Linux 9.0
Remove desktop kernel command line parameter.

Suse Linux 8.2
No additional kernel parameters required.

Suse Linux 8.1
No additional kernel parameters required.

Suse Linux 8.0
No additional kernel parameters required.

Suse Linux 7.3
No additional kernel parameters required.
Ubuntu



Ubuntu 9.04 with kernel 2.6.28-7.18 or later on ESX 4.0 or later
No additional kernel parameters required.

Ubuntu 9.04 with kernel prior to 2.6.28-7.18 on ESX 4.0 or later
Avoid using if possible. May cause guest operating system to stop responding.

Ubuntu 8.10 with kernel 2.6.27-12.28 or later on ESX 4.0 or later
No additional kernel parameters required.

Ubuntu 8.10 with kernel prior to 2.6.27-12.28 on ESX 4.0 or later
Avoid using if possible. May cause guest operating system to stop responding.

Ubuntu 8.10 clocksource=acpi_pm

Ubuntu 8.04 on ESX 3.5 and later
Use a VMI enabled kernel.

Ubuntu 8.04 on ESX 3.0.x and earlier clocksource=acpi_pm

Ubuntu 7.10 clocksource=acpi_pm

Ubuntu 7.04 clocksource=acpi_pm

Ubuntu 6.10 clock=pmtmr

Ubuntu 6.06 clock=pmtmr

Ubuntu 5.10 clock=pmtmr

Ubuntu 5.04 clock=pmtmr
Mandriva



Mandriva Corporate Desktop 4.0 clock=pmtmr

Mandriva Corporate Server 4 clock=pmtmr

Mandriva Linux 2008 clocksource=acpi_pm

Mandriva Linux 2007 clock=pmtmr

Mandriva Linux 2006 clock=pmtmr

Mandrake 10.1
Does not support pmtmr . Avoid using if possible.

Mandrake 10
Does not support pmtmr . Avoid using if possible.

Mandrake 9.2
No additional kernel parameters required.

Mandrake 9.1
No additional kernel parameters required.

Mandrake 9.0
No additional kernel parameters required.

Mandrake 8.2
No additional kernel parameters required.

Mandrake 8.1
No additional kernel parameters required.

Mandrake 8.0
No additional kernel parameters required.
Turbolinux



Turbolinux 10 Desktop
Does not support pmtmr. Avoid using if possible.

Turbolinux 10 Server clock=pmtmr

Turbolinux Enterprise 8
No additional kernel parameters required.

Turbolinux Workstation 8
No additional kernel parameters required.

Turbolinux 7.0
No additional kernel parameters required.
Asianux



Asianux 3.0 SP1
No additional kernel parameters required.
Use kernel 2.6.18-53.17AXS3 or later.

Asianux 3.0 divider=10 clocksource=acpi_pm
CentOS



CentOS 5.4
No additional kernel parameters required.
See the note below about the optional use of divider=10.

CentOS 5.3 divider=10 clocksource=acpi_pm

CentOS 5.2 divider=10 clocksource=acpi_pm

CentOS 5.1 divider=10 clocksource=acpi_pm

CentOS 5.0 clocksource=acpi_pm

CentOS 4.8 clock=pmtmr divider=10

CentOS 4.7 clock=pmtmr divider=10

CentOS 4.6 clock=pmtmr

CentOS 4.5 clock=pmtmr

CentOS 4.4 clock=pmtmr

CentOS 4.3 clock=pmtmr

CentOS 4.2 clock=pmtmr

CentOS 4.1 clock=pmtmr

CentOS 4.0 clock=pmtmr
Oracle



OEL 5.4
No additional kernel parameters required.
See the note below about the optional use of divider=10.

OEL 5.3 divider=10 clocksource=acpi_pm
 
OEL 5.2 divider=10 clocksource=acpi_pm

OEL 5.1 divider=10 clocksource=acpi_pm

OEL 5.0 clocksource=acpi_pm

OEL 4.8 clock=pmtmr divider=10

OEL 4.7 clock=pmtmr divider=10

OEL 4.6 clock=pmtmr

OEL 4.5 clock=pmtmr
Debian



Debian 4.x divider=10 clocksource=acpi_pm

64bit Kernels

Linux Flavor
Version
Kernel Parameters
Notes
RHEL
 
 
 

RHEL 5.4
No additional kernel parameters required.
See the note below about the optional use of divider=10.

RHEL 5.3 notsc divider=10
 
RHEL 5.2
notsc divider=10
 
 
RHEL 5.1 with RHSA-2007:0993-13
notsc divider=10
 
 
RHEL 5.1 without RHSA-2007:0993-13
notsc

 
RHEL 5.0
 
No additional kernel parameters required.  

RHEL 4.8 notsc divider=10
 
RHEL 4.7
notsc divider=10
 
 
RHEL 4.6
notsc
 
 
RHEL 4.5
notsc
 
 
RHEL 4.4
notsc
 
 
RHEL 4.3
notsc
 
 
RHEL 4.2
notsc
 
 
RHEL 4.1
 
Does not support notsc . Avoid using if possible.
 
RHEL 4.0
 
Does not support notsc . Avoid using if possible.
 
RHEL 3 Update 9 with RHSA-2008-0973
disable_lost_ticks 
 

RHEL 3 through Update 8
Has no workaround for lost tick overcompensation. Avoid using if possible.
SLES
 
 
 

SLES 11
No additional kernel parameters required.  
 
SLES 10 SP2 on ESX 3.5 and later
 
No additional kernel parameters required.

SLES 10 SP2 ESX 3.0.x and earlier
notsc

 
SLES 10 SP1
notsc
 
 
SLES 10
notsc
 
 
SLES 9 with kernel version 2.6.5-7.312 or later
ignore_lost_ticks
 
 
SLES 9 with kernel version 2.6.5-7.311 or earlier
 
Has no workaround for lost tick overcompensation. Avoid using if possible.
SLED



SLED 11
No additional kernel parameters required.  

SLED 10 clock=pmtmr
Suse Linux



Suse Linux 10.3 clocksource=acpi_pm

Suse Linux 10.2
notsc


Suse Linux 10.1
notsc


Suse Linux 10
notsc


Suse Linux 9.3
notsc


Suse Linux 9.2
Does not support notsc. Avoid using if possible.

Suse Linux 9.1
Does not support notsc. Avoid using if possible.
Ubuntu



Ubuntu 9.04 with kernel 2.6.28-7.18 or later on ESX 4.0 or later
No additional kernel parameters required.

Ubuntu 9.04 with kernel prior to 2.6.28-7.18 on ESX 4.0 or later
Avoid using if possible. May cause guest operating system to stop responding.

Ubuntu 8.10 with kernel 2.6.27-12.28 or later on ESX 4.0 or later
No additional kernel parameters required.

Ubuntu 8.10 with kernel prior to 2.6.27-12.28 on ESX 4.0 or later
Avoid using if possible. May cause guest operating system to stop responding.

Ubuntu 8.10 clocksource=acpi_pm

Ubuntu 8.04 with kernel 2.6.24-24.52 or later on ESX 4.0 or later
No additional kernel parameters required.

Ubuntu 8.04 with kernel prior to 2.6.24-24.52 on ESX 4.0 or later
Avoid using if possible. May cause guest operating system to stop responding.

Ubuntu 8.04 clocksource=acpi_pm

Ubuntu 7.10 clocksource=acpi_pm

Ubuntu 7.04
 
No additional kernel parameters required.  

Ubuntu 6.10
notsc


Ubuntu 6.06
notsc


Ubuntu 5.10
notsc


Ubuntu 5.04
Does not support notsc. Avoid using if possible.
Mandriva



Mandriva Corporate Desktop 4.0
notsc

 
Mandriva Corporate Server 4
notsc


Mandriva Linux 2008 clocksource=acpi_pm

Mandriva Linux 2007
notsc


Mandriva Linux 2006
notsc


Mandrake Linux 10.1
Does not support notsc. Avoid using if possible.
Turbolinux



Turbolinux 10 Desktop
Does not support notsc. Avoid using if possible.

Turbolinux 10 Server
Does not support notsc. Avoid using if possible.
Asianux



Asianux 3.0 SP1
 
No additional kernel parameters required.
Use kernel 2.6.18-53.17AXS3 or later.

Asianux 3.0
notsc

CentOS



CentOS 5.4
No additional kernel parameters required.
See the note below about the optional use of divider=10.

CentOS 5.3 notsc divider=10

CentOS 5.2
notsc divider=10


CentOS 5.1
notsc


CentOS 5.0
 
No additional kernel parameters required.  

CentOS 4.8 notsc divider=10

CentOS 4.7 notsc divider=10

CentOS 4.6 notsc

CentOS 4.5 notsc

CentOS 4.4 notsc

CentOS 4.3 notsc

CentOS 4.2 notsc

CentOS 4.1
Does not support notsc. Avoid using if possible.

CentOS 4.0
Does not support notsc. Avoid using if possible.
Oracle



OEL 5.4
No additional kernel parameters required.
See the note below about the optional use of divider=10.

OEL 5.3 notsc divider=10
 
OEL 5.2
notsc divider=10


OEL 5.1
notsc


OEL 5.0
 
No additional kernel parameters required.  

OEL 4.8
notsc divider=10


OEL 4.7
notsc divider=10


OEL 4.6
notsc


OEL 4.5
notsc

Debian



Debian 4.x notsc
 

Recommended Configurations

The following configurations are expected to have the best timekeeping behavior:
  • RHEL 5.4 32bit or 64bit running on ESX 3.5 or later
  • SLES 10 SP2 32bit or 64bit running on ESX 3.5 or later
  • SLES 11 32bit or 64bit running on ESX 4.0 or later
  • Ubuntu 8.04 32bit running on ESX 3.5 or later
  • Ubuntu 8.04, 8.10, or 9.04, 32bit or 64bit running on ESX 4.0 or later
Among different version of RHEL 5 and RHEL 4, RHEL 5.4 or later has the best timekeeping behavior.

Editing Kernel Configuration

Kernel command line parameters are specified in the /etc/lilo.conf or /boot/grub/grub.conf file, depending on your choice of boot loader.
For LILO, put the kernel command line parameters at the end of the "append" line. For example, if the append line looks like:
append="resume=/dev/hda6 splash=silent"
and you want to add "clock=pmtmr divider=10" the updated text is:
append="resume=/dev/hda6 splash=silent clock=pmtmr divider=10"
Remember to run /sbin/lilo after editing lilo.conf , so that your edits take effect.
For GRUB, put the kernel command line parameters at the end of the "kernel" line. For example if the kernel line looks like:
kernel /vmlinuz-2.6.18 ro root=/dev/hda2
and you want to add "clock=pmtmr divider=10" the updated text is:
kernel /vmlinuz-2.6.18 ro root=/dev/hda2 clock=pmtmr divider=10
For additional information about working with boot loaders, see your Linux distribution's documentation.

NTP Recommendations

Note: In all cases use NTP instead of VMware Tools periodic time synchronization. Also, you may need to open the firewall (UDP 123) to allow NTP traffic.
 
The following is a sample /etc/ntp.conf :
 
tinker panic 0
restrict 127.0.0.1
restrict default kod nomodify notrap
server 0.vmware.pool.ntp.org
server 1.vmware.pool.ntp.org
server 2.vmware.pool.ntp.org
driftfile /var/lib/ntp/drift
 
The following is a sample /etc/ntp/step-tickers :
 
0.vmware.pool.ntp.org
1.vmware.pool.ntp.org
 
The configuration directive tinker panic 0 instructs NTP not to give up if it sees a large jump in time. This is important for coping with large time drifts and also resuming virtual machines from their suspended state.
 
Note: The directive tinker panic 0 must be at the top of the ntp.conf file.
 
It is also important not to use the local clock as a time source, often referred to as the Undisciplined Local Clock. NTP has a tendency to fall back to this in preference to the remote servers when there is a large amount of time drift.
 
An example of such a configuration is:
 
server 127.127.1.0
fudge 127.127.1.0 stratum 10
 
Comment out both lines.
 
After making changes to NTP configuration, the NTP daemon must be restarted. Refer to your operating system vendor’s documentation.

VMware Tools time synchronization configuration

When using NTP in the guest, disable VMware Tools periodic time synchronization.
 
To disable VMware Tools periodic time sync you use one of the following options:  
  1. Set tools.syncTime = "False" in the configuration file (.vmx file) of the virtual machine.

    OR

  2. Deselect Time synchronization between the virtual machine and the host operating system in the VMware Tools toolbox GUI of the guest operating system.

    OR

  3. Run the vmware-guestd --cmd "vmx.set_option synctime 1 0" command in the guest operating system.
These options do not disable one-time synchronizations done by VMware Tools for events such as tools startup, taking a snapshot, resuming from a snapshot, resuming from suspend, or VMotion. These events synchronize time in the guest operating system with time in the host operating system, so it is important to make sure that the host operating system's time is correct.
 
To do this for VMware ACE, VMware Fusion, VMware GSX Server, VMware Player, VMware Server, and VMware Workstation run time synchronization software such as NTP or w32time in the host. For VMware ESX run NTP in the service console. For VMware ESXi, run NTP in the VMkernel.

Virtual Hardware clock configuration

When configuring the Linux guest operating system, if you are given a choice between keeping the “hardware” clock (that is, the virtual CMOS time of day clock) in UTC or local time, choose UTC. This avoids any confusion when your local time changes between standard and daylight saving time (in England, "summer time").
 
For additional information, see Timekeeping in VMware Virtual Machines.

Note on RHEL 5.4 or CentOS and divider=10

For RHEL 5.4 and CentOS 5.4 and later, divider=10 is a supported kernel configuration option, but is no longer necessary for accurate timekeeping. Using it reduces the frequency of timer interrupts by 10x, which reduces the CPU overhead of processing timer interrupts. This overhead is especially noticeable for idle virtual machines. The only drawback of using divider=10 is that the granularity of wakeups provided by the kernel changes from 1ms to 10ms. The vast majority of applications are not affected by this, but using divider=10 may not be the right tradeoff for some time sensitive applications. For RHEL 5.3 and earlier, divider=10 greatly improves timekeeping accuracy and is strongly recommended.