# Xen Hypervisor Command Line Options This document covers the command line options which the Xen Hypervisor. ## Types of parameter Most parameters take the form `option=value`. Different options on the command line should be space delimited. All options are case sensitive, as are all values unless explicitly noted. ### Boolean (``) All boolean option may be explicitly enabled using a `value` of > `yes`, `on`, `true`, `enable` or `1` They may be explicitly disabled using a `value` of > `no`, `off`, `false`, `disable` or `0` In addition, a boolean option may be enabled by simply stating its name, and may be disabled by prefixing its name with `no-`. ####Examples Enable noreboot mode > `noreboot=true` Disable x2apic support (if present) > `x2apic=off` Enable synchronous console mode > `sync_console` Explicitly specifying any value other than those listed above is undefined, as is stacking a `no-` prefix with an explicit value. ### Integer (``) An integer parameter will default to decimal and may be prefixed with a `-` for negative numbers. Alternatively, a hexadecimal number may be used by prefixing the number with `0x`, or an octal number may be used if a leading `0` is present. Providing a string which does not validly convert to an integer is undefined. ### Size (``) A size parameter may be any integer, with a size suffix * `G` or `g`: GiB (2^30) * `M` or `m`: MiB (2^20) * `K` or `k`: KiB (2^10) * `B` or `b`: Bytes Without a size suffix, the default will be kilo. Providing a suffix other than those listed above is undefined. ### String Many parameters are more complicated and require more intricate configuration. The detailed description of each individual parameter specify which values are valid. ### List Some options take a comma separated list of values. ### Combination Some parameters act as combinations of the above, most commonly a mix of Boolean and String. These are noted in the relevant sections. ## Parameter details ### acpi > `= force | ht | noirq | ` **String**, or **Boolean** to disable. The **acpi** option is used to control a set of four related boolean flags; `acpi_force`, `acpi_ht`, `acpi_noirq` and `acpi_disabled`. By default, Xen will scan the DMI data and blacklist certain systems which are known to have broken ACPI setups. Providing `acpi=force` will cause Xen to ignore the blacklist and attempt to use all ACPI features. Using `acpi=ht` causes Xen to parse the ACPI tables enough to enumerate all CPUs, but will not use other ACPI features. This is not common, and only has an effect if your system is blacklisted. The `acpi=noirq` option causes Xen to not parse the ACPI MADT table looking for IO-APIC entries. This is also not common, and any system which requires this option to function should be blacklisted. Additionally, this will not prevent Xen from finding IO-APIC entries from the MP tables. Finally, any of the boolean false options can be used to disable ACPI usage entirely. Because responsibility for ACPI processing is shared between Xen and the domain 0 kernel this option is automatically propagated to the domain 0 command line ### acpi\_apic\_instance > `= ` Specify which ACPI MADT table to parse for APIC information, if more than one is present. ### acpi\_pstate\_strict > `= ` ### acpi\_skip\_timer\_override > `= ` Instruct Xen to ignore timer-interrupt override. Because responsibility for ACPI processing is shared between Xen and the domain 0 kernel this option is automatically propagated to the domain 0 command line ### acpi\_sleep > `= s3_bios | s3_mode` ### allowsuperpage > `= ` > Default: `true` Permit Xen to use superpages when performing memory management. ### apic > `= bigsmp | default` Override Xen's logic for choosing the APIC driver. By default, if there are more than 8 CPUs, Xen will switch to `bigsmp` over `default`. ### allow\_unsafe > `= ` > Default: `false` Force boot on potentially unsafe systems. By default Xen will refuse to boot on systems with the following errata: * AMD Erratum 121. Processors with this erratum are subject to a guest triggerable Denial of Service. Override only if you trust all of your PV guests. ### apic\_verbosity > `= verbose | debug` Increase the verbosity of the APIC code from the default value. ### ats > `= ` > Default: `true` Permits Xen to set up and use PCI Address Translation Services, which is required for PCI Passthrough. ### availmem > `= ` > Default: `0` (no limit) Specify a maximum amount of available memory, to which Xen will clamp the e820 table. ### badpage > `= List of [ | - ]` Specify that certain pages, or certain ranges of pages contain bad bytes and should not be used. For example, if your memory tester says that byte `0x12345678` is bad, you would place `badpage=0x12345` on Xen's command line. ### bootscrub > `= ` > Default: `true` Scrub free RAM during boot. This is a safety feature to prevent accidentally leaking sensitive VM data into other VMs if Xen crashes and reboots. ### cachesize > `= ` If set, override Xen's calculation of the level 2 cache line size. ### clocksource > `= pit | hpet | acpi` If set, override Xen's default choice for the platform timer. ### com1,com2 > `= [/][,[DPS][,[|pci|amt][,[][,[][,[]]]]]]` Both option `com1` and `com2` follow the same format. * `` may be either an integer baud rate, or the string `auto` if the bootloader or other earlier firmware has already set it up. * Optionally, a clock speed measured in hz can be specified. * `DPS` represents the number of data bits, the parity, and the number of stop bits. `D` is an integer between 5 and 8 for the number of data bits. `P` is a single character representing the type of parity: * `n` No * `o` Odd * `e` Even * `m` Mark * `s` Space `S` is an integer 1 or 2 for the number of stop bits. * `` is an integer which specifies the IO base port for UART registers. * `` is the IRQ number to use, or `0` to use the UART in poll mode only. * `` is the PCI location of the UART, in `:.` notation. * `` is the PCI bridge behind which is the UART, in `:.` notation. * `pci` indicates that Xen should scan the PCI bus for the UART, avoiding Intel AMT devices. * `amt` indicated that Xen should scan the PCI bus for the UART, including Intel AMT devices if present. A typical setup for most situations might be `com1=115200,8n1` ### conring\_size > `= ` > Default: `conring_size=16k` Specify the size of the console ring buffer. ### console > `= List of [ vga | com1[H,L] | com2[H,L] | dbgp | none ]` > Default: `console=com1,vga` Specify which console(s) Xen should use. `vga` indicates that Xen should try and use the vga graphics adapter. `com1` and `com2` indicates that Xen should use serial ports 1 and 2 respectively. Optionally, these arguments may be followed by an `H` or `L`. `H` indicates that transmitted characters will have their MSB set, while received characters must have their MSB set. `L` indicates the converse; transmitted and received characters will have their MSB cleared. This allows a single port to be shared by two subsystems (e.g. console and debugger). `dbgp` indicates that Xen should use a USB debug port. `none` indicates that Xen should not use a console. This option only makes sense on its own. ### console\_timestamps > `= ` > Default: `false` Flag to indicate whether include a timestamp with each console line. ### console\_to\_ring > `= ` > Default: `false` Flag to indicate whether all guest console output should be copied into the console ring buffer. ### conswitch > `= [x]` > Default `conswitch=a` Specify which character should be used to switch serial input between Xen and dom0. The required sequence is CTRL-<switch char> three times. The optional trailing `x` indicates that Xen should not automatically switch the console input to dom0 during boot. Any other value, including omission, causes Xen to automatically switch to the dom0 console during dom0 boot. Use `conswitch=ax` to keep the default switch character, but for xen to keep the console. ### cpu\_type > `= arch_perfmon` If set, force use of the performance counters for oprofile, rather than detecting available support. ### cpufreq > `= dom0-kernel | none | xen` > Default: `xen` Indicate where the responsibility for driving power states lies. ### cpuid\_mask\_cpu (AMD only) > `= fam_0f_rev_c | fam_0f_rev_d | fam_0f_rev_e | fam_0f_rev_f | fam_0f_rev_g | fam_10_rev_b | fam_10_rev_c | fam_11_rev_b` If the other **cpuid\_mask\_{,ext\_}e{c,d}x** options are fully set (unspecified on the command line), specify a pre-canned cpuid mask to mask the current processor down to appear as the specified processor. It is important to ensure that all hosts in a pool appear the same to guests to allow successful live migration. ### cpuid\_mask\_ ecx,edx,ext\_ecx,ext\_edx,xsave_eax > `= ` > Default: `~0` (all bits set) These five command line parameters are used to specify cpuid masks to help with cpuid levelling across a pool of hosts. Setting a bit in the mask indicates that the feature should be enabled, while clearing a bit in the mask indicates that the feature should be disabled. It is important to ensure that all hosts in a pool appear the same to guests to allow successful live migration. ### cpuidle > `= ` ### cpuinfo > `= ` ### crashinfo_maxaddr > `= ` > Default: `4G` Specify the maximum address to allocate certain structures, if used in combination with the `low_crashinfo` command line option. ### crashkernel > `= :[,...][@]` ### credit2\_balance\_over > `= ` ### credit2\_balance\_under > `= ` ### credit2\_load\_window\_shift > `= ` ### dbgp > `= ehci[ | @pci:. ]` Specify the USB controller to use, either by instance number (when going over the PCI busses sequentially) or by PCI device (must be on segment 0). ### debug\_stack\_lines > `= ` > Default: `20` Limits the number lines printed in Xen stack traces. ### debugtrace > `= ` > Default: `128` Specify the size of the console debug trace buffer in KiB. The debug trace feature is only enabled in debugging builds of Xen. ### dma\_bits > `= ` Specify the bit width of the DMA heap. ### dom0\_ioports\_disable > `= List of -` Specify a list of IO ports to be excluded from dom0 access. ### dom0\_max\_vcpus Either: > `= `. The number of VCPUs to give to dom0. This number of VCPUs can be more than the number of PCPUs on the host. The default is the number of PCPUs. Or: > `= -` where `` and `` are integers. Gives dom0 a number of VCPUs equal to the number of PCPUs, but always at least `` and no more than ``. Using `` may give more VCPUs than PCPUs. `` or `` may be omitted and the defaults of 1 and unlimited respectively are used instead. For example, with `dom0_max_vcpus=4-8`: Number of PCPUs | Dom0 VCPUs 2 | 4 4 | 4 6 | 6 8 | 8 10 | 8 ### dom0\_mem > `= List of ( min: | max: | )` Set the amount of memory for the initial domain (dom0). If a size is positive, it represents an absolute value. If a size is negative, it is subtracted from the total available memory. * `` specifies the exact amount of memory. * `min:` specifies the minimum amount of memory. * `max:` specifies the maximum amount of memory. If `` is not specified, the default is all the available memory minus some reserve. The reserve is 1/16 of the available memory or 128 MB (whichever is smaller). The amount of memory will be at least the minimum but never more than the maximum (i.e., `max` overrides the `min` option). If there isn't enough memory then as much as possible is allocated. `max:` also sets the maximum reservation (the maximum amount of memory dom0 can balloon up to). If this is omitted then the maximum reservation is unlimited. For example, to set dom0's initial memory allocation to 512MB but allow it to balloon up as far as 1GB use `dom0_mem=512M,max:1G` If you use this option then it is highly recommended that you disable any dom0 autoballooning feature present in your toolstack. See the _xl.conf(5)_ man page or [Xen Best Practices](http://wiki.xen.org/wiki/Xen_Best_Practices#Xen_dom0_dedicated_memory_and_preventing_dom0_memory_ballooning). ### dom0\_shadow > `= ` ### dom0\_vcpus\_pin > `= ` > Default: `false` Pin dom0 vcpus to their respective pcpus ### e820-mtrr-clip > `= ` Flag that specifies if RAM should be clipped to the highest cacheable MTRR. > Default: `true` on Intel CPUs, otherwise `false` ### e820-verbose > `= ` > Default: `false` Flag that enables verbose output when processing e820 information and applying clipping. ### edd (x86) > `= off | on | skipmbr` Control retrieval of Extended Disc Data (EDD) from the BIOS during boot. ### edid (x86) > `= no | force` Either force retrieval of monitor EDID information via VESA DDC, or disable it (edid=no). This option should not normally be required except for debugging purposes. ### extra\_guest\_irqs > `= [][,]` > Default: `32,256` Change the number of PIRQs available for guests. The optional first number is common for all domUs, while the optional second number (preceded by a comma) is for dom0. Changing the setting for domU has no impact on dom0 and vice versa. For example to change dom0 without changing domU, use `extra_guest_irqs=,512` ### flask\_enabled > `= ` ### flask\_enforcing > `= ` ### font > `= ` where height is `8x8 | 8x14 | 8x16 '` Specify the font size when using the VESA console driver. ### gdb > `= [/][,DPS[,[,[,[,]]]] | pci | amt ] ` Specify the serial parameters for the GDB stub. ### gnttab\_max\_nr\_frames > `= ` Specify the maximum number of frames per grant table operation. ### guest\_loglvl > `= [/]` where level is `none | error | warning | info | debug | all` > Default: `guest_loglvl=none/warning` Set the logging level for Xen guests. Any log message with equal more more importance will be printed. The optional `` option instructs which severities should be rate limited. ### hap > `= ` > Default: `true` Flag to globally enable or disable support for Hardware Assisted Paging (HAP) ### hap\_1gb > `= ` > Default: `true` Flag to enable 1 GB host page table support for Hardware Assisted Paging (HAP). ### hap\_2mb > `= ` > Default: `true` Flag to enable 2 MB host page table support for Hardware Assisted Paging (HAP). ### hpetbroadcast > `= ` ### hvm\_debug > `= ` ### hvm\_port80 > `= ` ### highmem-start > `= ` Specify the memory boundary past which memory will be treated as highmem (x86 debug hypervisor only). ### idle\_latency\_factor > `= ` ### ioapic\_ack ### iommu ### iommu\_inclusive\_mapping > `= ` ### irq\_ratelimit > `= ` ### irq\_vector\_map ### ivrs_hpet[``] > `=[:]:.` Force the use of `[:]:.` as device ID of HPET `` instead of the one specified by the IVHD sub-tables of the IVRS ACPI table. ### ivrs_ioapic[``] > `=[:]:.` Force the use of `[:]:.` as device ID of IO-APIC `` instead of the one specified by the IVHD sub-tables of the IVRS ACPI table. ### lapic Force the use of use of the local APIC on a uniprocessor system, even if left disabled by the BIOS. This option will accept any value at all. ### lapic\_timer\_c2\_ok > `= ` ### ler > `= ` ### loglvl > `= [/]` where level is `none | error | warning | info | debug | all` > Default: `loglvl=warning` Set the logging level for Xen. Any log message with equal more more importance will be printed. The optional `` option instructs which severities should be rate limited. ### low\_crashinfo > `= none | min | all` > Default: `none` if not specified at all, or to `min` if **low_crashinfo** is present without qualification. This option is only useful for hosts with a 32bit dom0 kernel, wishing to use kexec functionality in the case of a crash. It represents which data structures should be deliberately allocated in low memory, so the crash kernel may find find them. Should be used in combination with **crashinfo_maxaddr**. ### max\_cstate > `= ` ### max\_gsi\_irqs > `= ` ### maxcpus > `= ` ### mce > `= ` ### mce\_fb > `= ` ### mce\_verbosity > `= verbose` Specify verbose machine check output. ### mem > `= ` Specify the maximum address of physical RAM. Any RAM beyond this limit is ignored by Xen. ### mmcfg > `= [,amd-fam10]` > Default: `1` Specify if the MMConfig space should be enabled. ### msi > `= ` > Default: `true` Force Xen to (not) use PCI-MSI, even if ACPI FADT says otherwise. ### mwait-idle > `= ` > Default: `true` Use the MWAIT idle driver (with model specific C-state knowledge) instead of the ACPI based one. ### nmi > `= ignore | dom0 | fatal` > Default: `nmi=fatal` Specify what Xen should do in the event of an NMI parity or I/O error. `ignore` discards the error; `dom0` causes Xen to report the error to dom0, while 'fatal' causes Xen to print diagnostics and then hang. ### noapic Instruct Xen to ignore any IOAPICs that are present in the system, and instead continue to use the legacy PIC. This is _not_ recommended with pvops type kernels. Because responsibility for APIC setup is shared between Xen and the domain 0 kernel this option is automatically propagated to the domain 0 command line. ### nofxsr > `= ` ### noirqbalance > `= ` Disable software IRQ balancing and affinity. This can be used on systems such as Dell 1850/2850 that have workarounds in hardware for IRQ routing issues. ### nolapic > `= ` > Default: `false` Ignore the local APIC on a uniprocessor system, even if enabled by the BIOS. This option will accept value. ### no-real-mode (x86) > `= ` Do not execute real-mode bootstrap code when booting Xen. This option should not be used except for debugging. It will effectively disable the **vga** option, which relies on real mode to set the video mode. ### noreboot > `= ` Do not automatically reboot after an error. This is useful for catching debug output. Defaults to automatically reboot after 5 seconds. ### noserialnumber > `= ` Disable CPU serial number reporting. ### nosmp > `= ` Disable SMP support. No secondary processors will be booted. Defaults to booting secondary processors. ### nr\_irqs > `= ` ### numa > `= on | off | fake= | noacpi` Default: `on` ### pci-phantom > `=[:]:,` Mark a group of PCI devices as using phantom functions without actually advertising so, so the IOMMU can create translation contexts for them. All numbers specified must be hexadecimal ones. This option can be specified more than once (up to 8 times at present). ### ple\_gap > `= ` ### ple\_window > `= ` ### reboot > `= t[riple] | k[bd] | n[o] [, [w]arm | [c]old]` Default: `0` Specify the host reboot method. `warm` instructs Xen to not set the cold reboot flag. `cold` instructs Xen to set the cold reboot flag. `triple` instructs Xen to reboot the host by causing a triple fault. `kbd` instructs Xen to reboot the host via the keyboard controller. `acpi` instructs Xen to reboot the host using RESET_REG in the ACPI FADT. ### sched > `= credit | credit2 | sedf | arinc653` > Default: `sched=credit` Choose the default scheduler. ### sched\_credit2\_migrate\_resist > `= ` ### sched\_credit\_tslice\_ms > `= ` Set the timeslice of the credit1 scheduler, in milliseconds. The default is 30ms. Reasonable values may include 10, 5, or even 1 for very latency-sensitive workloads. ### sched\_ratelimit\_us > `= ` In order to limit the rate of context switching, set the minimum amount of time that a vcpu can be scheduled for before preempting it, in microseconds. The default is 1000us (1ms). Setting this to 0 disables it altogether. ### sched\_smt\_power\_savings > `= ` Normally Xen will try to maximize performance and cache utilization by spreading out vcpus across as many different divisions as possible (i.e, numa nodes, sockets, cores threads, &c). This often maximizes throughput, but also maximizes energy usage, since it reduces the depth to which a processor can sleep. This option inverts the logic, so that the scheduler in effect tries to keep the vcpus on the smallest amount of silicon possible; i.e., first fill up sibling threads, then sibling cores, then sibling sockets, &c. This will reduce performance somewhat, particularly on systems with hyperthreading enabled, but should reduce power by enabling more sockets and cores to go into deeper sleep states. ### serial\_tx\_buffer > `= ` > Default: `16kB` Set the serial transmit buffer size. ### smep > `= ` > Default: `true` Flag to enable Supervisor Mode Execution Protection ### snb\_igd\_quirk > `= ` ### sync\_console > `= ` > Default: `false` Flag to force synchronous console output. Useful for debugging, but not suitable for production environments due to incurred overhead. ### tboot > `= 0x` Specify the physical address of the trusted boot shared page. ### tbuf\_size > `= ` Specify the per-cpu trace buffer size in pages. ### tdt > `= ` > Default: `true` Flag to enable TSC deadline as the APIC timer mode. ### tevt\_mask > `= ` Specify a mask for Xen event tracing. This allows Xen tracing to be enabled at boot. Refer to the xentrace(8) documentation for a list of valid event mask values. In order to enable tracing, a buffer size (in pages) must also be specified via the tbuf\_size parameter. ### tickle\_one\_idle\_cpu > `= ` ### timer\_slop > `= ` ### tmem > `= ` ### tmem\_compress > `= ` ### tmem\_dedup > `= ` ### tmem\_lock > `= ` ### tmem\_shared\_auth > `= ` ### tmem\_tze > `= ` ### tsc > `= unstable | skewed` ### ucode > `= [ | scan]` Specify how and where to find CPU microcode update blob. 'integer' specifies the CPU microcode update blob module index. When positive, this specifies the n-th module (in the GrUB entry, zero based) to be used for updating CPU micrcode. When negative, counting starts at the end of the modules in the GrUB entry (so with the blob commonly being last, one could specify `ucode=-1`). Note that the value of zero is not valid here (entry zero, i.e. the first module, is always the Dom0 kernel image). Note further that use of this option has an unspecified effect when used with xen.efi (there the concept of modules doesn't exist, and the blob gets specified via the `ucode=` config file/section entry; see [EFI configuration file description](efi.html)). 'scan' instructs the hypervisor to scan the multiboot images for an cpio image that contains microcode. Depending on the platform the blob with the microcode in the cpio name space must be: - on Intel: kernel/x86/microcode/GenuineIntel.bin - on AMD : kernel/x86/microcode/AuthenticAMD.bin ### unrestricted\_guest > `= ` ### vcpu\_migration\_delay > `= ` > Default: `0` Specify a delay, in microseconds, between migrations of a VCPU between PCPUs when using the credit1 scheduler. This prevents rapid fluttering of a VCPU between CPUs, and reduces the implicit overheads such as cache-warming. 1ms (1000) has been measured as a good value. ### vesa-map > `= ` ### vesa-mtrr > `= ` ### vesa-ram > `= ` ### vga > `= ( ask | current | text-80x | gfx-xx | mode- )[,keep]` `ask` causes Xen to display a menu of available modes and request the user to choose one of them. `current` causes Xen to use the graphics adapter in its current state, without further setup. `text-80x` instructs Xen to set up text mode. Valid values for `` are `25, 28, 30, 34, 43, 50, 80` `gfx-xx` instructs Xen to set up graphics mode with the specified width, height and depth. `mode-` instructs Xen to use a specific mode, as shown with the `ask` option. (N.B menu modes are displayed in hex, so `` should be a hexadecimal number) The optional `keep` parameter causes Xen to continue using the vga console even after dom0 has been started. The default behaviour is to relinquish control to dom0. ### vpid (Intel) > `= ` > Default: `true` Use Virtual Processor ID support if available. This prevents the need for TLB flushes on VM entry and exit, increasing performance. ### vpmu > `= ( bts )` > Default: `off` Switch on the virtualized performance monitoring unit for HVM guests. If the current cpu isn't supported a message like 'VPMU: Initialization failed. ...' is printed on the hypervisor serial log. For some Intel Nehalem processors a quirk handling exist for an unknown wrong behaviour (see handle\_pmc\_quirk()). If 'vpmu=bts' is specified the virtualisation of the Branch Trace Store (BTS) feature is switched on on Intel processors supporting this feature. *Warning:* As the BTS virtualisation is not 100% safe and because of the nehalem quirk don't use the vpmu flag on production systems with Intel cpus! ### watchdog > `= ` > Default: `false` Run an NMI watchdog on each processor. If a processor is stuck for longer than the **watchdog\_timeout**, a panic occurs. ### watchdog\_timeout > `= ` > Default: `5` Set the NMI watchdog timeout in seconds. Specifying `0` will turn off the watchdog. ### x2apic > `= ` > Default: `true` Permit use of x2apic setup for SMP environments. ### x2apic\_phys > `= ` > Default: `true` Use the x2apic physical apic driver. The alternative is the x2apic cluster driver. ### xsave > `= ` > Default: `true` Permit use of the `xsave/xrstor` instructions.