xm-test README Copyright (C) International Business Machines Corp., 2005 Author(s): Dan Smith Woody Marvel Overview ======== This suite provides a framework for testing the Xen userspace tools. The directory structure is: ./xm-test | +-/lib: Python support libraries | +-/ramdisk: Staging area for building the test ramdisk | +-/tests | | | +-/create: Tests for the 'xm create' command | +-/destroy: Tests for the 'xm destroy' command | . . . | +-/utils: Utility scripts for ramdisk building Reports are posted here: http://xmtest.dague.org Building ======== Before the test suite can be used, the ramdisk must be built from source. All source needed for this process is automatically downloaded, extracted, and compiled. Due to the need to create special files, this process must be done as root: # ./autogen # ./configure # make NB: If you have the initrd.img from another installation of xm-test, you can copy it into the ramdisk directory to eliminate the need to rebuild it. If you do this, there is no need to run 'make' again. Simply copy the initrd-X.Y-ARCH.img file into ramdisk/ and then run: # make existing Or, you can run: # INITRD="http://url.of.initrd.repo/" make existing You do not need to include the name of the image itself in the url, however, an initrd with the right name (initrd.X.Y-ARCH.img) and version number must exist at that location. The script will determine which version of the initrd it needs and try to download the right file from that location. This will set up the link so that xm-test will use the existing ramdisk. Next, just run "runtest.sh" normally. Note that in general, you should not attempt to use a ramdisk from a previous minor version of xm-test (i.e., don't use a ramdisk from 0.4.0 with 0.5.0. 0.5.0 should work for 0.5.3 though) BUILDING with HVM Support ========================= If you'd like to build and run this with hardware virtual machine assist (HVM) support to test fully virtualized disk images on VMX/SVM hardware, please add the --enable-hvm-support option to configure: # ./autogen # ./configure --enable-hvm-support # make The ramdisk/bin/create_disk_image script, which builds the full virt disk.img, requires Lilo 22.7+ to be installed on the system. Lilo is used to install the bootloader on the disk.img. If HVM support is enabled, the ramdisk/bin/create_disk_image script will be run to create a full virt disk.img in the ramdisk directory. The script, by default, will look in /boot for the first non-Xen kernel it runs across. If you'd like to set xm-test to use a specific kernel, rather than the first one it finds in /boot, you can configure it in with the "--with-hvm-kernel=KERNEL" option: # ./autogen # ./configure --enable-hvm-support --with-hvm-kernel=KERNEL # make Otherwise, you can always rerun the create script using the -k option to use a specific kernel. The disk.img created for HVM testing must contain a pcnet32 driver for network tests. The ramdisk/bin/create_disk_image script will, by default, look in the /lib/modules directory associated with the kernel being used. If you'd like to specify a different location for the driver or want to tell the script that the driver is built into the kernel, please use the "--with-driver-dir=DRVDIR" configure option. If built into the kernel, please use the key word "builtin" with the option: # ./autogen # ./configure --enable-hvm-support --with-driver-dir=builtin - or - # ./configure --enable-hvm-support --with-driver-dir=/driver/directory # make Xm-test will look for disk.img in the ramdisk directory when run by default. BUILDING for ACM Security Testing ================================= A number of tests have been added to test the access control module (ACM) in the Xen hypervisor and the tools for supporting ACM. Those tests are located in the security-acm directory. If ACM support is compiled into Xen (see the user guide for how to do this) those tests can be run with the following command from the xm-test directory ./runtest.sh [...] -g security Some of these tests will work even without support of ACM by Xen. The xm test suite has been extended to support labeling of resources as required by the existing tests. However, by default the test suite is not allowed to automatically label resources since this may affect existing labels. To enable this, the test suite must be configured with the following parameter passed to the configure scripts (in addition to any other desired parameters) ./configure --enable-full-labeling To revoke the privilege at a later time run the configure scripts without this parameter: ./configure If a 'make' has previously been run for building the test suite, it is not necessary to run 'make' again just for enabling or disabling the automatic labeling of resources. Running ======= To run the full test suite, do the following as root: # ./runtest.sh This will run all tests, as well as generate and submit a report at the end. All output files will begin with "." If you wish to prevent submission of a report, add "-d" to the command line like this: # ./runtest.sh -d It may be useful to run tests without submission as above, and then submit the report at a later time. To do so, run runtest.sh with the -s flag and the name of the previously-generated report: # ./runtest.sh -s Group test sets are supported in xm-test. This is form of layering of tests groups/cases/tests. In the framework directory "grouptest", files exist for group processing. The user can add groups, casenames and test lists as required. Default group run is "grouptest/default". # ./runtest.sh -g * NOTE: There is a quick set of tests in group mode, that was added to run certain casenames and tests, and there is a "medium" group, which is a medium-length run (around 20 minutes). Neither is a substitute for the full xm-test test suite. # ./runtest.sh -g quick # ./runtest.sh -g medium It may be desirable to run a specific test group. This can be accomplished by doing the following: # cd tests/create # TEST_VERBOSE=1 make check When developing or debugging a specific feature, a single test can be run to avoid having to run even a whole test group: # cd tests/create # TEST_VERBOSE=1 make check TESTS=01_create_basic_pos.test The runtest.sh script will create several files, including a .report file, which is the cleaned up, email-friendly report of failures. Additionally, the script will submit your results to the development team for trend analysis. This helps us determine the level of success people "out there" are having with different versions of Xen. Note: you should generally run xm-test with a minimum of memory allocated to Dom0. More memory available for allocation to DomUs means a more rigorous test. BIG FAT WARNING: The test framework assumes it is running on a dedicated machine. As such, the library automatically destroys any running DomUs on the system to provide each test with a "clean slate". Testing the XML-RPC and Xen-API interfaces of xend ================================================== The xm-test suite can be used to test xm's interface with xend using either XML-RPC or the Xen-API. In order to use either one of these modes, xm needs to be configured using its configuration file '/etc/xen/xm-config.xml'. Note: The current default configuration after a fresh install of the xen sources currently is to use the XML-RPC interface for communication with xend. Example content for the xm-config.xml for using the Xen-API looks as follows: This configuration makes xm talk to xend using port 9363. For this to work, also xend needs to be configured to listen to port 9363. Therefore The following line must be in /etc/xen/xend-config.sxp. (xen-api-server (( 127.0.0.1:9363 none ))) To communicate via the legacy XML-RPC interface, the file '/etc/xen/xm-config.xml' may simply have the following content or may be complete remove from the /etc/xen directory. A few tests have been written for the xm-test suite that test the Xen-API interface directly without relying on 'xm'. These tests can be found in the grouptest 'xapi' and for them to work properly, xm must have been configured to use the Xen-API following the instructions above. To run these test, the following command line can be invoked: # ./runtest.sh -g xapi Extending ========= Additional tests may be added in existing groups to test additional cases for a given xm subcommand. Test programs should be named according to the following scheme: XY_group_name_{pos,neg}.py Where: XY is the next number in line group is the name of the subcommand being tested name is the short name of the test {pos,neg} denotes whether this is a positive or negative test case New subcommand groups should be added as directories named after the subcommand itself. The "Makefile.am.template" should be copied into the new group directory as "Makefile.am". See the Writing_Tests_HOWTO file for more detailed information on adding tests to the suite. Developer Notes =============== Our library provides a DomU console abstraction for automated execution of commands. Please note that this is relatively fragile, and is intended for use only with the ramdisk built by the framework. Because the console experiences some occasional corruption, this method is not completely perfect at the moment, although the authors use it with relatively few problems. Known Issues ============ If you create a domain with a small amount of memory, under 32MBs, you may run into out of memory situations for the domain. There's no way to know the amount of memory needed by the kernel and modules used. Xm-test uses 64MBs as default and that should work. If there are out of memory issues, the default can be changed. Edit xm-test/lib/XmTestLib/XenDomain.py and change ParavirtDefaults and HVMDefaults "memory". There are two tests that work with small memory, 15_create_smallmem_pos.py and 16_create_smallmem_neg.py. The first makes sure the default 32 MBs limit works. The second checks a low memory fail situation. These tests are located in the xm-test/tests/create directory and can be easily edited to change the MEM value they should test. If the 32MBs test fails, the failure should be reported to the Xen xen-devel mailing list. The Xen tools use 32MBs as a lower acceptable limit for domain creation. The Xen mailing lists are located here: http://lists.xensource.com/ Reporting Bugs ============== If you find a bug in the test framework, report it to: Dan Smith If you find a bug in a specific test case, contact the author of the test case first.