Writing Tests HOWTO
===================

One goal for xm-test is to make test writing very easy. Xm-test includes
a library in lib/XmTestLib that contains all the necessary methods for
creating and shutting down domains. Include the library in your test by
importing it:

  from XmTestLib import *


Guidelines
==========

1. Tests should be short and single purposed. This means testing as
   little as possible in a single test. Do not overload tests. The more
   that's in a test the more difficult it is to track down what failed.

2. Tests should report as much information as possible when calling
   FAIL() or SKIP().

3. A test should report SKIP() only if it cannot be run on the current
   machine or it makes no sense to run it. SMP tests on an UP system, 
   for example, may not make sense. Or, there are some tests for
   para-virtualized guests that won't work on a fully virtualized 
   guest.

4. Use the traceCommand() function to run commands on Domain0, the 
   Xen management domain. This function logs everything, which is useful
   in case of failure.

5. Use the domain's console.runCmd method to run commands on a guest
   domain. This ensures logging and consistency. Please see 'Creating
   and Using Domains' below for an example.

6. Tests need to capture and handle libary exceptions such as:

 - ConsoleError can be raised when sending a command to a console.
 - DomainError can be raised when a domain is started, indicating
   a possible configuration error.
 - TimeoutError can be raised when the traceCommand() method is used.

7. Tests shouldn't depend on where the test is being run from or the
   system on which it is run.

8. Please use the existing tests for a guide, especially:

 - create/01_create_basic_pos.py
 - create/13_create_multinic_pos.py
 - memset/01_memset_basic_pos.py
 - reboot/01_reboot_basic_pos.py
 - migrate/01_migrate_localhost_pos.py


Creating and Using Domains
==========================

Xen domains, both full and para virtualized, are represented by the 
XmTestDomain class. The class contains methods for starting a Xen domain,
shutting it down, or destroying it. Consoles, used to execute commands
on guest test domains, are opened and closed through the XmTestDomain
class. Devices, which are represented by the XenDevice class, are also
added and removed using XmTestDomain methods. 

Here's a simple example for creating a domain, opening a console, running
a command, and then shutting down the domain:

1) Create a domain:

    domain = XmTestDomain()

2) Start the domain and grab a console:

    try:
        console = domain.start()
    except DomainError, e:
        if verbose:
            print "Failed to create test domain because:"
            print e.extra
        FAIL(str(e))

3) Run a command inside the new domain using the console, saving the
   console log if an error is encountered:

    try:
        # Run 'ls'
        run = console.runCmd("ls")
    except ConsoleError, e:
        saveLog(console.getHistory())
        FAIL(str(e))

4) Stop the domain, which nicely shuts it down:

    domain.stop()


Writing Tests with Networking
=============================

The xm-test suite includes the ability to test networking for domains. 
Networking is configured at configuration time. While testing NAT and
routing environments in the future, the current xm-test only supports
a bridging environment. Xm-test currently only supports a range of
IPs, the dhcp feature will be added soon.

The network tests will need to know what IPs to use. IPs are
configured when you build xm-test. Xm-test by default a range chosen
at random from the RFC1918 private use space, and published at
www.ucam.org/cam-grin, 172.30.206.1-172.30.206.254 from
172.30.206.0/24. If you'd like to set a new range, do so at configure
time, a netmask and network address must also be defined:

    # ./configure --with-net-ip-range=192.0.2.1-192.0.2.100 --with-network-address=192.0.2.0 --with-netmask=255.255.255.0

The tests will not need to set network information, this is done by 
the library once it's configured.

As mentioned above, xm-test's goal is to make writing tests easy. Creating
domains with networking should also be easy. To create a domain with
a single network interface, tests can use a XmTestNetDomain object. It
creates a XenNetDevice for the domain automatically using the pre-configured
IP information. Otherwise, a network interface can be added to a domain
prior to starting it (the ability to attach devices will be added):

    domain = XmTestDomain()
    domain.newDevice(XenNetDevice, "eth0")
    domain.newDevice(XenNetDevice, "eth1")

Here, the domain is created and then the XenDomain factory newDevice
is called to create a new device of class XenNetDevice to the domain. 
The xm-test library will automatically assign an IP from the configured
list, execute ifconfig on the guest domain console, and create an
alias on Domain0.