############################# __ __ _ ___ \ \/ /___ _ __ / | / _ \ \ // _ \ '_ \ | || | | | / \ __/ | | | | || |_| | /_/\_\___|_| |_| |_(_)___/ ############################# University of Cambridge Computer Laboratory 31 Aug 2003 http://www.cl.cam.ac.uk/netos/xen About the Xen Virtual Machine Monitor ===================================== "Xen" is a Virtual Machine Monitor (VMM) developed by the Systems Research Group of the University of Cambridge Computer Laboratory, as part of the UK-EPSRC funded XenoServers project. The XenoServers project aims to provide a "public infrastructure for global distributed computing", and Xen plays a key part in that, allowing us to efficiently partition a single machine to enable multiple independent clients to run their operating systems and applications in an environment providing protection, resource isolation and accounting. The project web page contains further information along with pointers to papers and technical reports: http://www.cl.cam.ac.uk/xeno Xen has since grown into a project in its own right, enabling us to investigate interesting research issues regarding the best techniques for virtualizing resources such as the CPU, memory, disk and network. The project has been bolstered by support from Intel Research Cambridge, who are now working closely with us. We're also in receipt of support from Microsoft Research Cambridge to port Windows XP to run on Xen. Xen enables multiple operating system images to execute concurrently on the same hardware with very low performance overhead --- much lower than commercial offerings for the same x86 platform. This is achieved by requiring OSs to be specifically ported to run on Xen, rather than allowing unmodified OS images to be used. Crucially, only the OS needs to be changed -- all of the user-level application binaries, libraries etc can run unmodified. Hence the modified OS kernel can typically just be dropped into any existing OS distribution or installation. Xen currently runs on the x86 architecture, but could in principle be ported to others. In fact, it would have been rather easier to write Xen for pretty much any other architecture as x86 is particularly tricky to handle. A good description of Xen's design, implementation and performance is contained in our October 2003 SOSP paper, available at http://www.cl.cam.ac.uk/netos/papers/2003-xensosp.pdf We have been working on porting 3 different operating systems to run on Xen: Linux 2.4, Windows XP, and NetBSD. The Linux 2.4 port (currently Linux 2.4.22) works very well -- we regularly use it to host complex applications such as PostgreSQL, Apache, BK servers etc. It runs every user-space applications we've tried. We refer to our version of Linux ported to run on Xen as "XenoLinux", although really it's just standard Linux ported to a new virtual CPU architecture that we call xeno-x86 (abbreviated to just "xeno"). Unfortunately, the NetBSD port has stalled due to lack of man power. We believe most of the hard stuff has already been done, and are hoping to get the ball rolling again soon. In hindsight, a FreeBSD 4 port might have been more useful to the community. Any volunteers? :-) The Windows XP port is nearly finished. It's running user space applications and is generally in pretty good shape thanks to some hard work by the team over the summer. Of course, there are issues with releasing this code to others. We should be able to release the source and binaries to anyone that has signed the Microsoft academic source license, which these days has very reasonable terms. We are in discussions with Microsoft about the possibility of being able to make binary releases to a larger user community. Obviously, there are issues with product activation in this environment which need to be thought through. So, for the moment, you only get to run multiple copies of Linux on Xen, but we hope this will change before too long. Even running multiple copies of the same OS can be very useful, as it provides a means of containing faults to one OS image, and also for providing performance isolation between the various OS, enabling you to either restrict, or reserve resources for, particular VM instances. It's also useful for development -- each version of Linux can have different patches applied, enabling different kernels to be tried out. For example, the "vservers" patch used by PlanetLab applies cleanly to our ported version of Linux. We've successfully booted over 128 copies of Linux on the same machine (a dual CPU hyperthreaded Xeon box) but we imagine that it would be more normal to use some smaller number, perhaps 10-20. Hardware support ================ Xen is intended to be run on server-class machines, and the current list of supported hardware very much reflects this, avoiding the need for us to write drivers for "legacy" hardware. It is likely that some desktop chipsets will fail to work properly with the default Xen configuration: specifying 'noacpi' or 'ignorebiostables' when booting Xen may help in these cases. Xen requires a "P6" or newer processor (e.g. Pentium Pro, Celeron, Pentium II, Pentium III, Pentium IV, Xeon, AMD Athlon, AMD Duron). Multiprocessor machines are supported, and we also have basic support for HyperThreading (SMT), although this remains a topic for ongoing research. We're also looking at an AMD x86_64 port (though it should run on Opterons in 32-bit mode just fine). Xen can currently use up to 4GB of memory. It's possible for x86 machines to address more than that (64GB), but it requires using a different page table format (3-level rather than 2-level) that we currently don't support. Adding 3-level PAE support wouldn't be difficult, but we'd also need to add support to all the guest OSs. Volunteers welcome! We currently support a relatively modern set of network cards: Intel e1000, Broadcom BCM 57xx (tg3), 3COM 3c905 (3c59x). Adding support for other NICs that support hardware DMA scatter/gather from half-word aligned addresses is relatively straightforward, by porting the equivalent Linux driver. Drivers for a number of other older cards have recently been added [pcnet32, e100, tulip], but these are not recommended since they require extra packet copies. Building Xen and XenoLinux ========================== The public master BK repository for the 1.0 release lives at: bk://xen.bkbits.net/xeno-1.0.bk To fetch a local copy, install the BitKeeper tools, then run: 'bk clone bk://xen.bkbits.net/xeno-1.0.bk' To see how to build Xen, Xenolinux, and all the control tools, inspect the tools/misc/xen-clone script in the BK repository. This script can be used to clone the repostitory and perform a full build. The build procedure for xenolinux is slightly complicated as its done by running the 'mkbuildtree' script over a pristine Linux tree to turn it into a xenolinux tree by adding the 'xeno' architecture. There's also a pre-built source tree on the project downloads page. Using the domain control tools ============================== The README.CD file contains some examples of how to use 'xenctl' and the other domain control tools. Invoking the tool without any arguments prints some usage inforamtion. There's also some documentation in the the repository under tools/control/doc Ian Pratt 9 Sep 2003