1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
|
Title : How to do PCI Passthrough with VT-d
Authors : Allen Kay <allen.m.kay@intel.com>
Weidong Han <weidong.han@intel.com>
Created : October-24-2007
Updated : August-06-2008
How to turn on VT-d in Xen
--------------------------
1 ) cd xen-unstable.hg
2 ) make install
3 ) make linux-2.6-xen-config CONFIGMODE=menuconfig
4 ) change XEN->"PCI-device backend driver" from "M" to "*".
5 ) make linux-2.6-xen-build
6 ) make linux-2.6-xen-install
7 ) depmod 2.6.18.8-xen
8 ) mkinitrd -v -f --with=ahci --with=aacraid --with=sd_mod --with=scsi_mod initrd-2.6.18-xen.img 2.6.18.8-xen
9 ) cp initrd-2.6.18-xen.img /boot
10) lspci - select the PCI BDF you want to assign to guest OS
11) "hide" pci device from dom0 as following sample grub entry:
title Xen-Fedora Core (2.6.18-xen)
root (hd0,0)
kernel /boot/xen.gz com1=115200,8n1 console=com1 iommu=1
module /boot/vmlinuz-2.6.18.8-xen root=LABEL=/ ro xencons=ttyS console=tty0 console=ttyS0, pciback.hide=(01:00.0)(03:00.0)
module /boot/initrd-2.6.18-xen.img
12) reboot system
13) add "pci" line in /etc/xen/hvm.conf for to assigned devices
pci = [ '01:00.0', '03:00.0' ]
15) start hvm guest and use "lspci" to see the passthru device and
"ifconfig" to see if IP address has been assigned to NIC devices.
Enable MSI/MSI-x for assigned devices
-------------------------------------
Add "msi=1" option in kernel line of host grub.
Caveat on Conventional PCI Device Passthrough
---------------------------------------------
VT-d spec specifies that all conventional PCI devices behind a
PCIe-to-PCI bridge have to be assigned to the same domain.
PCIe devices do not have this restriction.
VT-d Works on OS:
-----------------
1) Host OS: PAE, 64-bit
2) Guest OS: 32-bit, PAE, 64-bit
Combinations Tested:
--------------------
1) 64-bit host: 32/PAE/64 Linux/XP/Win2003/Vista guests
2) PAE host: 32/PAE Linux/XP/Win2003/Vista guests
VTd device hotplug:
-------------------
2 virtual PCI slots (6~7) are reserved in HVM guest to support VTd hotplug. If you have more VTd devices, only 2 of them can support hotplug. Usage is simple:
1. List the VTd device by dom. You can see a VTd device 0:2:0.0 is inserted in the HVM domain's PCI slot 6. '''lspci''' inside the guest should see the same.
[root@vt-vtd ~]# xm pci-list HVMDomainVtd
VSlt domain bus slot func
0x6 0x0 0x02 0x00 0x0
2. Detach the device from the guest by the physical BDF. Then HVM guest will receive a virtual PCI hot removal event to detach the physical device
[root@vt-vtd ~]# xm pci-detach HVMDomainVtd 0:2:0.0
3. Attach a PCI device to the guest by the physical BDF and desired virtual slot(optional). Following command would insert the physical device into guest's virtual slot 7
[root@vt-vtd ~]# xm pci-attach HVMDomainVtd 0:2:0.0 7
VTd hotplug usage model:
------------------------
* For live migration: As you know, VTd device would break the live migration as physical device can't be save/restored like virtual device. With hotplug, live migration is back again. Just hot remove all the VTd devices before live migration and hot add new VTd devices on target machine after live migration.
* VTd hotplug for device switch: VTd hotplug can be used to dynamically switch physical device between different HVM guest without shutdown.
VT-d Enabled Systems
--------------------
1) For VT-d enabling work on Xen, we have been using development
systems using following Intel motherboards:
- DQ35MP
- DQ35JO
2) As far as we know, following OEM systems also has vt-d enabled.
Feel free to add others as they become available.
- Dell: Optiplex 755
http://www.dell.com/content/products/category.aspx/optix?c=us&cs=555&l=en&s=biz
- HP Compaq: DC7800
http://h10010.www1.hp.com/wwpc/us/en/en/WF04a/12454-12454-64287-321860-3328898.html
For more information, pls refer to http://wiki.xensource.com/xenwiki/VTdHowTo.
|
