WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-users

Re: [Xen-users] pci passthrough under xen-3.3 without VT-d

To: "Benjamin Huntsman" <BHuntsman@xxxxxxxxxxxxxxxxxxxxx>
Subject: Re: [Xen-users] pci passthrough under xen-3.3 without VT-d
From: "Todd Deshane" <deshantm@xxxxxxxxx>
Date: Mon, 15 Dec 2008 18:47:21 -0500
Cc: xen-users@xxxxxxxxxxxxxxxxxxx
Delivery-date: Mon, 15 Dec 2008 15:48:09 -0800
Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:reply-to :to:subject:cc:in-reply-to:mime-version:content-type :content-transfer-encoding:content-disposition:references; bh=7mPtinTESVJmM1DkoBex/T7/cHpAjCr+gAmz4ZTzVqI=; b=JZm7CthFPANJCwF1Z6PDkVkYbQXJGb7PZpNzMHYiZ6K13qESN2AnwhXZFTACCTN6GK COjQx464rwIjNroKMExDDmAmVW2MJVqCV1clXLBSZKcViGR6b6JZ/yoUxYreQZMjXOhN g7B6uUR0YdAnMVY2N13PyaDgCedDtnmUyL1QA=
Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:reply-to:to:subject:cc:in-reply-to :mime-version:content-type:content-transfer-encoding :content-disposition:references; b=MVF4lwREDpTKxEcjmiGTiRaQAXLCQQAvSs4WgNY87KOAFUuxVmgHnbJuD8Mp4z0s3p TA8H938wsr7D6kqz1f58WVuaZEW2m0yMQkFht7oEnT2JEZJXV9+HNFEEr6D1hri0FAaE DF26JEYS+ZUFNukhlOYATlvJCmQQND9kTiCnc=
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <621112A569DAE948AD25CCDCF1C075331AB3E1@xxxxxxxxxxxxxxxxx>
List-help: <mailto:xen-users-request@lists.xensource.com?subject=help>
List-id: Xen user discussion <xen-users.lists.xensource.com>
List-post: <mailto:xen-users@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=unsubscribe>
References: <621112A569DAE948AD25CCDCF1C075331AB3E0@xxxxxxxxxxxxxxxxx> <621112A569DAE948AD25CCDCF1C075331AB3E1@xxxxxxxxxxxxxxxxx>
Reply-to: deshantm@xxxxxxxxx
Sender: xen-users-bounces@xxxxxxxxxxxxxxxxxxx
On Mon, Dec 15, 2008 at 6:36 PM, Benjamin Huntsman
<BHuntsman@xxxxxxxxxxxxxxxxxxxxx> wrote:
> Hi all!
>   I'm hoping someone here might be able to help.  I'm familiar with several 
> UNIX'es, though farily new to Linux...
> I'm running RHEL 5.2.  I've got the RHEL kernel headers installed, etc, and 
> compiled xen-3.3.
> Once that was built, I rebuilt libvirt, etc, the kernel, and all the modules. 
>  The whole mess boots and runs fine... mostly.
>
> My hardware is a Dell OptiPlex 745, Intel Core2 Duo, though the board doesn't 
> have VT-d capabilities.  Vitalization is otherwise enabled in the BIOS.  I 
> have a PCI NIC and PCI VGA card I want to pass through to a guest.  I have 
> pciback built as a module, and it's preloaded in my initrd.
>
> As I understand it though, even w/o VT-d, I should still be able to do PCI 
> passthrough, right?
>

Yes, as long as you understand the security implications (i.e driver
DMA to memory it doesn't own)


> Here's the config details (pardon the verbosity):
>
> from /boot/grub/menu.lst:
> ...
> title Red Hat Enterprise Linux Server (2.6.18-92.1.18.el6xen)(Xen-3.3.0)
>        root (hd0,1)
>        kernel /xen-3.3.0.gz iommu=1 vtd=1

I am pretty sure that you don't want the iommu line in here, since you
don't have
an iommu


>        module /vmlinuz-2.6.18-92.1.18.el5xen ro root=/dev/VolGroup00/LogVol00 
> rhgb quiet

instead of doing the modprobe below, you could instead pass the pciback.hide
on the dom0 command line. It should give a similar effect, but still
worth a shot.


>        module /initrd-2.6.18-92.1.18.el5xen.img
> ...
>
> from /etc/modprobe.conf:
> ...
> install e100 /sbin/modprobe pciback ; /sbin/modprobe --first-time 
> --ignore-install e100
> options pciback hide=(04:00.0)(04:02.0)
> ...
>
> [root@hx80kc1 ~]# dmesg | grep pciback
> pciback 0000:04:02.0: seizing device
> pciback 0000:04:00.0: seizing device
> [root@hx80kc1 ~]#
>
> [root@hx80kc1 ~]# xm dmesg
>  __  __            _____  _____  ___
>  \ \/ /___ _ __   |___ / |___ / / _ \
>  \  // _ \ '_ \    |_ \   |_ \| | | |
>  /  \  __/ | | |  ___) | ___) | |_| |
>  /_/\_\___|_| |_| |____(_)____(_)___/
>
> (XEN) Xen version 3.3.0 (root@xxxxxxxxxxxxxxx) (gcc version 4.1.2 20071124 
> (Red Hat 4.1.2-42)) Fri Dec  5 17:21:39 PST 2008
> (XEN) Latest ChangeSet: unavailable
> (XEN) Command line: iommu=1 vtd=1
> (XEN) Video information:
> (XEN)  VGA is text mode 80x25, font 8x16
> (XEN)  VBE/DDC methods: V2; EDID transfer time: 1 seconds
> (XEN) Disc information:
> (XEN)  Found 1 MBR signatures
> (XEN)  Found 1 EDD information structures
> (XEN) Xen-e820 RAM map:
> (XEN)  0000000000000000 - 000000000009fc00 (usable)
> (XEN)  00000000000f0000 - 0000000000100000 (reserved)
> (XEN)  0000000000100000 - 00000000bfdffc00 (usable)
> (XEN)  00000000bfdffc00 - 00000000bfe01c00 (ACPI NVS)
> (XEN)  00000000bfe03c00 - 00000000bfe53c00 (reserved)
> (XEN)  00000000bfe53c00 - 00000000bfe55c00 (ACPI data)
> (XEN)  00000000bfe55c00 - 00000000c0000000 (reserved)
> (XEN)  00000000e0000000 - 00000000f0000000 (reserved)
> (XEN)  00000000fec00000 - 00000000fed00400 (reserved)
> (XEN)  00000000fed20000 - 00000000feda0000 (reserved)
> (XEN)  00000000fee00000 - 00000000fef00000 (reserved)
> (XEN)  00000000ffb00000 - 0000000100000000 (reserved)
> (XEN)  0000000100000000 - 000000013c000000 (usable)
> (XEN) System RAM: 4029MB (4126328kB)
> (XEN) ACPI: RSDP 000FEBF0, 0024 (r2 DELL  )
> (XEN) ACPI: XSDT 000FCE90, 006C (r1 DELL    B8K           14 ASL        61)
> (XEN) ACPI: FACP 000FCFB8, 00F4 (r3 DELL    B8K           14 ASL        61)
> (XEN) ACPI: DSDT FFF69C33, 4757 (r1   DELL    dt_ex     1000 INTL 20050624)
> (XEN) ACPI: FACS BFDFFC00, 0040
> (XEN) ACPI: SSDT FFF6E662, 009A (r1   DELL    st_ex     1000 INTL 20050624)
> (XEN) ACPI: APIC 000FD0AC, 0092 (r1 DELL    B8K           14 ASL        61)
> (XEN) ACPI: BOOT 000FD13E, 0028 (r1 DELL    B8K           14 ASL        61)
> (XEN) ACPI: ASF! 000FD166, 0092 (r32 DELL    B8K           14 ASL        61)
> (XEN) ACPI: MCFG 000FD1F8, 003E (r1 DELL    B8K           14 ASL        61)
> (XEN) ACPI: HPET 000FD236, 0038 (r1 DELL    B8K           14 ASL        61)
> (XEN) ACPI: TCPA 000FD492, 0032 (r1 DELL    B8K           14 ASL        61)
> (XEN) ACPI: SLIC 000FD26E, 0176 (r1 DELL    B8K           14 ASL        61)
> (XEN) Xen heap: 14MB (14624kB)
> (XEN) Domain heap initialised
> (XEN) Processor #0 6:15 APIC version 20
> (XEN) Processor #1 6:15 APIC version 20
> (XEN) IOAPIC[0]: apic_id 8, version 32, address 0xfec00000, GSI 0-23
> (XEN) Enabling APIC mode:  Flat.  Using 1 I/O APICs
> (XEN) Using scheduler: SMP Credit Scheduler (credit)
> (XEN) Detected 2126.508 MHz processor.
> (XEN) HVM: VMX enabled
> (XEN) CPU0: Intel(R) Core(TM)2 CPU          6400  @ 2.13GHz stepping 06
> (XEN) Booting processor 1/1 eip 8c000
> (XEN) CPU1: Intel(R) Core(TM)2 CPU          6400  @ 2.13GHz stepping 06
> (XEN) Total of 2 processors activated.
> (XEN) ENABLING IO-APIC IRQs
> (XEN)  -> Using new ACK method
> (XEN) checking TSC synchronization across 2 CPUs: passed.
> (XEN) Platform timer is 14.318MHz HPET
> (XEN) Brought up 2 CPUs
> (XEN) I/O virtualisation disabled
> (XEN) *** LOADING DOMAIN 0 ***
> (XEN)  Xen  kernel: 64-bit, lsb, compat32
> (XEN)  Dom0 kernel: 64-bit, lsb, paddr 0xffffffff80200000 -> 
> 0xffffffff807014e4
> (XEN) PHYSICAL MEMORY ARRANGEMENT:
> (XEN)  Dom0 alloc.:   0000000134000000->0000000136000000 (969614 pages to be 
> allocated)
> (XEN) VIRTUAL MEMORY ARRANGEMENT:
> (XEN)  Loaded kernel: ffffffff80200000->ffffffff807014e4
> (XEN)  Init. ramdisk: ffffffff80702000->ffffffff80ea6000
> (XEN)  Phys-Mach map: ffffffff80ea6000->ffffffff8161bc70
> (XEN)  Start info:    ffffffff8161c000->ffffffff8161c4a4
> (XEN)  Page tables:   ffffffff8161d000->ffffffff8162c000
> (XEN)  Boot stack:    ffffffff8162c000->ffffffff8162d000
> (XEN)  TOTAL:         ffffffff80000000->ffffffff81800000
> (XEN)  ENTRY ADDRESS: ffffffff80200000
> (XEN) Dom0 has maximum 2 VCPUs
> (XEN) Scrubbing Free RAM: .done.
> (XEN) Xen trace buffers: disabled
> (XEN) Std. Loglevel: Errors and warnings
> (XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings)
> (XEN) Xen is relinquishing VGA console.
> (XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to 
> Xen)
> (XEN) Freed 108kB init memory.
> (XEN) mtrr: type mismatch for c0000000,1000000 old: write-back new: 
> write-combining
> (XEN) mtrr: type mismatch for c0000000,1000000 old: write-back new: 
> write-combining
> [root@hx80kc1 ~]#
>
> [root@hx80kc1 ~]# lspci
> ...
> 04:00.0 Ethernet controller: Intel Corporation 82557/8/9/0/1 Ethernet Pro 100 
> (rev 05)
> ...
> 04:02.0 VGA compatible controller: nVidia Corporation NV5M64 [RIVA TNT2 Model 
> 64/Model 64 Pro] (rev 15)
> ...
> [root@hx80kc1 ~]#
>
> [root@hx80kc1 ~]# lspci -n
> ...
> 04:00.0 0200: 8086:1229 (rev 05)
> 04:02.0 0300: 10de:002d (rev 15)
> #
>
> # tail -n 10 /etc/xen/xend-pci-permissive.sxp
> # (unconstrained_dev_ids
> #     ('XXXX:XXXX:XXXX:XXXX'    # existing entry
> #      'YYYY:YYYY:YYYY:YYYY'    # new entry 1
> #      'ZZZZ:ZZZZ')             # new entry 2
> # )
> ###############################################################################
> (unconstrained_dev_ids
>     #('0123:4567:89AB:CDEF')
>     ('10de:002d')
> )
> [root@hx80kc1 ~]#
>
> [root@hx80kc1 ~]# grep pci /etc/xen/winxp
> pci = [ "04:00.0" ]
> [root@hx80kc1 ~]#
>
> [root@hx80kc1 ~]# xm pci-list-assignable-devices

I'm not sure it this command is setup to work with anything
other than the VT-d stuff.

> [root@hx80kc1 ~]#
>
> [root@hx80kc1 ~]# xm create /etc/xen/winxp
> Using config file "/etc/xen/winxp".
> Error: failed to assign device: maybe the platform doesn't support VT-d, or 
> VT-d
>  isn't enabled properly?


I think you are getting this error because you passed iommu=1

> [root@hx80kc1 ~]#
>
>
> --------------------------------------
> Okay, there's as much detail as I can think to include right now.  I'm 
> concerned that 'xm dmesg' shows the following message:
> (XEN) I/O virtualisation disabled
> Thoung I don't know if that would actually prevent PCI passthrough.
>
> So, how do I get the PCI card passed through to my hvm domain?  What am I 
> missing?
>

You can't pass to a HVM domain without VT-d. You can only pass to a PV guest.

Hope that helps.

Cheers,
Todd


-- 
Todd Deshane
http://todddeshane.net
http://runningxen.com

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users