WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-devel

Re: [Xen-devel] how to PCI-passthrough 1 card to a DomU when another @ D

To: dev001 <pgngw+dev001+xen-users@xxxxxx>
Subject: Re: [Xen-devel] how to PCI-passthrough 1 card to a DomU when another @ Dom0 uses the same driver?
From: Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx>
Date: Wed, 2 Feb 2011 15:27:05 -0500
Cc: xen-devel@xxxxxxxxxxxxxxxxxxx
Delivery-date: Wed, 02 Feb 2011 12:29:35 -0800
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <1296623011.3053.1418495679@xxxxxxxxxxxxxxxxxxxxxxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
References: <1296623011.3053.1418495679@xxxxxxxxxxxxxxxxxxxxxxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
User-agent: Mutt/1.5.20 (2009-06-14)
On Tue, Feb 01, 2011 at 09:03:31PM -0800, dev001 wrote:
> it was suggested (thx!) @ my OP
> (http://lists.xensource.com/archives/html/xen-users/2011-02/msg00013.html)
> might be more appropriate here ... hope that's the case (?)
> 
> my Dom0 host runs,
> 
> uname -a
>         Linux test 2.6.34.8-7-xen #1 SMP 2011-01-20 21:58:39 +0100
>         x86_64 x86_64 x86_64 GNU/Linux
> lsb_release -d
>         Description:    openSUSE 11.3 (x86_64)
> rpm -qa | grep -i Xen-4
>         xen-4.0.1_21326_02-0.3.1.x86_64
> 
> i've got 3 NICs,
> 
> lspci | grep -i eth
>         02:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd.
>         RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 01)
>         03:00.0 Ethernet controller: Marvell Technology Group Ltd.
>         88E8053 PCI-E Gigabit Ethernet Controller (rev 20)
>         04:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd.
>         RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 02)
> 
> where 04:00.0 is assigned as the internal-LAN/Bridge port, and 02:00.0 &
> 03:00.0 are PCIe cards that I intend to PCI-passthrough to a DomU guest.
> 
> so far, I've:
> 
> cat /boot/grub/menu.lst
>         ...
>     kernel /xen.gz ... iommu=verbose dom0_vcpus_pin=true sched=credit
>     cpuidle=1 cpufreq=xen clocksource=acpi numa=on
>     module /vmlinuz-xen ... elevator=cfq clocksource=xen mce=off
>     nomodeset pciback.permissive
>     pciback.hide=(0000:02:00.0)(0000:03:00.0)

>     module /initrd-xen
>         ...
> 
> cat /etc/modprobe.d/99-local.conf
>         options pciback hide=(0000:02:00.0)(0000:03:00.0)
>         install sky2       /sbin/modprobe pciback ; /sbin/modprobe
>         --first-time --ignore-install sky2
>         install r8169      /sbin/modprobe pciback ; /sbin/modprobe
>         --first-time --ignore-install r8169
> 
> cat /etc/init.d/boot.local
>         modprobe pciback hide='(02:00.0)(03.00.0)'

Hmm, the 03.00 is incorrect. Should be 03:00.0

>         echo -n 0000:02:00.0 > sys/bus/pci/drivers/pciback/permissive
>         echo -n 0000:03:00.0 > sys/bus/pci/drivers/pciback/permissive
> 
> @ Dom0 boot, i only see ONE card seized,
> 
> dmesg | egrep -i seiz
>         [    2.081944] pciback 0000:03:00.0: seizing device
> 
> and, of course as a result, for a guest,

So if you look in /sys/pci .. something for 02:00.0 is there any driver
that is using it? Or is it attached ot pciback?

> 
> cat test.cfg
>         ...
>         extra = 'textmode=1 xencons=xvc0 elevator=noop iommu=soft'
>         pci   = [ '02:00.0', '03:00.0']
>         ...
> 
> xm create test.cfg
>         Error: pci: PCI Backend and pci-stub don't own device
>         0000:02:00.0
> 
> 
> if i simply replace the 04:00.0 Realtek card with a non-r8619 driver
> card, e.g.,
> 
>         04:00.0 Ethernet controller: Atheros Communications
>         AR8121/AR8113/AR8114 Gigabit or Fast Ethernet (rev b0)
> 
> and repeat the experiment, then both cards are CORRECTLY seized,
> 
> dmesg | egrep -i seiz
>         [    3.027654] pciback 0000:02:00.0: seizing device
>         [    3.321515] pciback 0000:03:00.0: seizing device
> 
> and I can subsequently launch the DomU, with both cards successfully
> passed through.
> 
> i.e., using two cards with the same driver type FAILS to allow one of
> the cards to be seized.
> 
> i suspect this has to do with the driver not being unloadable at the
> time of device seizure, as it's in use.

Well, the pciback.hide=(..) during bootup should have taken care of that.

Oh, unless the pciback is a _module_ at which point it would not do it.
> 
> if that's the case -- how do i configure this so that the _same_ driver
> can be used for _both_ the Dom0 card instance, and then appropriately
> seized for the PCI-passthru target'd 2nd card?

Did you follow the rules for doing it detailed in the VT-d howto? Just use
the pciback instead of pci-stub in it.

What happens if you 'rmmod sky2' or blacklist the sky2 driver?
> 
> or, is this another issue & there's addition/other config necessary?
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel