Re: [Xen-users] pci passthrough how to that works?

[Matthias Reif <mattreif@xxxxxxxxxxxxx>]

Brian Krusic <brian@xxxxxxxxxx> wrote on 08/01/2010
06:23:11 AM:

> > On Jan 6, 2010, at 2:57 PM, Matthias Reif wrote: 
> > 
> > 
> > 
> > xen-users-bounces@xxxxxxxxxxxxxxxxxxx wrote on 07/01/2010 09:32:23
AM:
> > 
> > > Hi,
> > > 
> > > I'm running Centos 5.3 for dom0 and domUs and using Xen
3.4.1 (AMD V  
> > > enabled Sun box).
> > > 
> > > Using lspci, I see that my device I wished ignored by dom0
so that I  
> > > can use it for 1 of my domUs is;
> > > 
> > > 07:00.0
> > > 
> > > In grub.conf, I added this to my module line;
> > > 
> > > pciback.permissive pciback.hide (07:00.0)
> > > 
> > > Upon reboot, lspci on dom0 still shows that pci device.
> > > 
> > > I've also modded my modprob.conf;
> > > 
> > > options pciback hide=(07:00.0)
> > > 
> > > Any ideas how I can get pci passthrough to work?
> > > 
> > > - Brian
> > > 
> > 
> > The pciback module needs to be loaded into initrd, so it can
seize 
> > the device at boot time. 
> > Your boot log should confirm this if it says something like "unknwon
> > kernel option pciback.hide 
> > ignoring." 
> > 
> > Try something like this: 
> > 
> > mkinitrd -f --preload=pciback /boot/initrd-$(uname -r).img $(uname
-r) 
> > 
> > Also, pciback has to be loaded before the driver for your device.
> > This is not always the case. 
> > I tried to seize a SCSI controller to pass a tape drive to a
domU, 
> > but its driver had to be loaded 
> > first to get access to the disks where the initrd etc is stored
(I 
> > presume). When pciback was loaded 
> > the driver had already taken control of the device. 
> > If that's the case, late binding may be an option for you, where
you
> > unbind the driver from the 
> > device after the system has booted and then bind it to pciback
prior
> > to starting your domU. 
> > 
> > Eg. 
> > # echo -n 0000:07:00.0 > /sys/bus/pci/drivers/<your_driver_name>/unbind
> > # echo -n 0000:07:00.0 > /sys/bus/pci/drivers/pciback/new_slot
> > # echo -n 0000:07:00.0 > /sys/bus/pci/drivers/pciback/bind
> > 
> > 
> > Matthias 
> > 
> > 
> > 
> > 
> Brian Krusic <brian@xxxxxxxxxx> wrote on 07/01/2010 12:26:27
PM:
> 
> > Hi Matthias, 
> > 
> > Heres what I did; 
> > 
> > mkinitrd -f –preload=pciback /boot/initrd-$(uname -r).img $(uname
-r) 
> > 
> > Then added this line to modprob.conf 
> > 
> > options pciback hide=(07:00.0) 
> > 
> > After a reboot, I still see the device under lspci, is this normal?
> > 
> > I went ahead and added this to the file xend-pci-permissive.sxp;
> > 
> > (unconstrained_dev_ids 
> > #(’0123:4567:89AB:CDEF’) 
> > (’10de:042f’) 
> > ) 
> > 
> > I got the 10de:042f from doing this; 
> > 
> > lspci -n 
> > 
> > Which gave; 
> > 
> > 07:00.00 0300: 10de:042f (rev a1) 
> > 
> > The next step is to assign this PCI device to domU.  In
normal 
> > syntax I would simply edit domU config file and add; 
> > 
> > pci = [ "07:00.0"] 
> > 
> > But my domU configs are now using config.sxp which is this new
> > bloody XML format. 
> > 
> > How do I put pci = [ "07:00.0"] in this XML format?
> > 
> >   
> > - Brian 
> > 
> 
> 
> First make sure that the new initrd that you built with pciback included
is 
> also referred to in your /boot/grub/grub.conf 
> 
> Secondly I would put the pciback module options also into grub.conf.
I am not
> sure whether modprobe.conf is actually used at boot time. 
> 
> Eg. 
> title CentOS (2.6.18-128.4.1.el5xen) Hypervisor 3.4.1 CentOS dom0
>         root (hd0,0) 
>         kernel /xen.gz-3.4.1 
>         module /vmlinuz-2.6.18-128.4.1.el5xen
ro root=/dev/md1 
> pciback.hide=(07:00.0) 
>         module /initrd-2.6.18-128.4.1.el5xen.img
> 
> 
> Then check dmesg to confirm that pciback is loaded and is trying to
> seize the device. 
> 
> Yes, doing;
> 
> dmesg | grep pciback
> 
> yields;
> 
> pciback 0000:07:00.0: seizing device
> 
> How do I get it visible in domU?
> 
> Its a pci-e based add on graphics card, my primary being on the motherboard.
> 
> In xend-pci-permissive.sxp I have added;
> 
> (unconstrained_dev_ids 
> (’10de:042f’) 
> ) 
> 
> Keep in mind my domU configs are in this XML format and look 
> something like this;
> 
> domain
>     (domid 2)
>     (vcpus_params ((cap 0) (weight
256)))
>     (PV_args )
>     (PV_bootloader /usr/bin/pygrub)
>     (cpus ((2) (3)))
>     (pci '=')
>     (VCPUs_live 2)
>     (actions_after_crash restart)
>     (uuid c58c75ca-b6ca-0888-37e1-4e43137c5e24)
>     (PV_ramdisk )
>     (name_label logantools)
>     (VCPUs_at_startup 1)
>     (HVM_boot_params ())
>     (platform ((device_model /usr/lib64/xen/bin/qemu-dm)
(pci '=')))
>     (PV_kernel )
>     (memory_static_min 0)
>     (HVM_boot_policy )
>     (VCPUs_max 2)
>     (memory_static_max 1073741824)
>     (actions_after_shutdown destroy)
>     (memory_dynamic_max 1073741824)
>     (is_a_template False)
>     (memory_dynamic_min 1073741824)
>     (target 0)
>     (PV_bootloader_args )
>     (other_config ())
>     (auto_power_on False)
>     (actions_after_reboot restart)
>     (on_crash restart)
>     (uuid c58c75ca-b6ca-0888-37e1-4e43137c5e24)
>     (bootloader_args )
>     (vcpus 2)
>     (name logantools)
>     (on_poweroff destroy)
>     (on_reboot restart)
>     (cpus ((2) (3)))
>     (bootloader /usr/bin/pygrub)
>     (maxmem 1024)
>     (memory 1024)
>     (shadow_memory 0)
>     (vcpu_avail '3')
>     (features )
>     (on_xend_start ignore)
>     (on_xend_stop ignore)
>     (start_time 1262827803.79)
>     (cpu_time 15.85152715)
>     (image
>         (linux
>             (kernel
)
>             (pci
'=')
>             (device_model
/usr/lib64/xen/bin/qemu-dm)
>             (notes
>              
  (FEATURES
>              
      
> 'writable_page_tables|writable_descriptor_tables|auto_translated_physmap|pae_pgdir_above_4gb|supervisor_mode_kernel'
>              
  )
>              
  (VIRT_BASE 18446744071562067968)
>              
  (GUEST_VERSION 2.6)
>              
  (PADDR_OFFSET 18446744071562067968)
>              
  (GUEST_OS linux)
>              
  (HYPERCALL_PAGE 18446744071564189696)
>              
  (LOADER generic)
>              
  (ENTRY 18446744071564165120)
>              
  (XEN_VERSION xen-3.0)
>             )
>         )
>     )
>     (status 1)
>     (state ------)
>     (store_mfn 1625844)
>     (console_mfn 1625843)
>     (device
>         (vif
>             (bridge
bond0)
>             (mac
00:16:36:60:3f:34)
>             (script
/etc/xen/scripts/vif-bridge)
>             (uuid
b088f989-4eb2-bd1e-a372-e8278e86bfa9)
>             (backend
0)
>         )
>     )
>     (device (vkbd (backend 0)))
>     (device
>         (vfb
>             (vncunused
1)
>             (keymap
en-us)
>             (vnc
1)
>             (uuid
9394a20e-3ae2-29c1-17a8-7148e4b3df2d)
>             (location
127.0.0.1:5901)
>         )
>     )
>     (device
>         (console
>             (protocol
vt100)
>             (location
2)
>             (uuid
0a7d47e5-d259-dbee-0f24-25c93c5bd0cc)
>         )
>     )
>     (device
>         (tap
>             (protocol
x86_64-abi)
>             (uuid
4fa9e83c-9940-b9ff-94a1-526c26ced26c)
>             (bootable
1)
>             (dev
xvda:disk)
>             (uname
tap:aio:/var/lib/libvirt/images/logantools.img)
>             (mode
w)
>             (backend
0)
>             (bootable
1)
>             (VDI
)
>         )
>     )
> )
> 
>  

I am not sure myself. I still use the old style config
file format,

ie.

pci = [ '0000:07:00.0' ]

Maybe someone else on the list can help.


Matthias






SOUTH PACIFIC SEEDS
PO Box 934, GRIFFITH N.S.W. 2680. AUSTRALIA
Phone: 02 6962 7333      Fax: 02 6964 1311

---

CAUTION: The information contained
in this e-mail is privileged and confidential. If you are reading this
message and you are not the intended recipient, you are hereby notified
that any use, dissemination, distribution or reproduction of all or part
of the contents is prohibited. If you receive this message in error, please
notify the sender immediately. Any opinions or views expressed in this
message are those of the individual sender and may not represent those
of their employer.

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users