Re: [Xen-users] PCI Passthrough without VT-d

["Sergio Charpinel Jr." <sergiocharpinel@xxxxxxxxx>]

Pasi,

I dont know if I unbind the device correctly. I'm using this script:

modprobe pciback

BDF=0000:00:1d.3

# Unbind a PCI function from its driver as necessary

[ ! -e /sys/bus/pci/devices/$BDF/driver/unbind ] || \

        echo -n $BDF > /sys/bus/pci/devices/$BDF/driver/unbind

# Add a new slot to the PCI Backend's list

echo -n $BDF > /sys/bus/pci/drivers/pciback/new_slot

# Now that the backend is watching for the slot, bind to it

echo -n $BDF > /sys/bus/pci/drivers/pciback/bind

And I got this in dmesg after running it:

uhci_hcd 0000:00:1d.3: remove, state 1

usb usb5: USB disconnect, address 1

usb 5-1: USB disconnect, address 2

drivers/media/video/stv680.c: [usb_stv680_remove_disconnected:1451] STV(i): STV0680 disconnected

uhci_hcd 0000:00:1d.3: USB bus 5 deregistered

ACPI: PCI interrupt for device 0000:00:1d.3 disabled

pciback 0000:00:1d.3: seizing device

PCI: Enabling device 0000:00:1d.3 (0000 -> 0001)

ACPI: PCI Interrupt 0000:00:1d.3[D] -> GSI 22 (level, low) -> IRQ 21

ACPI: PCI interrupt for device 0000:00:1d.3 disabled

tap tap-1-51712: 2 getting info

pciback: vpci: 0000:00:1d.3: assign to virtual slot 0

ACPI: PCI interrupt for device 0000:00:1d.3 disabled

tap tap-2-51712: 2 getting info

pciback: vpci: 0000:00:1d.3: assign to virtual slot 0

device vif2.0 entered promiscuous mode

ADDRCONF(NETDEV_UP): vif2.0: link is not ready

PCI: Enabling device 0000:00:1d.3 (0000 -> 0001)

ACPI: PCI Interrupt 0000:00:1d.3[D] -> GSI 22 (level, low) -> IRQ 21

ACPI: PCI Interrupt 0000:00:1d.3[D] -> GSI 22 (level, low) -> IRQ 21

PCI: Setting latency timer of device 0000:00:1d.3 to 64

pciback 0000:00:1d.3: Driver tried to write to a read-only configuration space field at offset 0xc0, size 2. This may be harmless, but if you have problems with your device:

1) see permissive attribute in sysfs

2) report problems to the xen-devel mailing list along with details of your device obtained from lspci.

blktap: ring-ref 9, event-channel 7, protocol 1 (x86_64-abi)

Also, here is the output of cat /proc/interrupts in dom0:

cat /proc/interrupts 

           CPU0              CPU1              CPU2              CPU3              

 ..

 20:         65          0          0          0        Phys-irq  ehci_hcd:usb1, uhci_hcd:usb2, uhci_hcd:usb4

 21:         26          0          0          0        Phys-irq  uhci_hcd:usb3

245:     127288          0          0         26        Phys-irq  peth0

...

and in domU:

cat /proc/interrupts 

           CPU0              

 21:          0        Phys-irq  uhci_hcd:usb1

As you see, I got an warning message in dmesg.

Do you know what could be?

2010/3/8 Pasi Kärkkäinen <pasik@xxxxxx>

On Mon, Mar 08, 2010 at 11:28:41AM -0300, Sergio Charpinel Jr. wrote:
>    Not sure, how can I confirm?
>

Check /proc/interrupts and "lspci -vvv" in dom0.
Also read dom0 "dmesg" output for the devices in question.

-- Pasi

>    2010/3/8 Pasi Kärkkäinen <[1]pasik@xxxxxx>

>
>      On Mon, Mar 08, 2010 at 11:23:44AM -0300, Sergio Charpinel Jr. wrote:
>      >    Thanks Pasi,
>      >    Tried with Xen from CentOS 5.4 repo, and got the same error.
>      >    Also with extra = 'swiotlb=force' in DomU conf file.
>      >    DomU boots, but when I start zoneminder (a software that access the
>      >    webcam), the kernel panic occurs.
>      >    I will test another application.
>      >
>
>      Is the PCI device you passthrough using a shared interrupt (IRQ)?
>      If yes, that can (and will) cause problems.
>
>      -- Pasi
>

>      >    2010/3/5 Pasi Kärkkäinen <[1][2]pasik@xxxxxx>

>      >
>      >      On Fri, Mar 05, 2010 at 02:01:28PM -0300, Sergio Charpinel Jr.
>      wrote:
>      >      >    Hi,
>      >      >    I created a PV guest, and did a passthrough on my PCI USB
>      >      Controller, to
>      >      >    use the webcam.
>      >      >    But, I'm receiving a kernel panic message on my domU afte
>      boot.
>      >      >    I'm using CentOS 5.4 and Xen 3.4.2
>      >      >    Password: Fatal DMA error! Please use 'swiotlb=force'
>      >
>      >      Did you try that 'swiotlb=force' option for the guest kernel?
>      >      Also, does the same problem happen if you run the official EL5
>      Xen
>      >      (3.1.2)
>      >      instead of 3.4.2 ?
>      >
>      >      -- Pasi
>      >      >    ----------- [cut here ] --------- [please bite here ]
>      ---------
>      >      >    Kernel BUG at
>      >      arch/x86_64/kernel/../../i386/kernel/pci-dma-xen.c:394
>      >      >    invalid opcode: 0000 [1] SMPÂ
>      >      >    last sysfs file:
>      >      >
>      >
>      /devices/xen/pci-0/pci0000:00/0000:00:00.3/usb1/1-1/1-1:1.0/usbdev1.2_ep82/dev
>      >      >    CPU 0Â
>      >      >    Modules linked in: autofs4 hidp rfcomm l2cap bluetooth lockd
>      sunrpc
>      >      >    ip_conntrack_netbios_ns ipt_MASQUERADE xt_mark iptable_nat
>      ip_nat
>      >      xt_MARK
>      >      >    iptable_mangle ipt_REJECT xt_state ip_conntrack nfnetlink
>      >      iptable_filter
>      >      >    ip_tables ip6t_REJECT xt_tcpudp ip6table_filter ip6_tables
>      x_tables
>      >      ipv6
>      >      >    xfrm_nalgo crypto_api dm_mirror dm_multipath scsi_dh
>      scsi_mod
>      >      parport_pc
>      >      >    lp parport stv680 compat_ioctl32 videodev v4l1_compat pcspkr
>      >      v4l2_common
>      >      >    xennet dm_raid45 dm_message dm_region_hash dm_log dm_mod
>      >      dm_mem_cache
>      >      >    xenblk ext3 jbd uhci_hcd ohci_hcd ehci_hcd
>      >      >    Pid: 1752, comm: zmc Not tainted 2.6.18-164.11.1.el5xen #1
>      >      >    RIP: e030:[<ffffffff8027217b>] Â [<ffffffff8027217b>]
>      >      >    dma_map_single+0x12d/0x17d
>      >      >    RSP: e02b:ffff88000cfb1c78 Â EFLAGS: 00010282
>      >      >    RAX: 000000000000002f RBX: ffff88000e860000 RCX:
>      ffffffff804f3ba8
>      >      >    RDX: 0000000000000000 RSI: 0000000000000001 RDI:
>      0000000000000000
>      >      >    RBP: 0000000035032000 R08: ffffffff804f3ba8 R09:
>      0000000000000000
>      >      >    R10: 000000000000002d R11: ffff88001f4230c0 R12:
>      0000000000019610
>      >      >    R13: ffff88001fdef870 R14: ffff88001fb5f980 R15:
>      ffff880010296ad4
>      >      >    FS: Â 00002b8745423c10(0000) GS:ffffffff805ca000(0000)
>      >      >    knlGS:0000000000000000
>      >      >    CS: Â e033 DS: 0000 ES: 0000
>      >      >    Process zmc (pid: 1752, threadinfo ffff88000cfb0000, task
>      >      >    ffff88000f4530c0)
>      >      >    Stack: Â ffff88000f4530c0 Â ffff880010296ac0 Â
>      00000000ffffffff
>      >      >    Â ffff880010296ac0Â
>      >      >    Â ffff88001ff7d400 Â ffffffff803e4200 Â 0000000000000000
>      >      >    Â 000000d0802572b0Â
>      >      >    Â fffffffffff7ffff  ffffffff802c2e18Â
>      >      >    Call Trace:
>      >      >    Â [<ffffffff803e4200>] hcd_submit_urb+0x693/0x748
>      >      >    Â [<ffffffff802c2e18>] mod_zone_page_state+0x27/0x3d
>      >      >    Â [<ffffffff8020b242>] kmem_cache_alloc+0x62/0x6d
>      >      >    Â [<ffffffff8025e6e3>] cache_alloc_refill+0x13c/0x4e5
>      >      >    Â [<ffffffff802cfc34>] __kmalloc+0x8f/0x9f
>      >      >    Â [<ffffffff881377e0>]
>      :stv680:stv680_start_stream+0x18b/0x1c5
>      >      >    Â [<ffffffff881397a0>] :stv680:stv680_do_ioctl+0x4e1/0x57b
>      >      >    Â [<ffffffff8811ddda>] :videodev:video_usercopy+0x1a1/0x265
>      >      >    Â [<ffffffff881392bf>] :stv680:stv680_do_ioctl+0x0/0x57b
>      >      >    Â [<ffffffff8031a360>] inode_has_perm+0x56/0x63
>      >      >    Â [<ffffffff80222789>] __up_read+0x19/0x7f
>      >      >    Â [<ffffffff802676cd>] do_page_fault+0xfae/0x12e0
>      >      >    Â [<ffffffff8026ef47>] monotonic_clock+0x35/0x7b
>      >      >    Â [<ffffffff80243f5c>] do_ioctl+0x55/0x6b
>      >      >    Â [<ffffffff802316b5>] vfs_ioctl+0x457/0x4b9
>      >      >    Â [<ffffffff8024e68e>] sys_ioctl+0x59/0x78
>      >      >    Â [<ffffffff802602f9>] tracesys+0xab/0xb6
>      >      >    Code: 0f 0b 68 3a 2b 49 80 c2 8a 01 4d 85 ed 74 11 49 8b 85
>      18 02Â
>      >      >    RIP Â [<ffffffff8027217b>] dma_map_single+0x12d/0x17d
>      >      >    Â RSP <ffff88000cfb1c78>
>      >      >    Â <0>Kernel panic - not syncing: Fatal exception

>      >      >    2010/3/2 Jan Ä*eÅ¡Ä*ut <[1][2][3]Jan.Cescut@xxxxxx>

>      >      >
>      >      >      Thanks for thorough explanation.
>      >      >
>      >      >      Have a nice day,
>      >      >      Jan
>      >      >      -----Original Message-----

>      >      >      From: Pasi KÃ*rkkÃ*inen [mailto:[2][3][4]pasik@xxxxxx]
>      >      >      Sent: 27. februar 2010 14:04
>      >      >      To: Jan Ä*eÅ¡Ä*ut

>      >      >      Cc: [3][4][5]xen-users@xxxxxxxxxxxxxxxxxxx
>      >      >      Subject: Re: [Xen-users] PCI Passthrough without VT-d
>      >      >
>      >      >      On Fri, Feb 26, 2010 at 11:29:22PM +0100, Jan ?eÅ¡?ut
>      wrote:
>      >      >      > Â  Â As I read XEN supports assigning a pci device to an
>      >      unprivileged
>      >      >      domain
>      >      >      > Â  Â without hardware supporting it. Has anyone already
>      tried
>      >      it? Are
>      >      >      there any
>      >      >      > Â  Â security risks? If I understand correctly how PCI
>      >      passthrough
>      >      >      works the
>      >      >      > Â  Â performance should be the same as using the pci
>      device in
>      >      native
>      >      >      mode. Is
>      >      >      > Â  Â it so? I have a PCI video card which would like to
>      use
>      >      inside a
>      >      >      VM running
>      >      >      > Â  Â Windows XP.
>      >      >      >
>      >      >
>      >      >      Xen supports PCI passthrough to _PV_ (paravirtual) guests
>      without
>      >      VT-d,
>      >      >      and has actually supported that for years. There are some
>      >      potential
>      >      >      security
>      >      >      risks in this, since the PV guest gets full DMA control of
>      the
>      >      PCI
>      >      >      device
>      >      >      and could use it for malicious purposes.
>      >      >
>      >      >      Xen PCI passthrough to HVM guests (=Windows) requires VT-d
>      >      hardware
>      >      >      support.
>      >      >
>      >      >      Also, PCI passthrough of a VGA/video card is not as simple
>      as PCI
>      >      >      passthrough
>      >      >      of other cards (nic, disk controller, usb controller).
>      >      >
>      >      >      VGA has lots of legacy stuff related to it, some memory
>      ranges,
>      >      IO
>      >      >      ports, VGA BIOS,
>      >      >      etc that have to be 'passed through' aswell, and emulated.
>      >      >
>      >      >      Xen 4.0.0 will have PCI passthrough support of primary VGA
>      >      adapters, but
>      >      >      it requires
>      >      >      VT-d support as stated already earlier.
>      >      >
>      >      >      -- Pasi
>      >      >
>      >      >      ps. There is actually a hack/patch available that allows
>      PCI
>      >      passthrough
>      >      >      to HVM guest
>      >      >      without VT-d, but that only works for the _first_ started
>      HVM
>      >      guest, and
>      >      >      it's experimental
>      >      >      and not supported in any way. iirc the patch is available
>      in
>      >      xen-devel
>      >      >      archives.
>      >      >
>      >      >      _______________________________________________
>      >      >      Xen-users mailing list

>      >      >      [4][5][6]Xen-users@xxxxxxxxxxxxxxxxxxx
>      >      >      [5][6][7]http://lists.xensource.com/xen-users

>      >      >
>      >      >    --
>      >      >    Sergio Roberto Charpinel Jr.
>      >      >
>
>    --
>    Sergio Roberto Charpinel Jr.
>

> References
>
>    Visible links
>    1. mailto:pasik@xxxxxx

>    2. mailto:pasik@xxxxxx

>    3. mailto:Jan.Cescut@xxxxxx
>    4. mailto:pasik@xxxxxx
>    5. mailto:xen-users@xxxxxxxxxxxxxxxxxxx
>    6. mailto:Xen-users@xxxxxxxxxxxxxxxxxxx
>    7. http://lists.xensource.com/xen-users

--
Sergio Roberto Charpinel Jr.

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users