WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-devel

Re: [Xen-devel] pv-grub boot hangs when iommu=soft. Booting kernel direc

To: listmail <listmail@xxxxxxxxxxxx>, samuel.thibault@xxxxxxxxxxxx
Subject: Re: [Xen-devel] pv-grub boot hangs when iommu=soft. Booting kernel directly with the flags works
From: Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx>
Date: Mon, 5 Apr 2010 18:18:24 -0400
Cc: xen-devel@xxxxxxxxxxxxxxxxxxx
Delivery-date: Mon, 05 Apr 2010 15:19:53 -0700
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <4BBA5E4B.5010307@xxxxxxxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
References: <4BBA5E4B.5010307@xxxxxxxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
User-agent: Mutt/1.5.19 (2009-01-05)
On Mon, Apr 05, 2010 at 06:03:55PM -0400, listmail wrote:
> Per request of Konrad I am forwarding details on an issue I ran into.   

Thanks.

Samuel,

It looks to be be related to the arguments being passed in to pvgrub.

With "extra='root=UUID=2bb30c38-70fc-4e9d-ae69-52db68589a2e ro iommu=soft 
swiotlb=force"
it fails. If the 'iommu=soft swiotlb=force' is removed it boots. I've
asked the user to move the 'ro' before the 'root' but still had the same
issue.


> The goal here was non VT-D pci passthrough to a pv guest.  Konrad  
> provided a working solution as detailed at the end of this post.
>
> dom 0 grub
> menuentry "Xen 4.0.0-rc8 / Ubuntu 10.4 kernel 2.6.32.10-xen2 pvops" {
> insmod ext2
> set root='(hd0,2)'
> search --no-floppy --fs-uuid --set 321dedfa-be48-4b4d-bff0-0e5ecce240fd
> multiboot /boot/xen-4.0.0-rc8.gz dummy=dummy iommu=0
> module /boot/vmlinuz-2.6.32.10-xen2 dummy=dummy root=/dev/sdc2 ro  
> max_loop=64 xen-pciback.hide=(07:01.0)
> module /boot/initrd.img-2.6.32.10-xen2
> }
>
> # xm pci-list-assignable-devices
> 0000:07:01.0
>
> # dmesg |grep 07:01.0
> [    0.000000] Command line: root=/dev/sdc2 ro max_loop=64  
> xen-pciback.hide=(07:01.0)
> [    0.211959] Kernel command line: root=/dev/sdc2 ro max_loop=64  
> xen-pciback.hide=(07:01.0)
> [    0.520930] pci 0000:07:01.0: reg 10 32bit mmio: [0xf4000000-0xf7ffffff]
> [    0.558471] pciback 0000:07:01.0: seizing device
> [    0.609245] pciback 0000:07:01.0: PCI INT A -> GSI 16 (level, low) ->  
> IRQ 16
> [    0.609299] pciback 0000:07:01.0: PCI INT A disabled
> [  644.828556] pciback: vpci: 0000:07:01.0: assign to virtual slot 0
> [  731.747948] pciback: vpci: 0000:07:01.0: assign to virtual slot 0
> <note this shows up when i start the domU>
> [  731.748392] pciback 0000:07:01.0: device has been assigned to another  
> domain! Over-writting the ownership, but beware.
>
> at which point i can see card assigned:
> # xm pci-list lucidpv
> domain bus  slot func
> 0x0000 0x07 0x01 0x0
>
>
>
> domU grub
> title        Ubuntu lucid (development branch), kernel 2.6.32.10-xen2
> root        (hd0)
> kernel        /boot/vmlinuz-2.6.32.10-xen2  
> root=UUID=2bb30c38-70fc-4e9d-ae69-52db68589a2e ro iommu=soft 
> swiotlb=force
> initrd        /boot/initrd.img-2.6.32.10-xen2
>
>
> domU config
> kernel="/usr/lib/xen/boot/pv-grub-x86_64.gz"
> extra="(hd0)/boot/grub/menu.lst"
>
> name="lucidpv"
> memory=512
> maxmem=512
>
> vif=['bridge=xenbr0']
>
> disk = [ 'tap:aio:/etc/xen/images/lucidhvm.img,ioemu:xvda,w' ]
>
> pci = [ '07:01.0' ]
>
> on_reboot = 'destroy'
>
>
> domU output
> # xm create -c ubstub.cfg
> Using config file "/etc/xen/ubstub.cfg".
> Started domain lucidpv (id=2)
> Xen Minimal OS!
>  start_info: 0xa9a000(VA)
>    nr_pages: 0x20000
>  shared_inf: 0xbf44d000(MA)
>     pt_base: 0xa9d000(VA)
> nr_pt_frames: 0x9
>    mfn_list: 0x99a000(VA)
>   mod_start: 0x0(VA)
>     mod_len: 0
>       flags: 0x0
>    cmd_line: (hd0)/boot/grub/menu.lst
>  stack:      0x959980-0x979980
> MM: Init
>      _text: 0x0(VA)
>     _etext: 0x6987c(VA)
>   _erodata: 0x83000(VA)
>     _edata: 0x8bae0(VA)
> stack start: 0x959980(VA)
>       _end: 0x999f88(VA)
>  start_pfn: aa9
>    max_pfn: 20000
> Mapping memory range 0xc00000 - 0x20000000
> setting 0x0-0x83000 readonly
> skipped 0x1000
> MM: Initialise page allocator for ba3000(ba3000)-20000000(20000000)
> MM: done
> Demand map pfns at 20001000-2020001000.
> Heap resides at 2020002000-4020002000.
>  Booting command-list
>
> root  (hd0)
> Filesystem type is ext2fs, using whole disk
> kernel  /boot/vmlinuz-2.6.32.10-xen2  
> root=UUID=2bb30c38-70fc-4e9d-ae69-52db68589a
> 2e ro iommu=soft swiotlb=force
> initrd  /boot/initrd.img-2.6.32.10-xen2
>
> block error -2 for op 2
> close blk: backend=/local/domain/0/backend/vbd/2/51712 node=device/vbd/51712
> port 5 still bound!
>
>
> SOLUTION:
> Setting the following in the .cfg works:
>
> kernel = '/boot/vmlinuz-2.6.32.10-xen2'
> ramdisk= '/boot/initrd.img-2.6.32.10-xen2'
> extra='root=UUID=2bb30c38-70fc-4e9d-ae69-52db68589a2e ro iommu=soft  
> swiotlb=force'
>
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel