On 03/18/10 14:38, Pasi Kärkkäinen wrote:
> On Thu, Mar 18, 2010 at 12:46:11PM -0700, Andrew Evans wrote:
>
>> I've been beating my head against the wall trying to get stub domains
>> working with PV on HVM guests running RHEL 5. Up to now I've been using
>> the RHEL 5 dom0 kernel, but that doesn't work with stub domains because
>> gntdev isn't built in (and can't be built as a module either -- I
>> tried). So I built the 2.6.18.8 dom0 kernel from xen-unstable.hg. That
>> got stub domains working with RHEL 5 HVM guests, but PV on HVM guests
>> fail to find the root filesystem when initrd gets ready to pivot to the
>> real root filesystem. PV on HVM without stub domain has the same problem.
>>
>> --- begin domU console output ---
>> Creating block device nodes.
>> Loading xen-platform-pci.ko module
>> GSI 16 sharing vector 0xA9 and IRQ 16
>> ACPI: PCI Interrupt 0000:00:03.0[A] -> GSI 28 (level, low) -> IRQ 169
>> Xen version 3.4.
>> Hypercall area is 1 pages.
>> Grant table initialized
>> Loading xen-vbd.ko module
>> Loading xen-balloon.ko module
>> xen_mem: Initialising balloon driver.
>> Loading xen-vnif.ko module
>> netfront: Initialising virtual ethernet driver.
>> Loading jbd.ko module
>> netfront: device eth0 has copying receive path.
>> Loading ext3.ko module
>> Loading xen-vbd.ko module
>> insmod: error inserting '/lib/xen-vbd.ko': -1 File exists
>> Loading xen-vbd.ko module
>> insmod: error inserting '/lib/xen-vbd.ko': -1 File exists
>> Trying to resume from /dev/xvda2
>> Unable to access resume device (/dev/xvda2)
>>
>
> Unable to access... hmm.
>
>
>
>> Creating root device.
>> Registering block device major 202
>> xvda:Mounting root filesystem.
>> xvda1 xvda2 xvda3
>> mount: could not find filesystem '/dev/root'
>>
> I guess it's the same problem here..
>
>
>
>> Setting up other filesystems.
>> Setting up new root fs
>> setuproot: moving /dev failed: No such file or directory
>> no fstab.sys, mounting internal defaults
>> setuproot: error mounting /proc: No such file or directory
>> setuproot: error mounting /sys: No such file or directory
>> Switching to new root and running init.
>> unmounting old /dev
>> unmounting old /proc
>> unmounting old /sys
>> switchroot: mount failed: No such file or directory
>> Kernel panic - not syncing: Attempted to kill init!
>> --- end domU console output ---
>>
>> Notice that the PV disks and their partitions are detected.
>>
>>
> Yeah, but they don't work for some reason..
>
...until the system finishes booting (if I remove 'hda=noprobe
ide0=noprobe' from domU kernel command line, allowing initrd/nash to
mount disks via /dev/hd*)
>> Kernel command line is root=LABEL=ROOT, but root=/dev/xvda3 doesn't work
>> either.
>>
>> This same VM boots fine if I reboot the hypervisor with the RHEL 5 dom0
>> kernel.
>>
>> /var/log/xen/* show nothing out of the ordinary. The only hint of a
>> problem are these lines from xm dmesg:
>>
>> (XEN) event_channel.c:192:d0 EVTCHNOP failure: domain 13, error -22
>> (XEN) event_channel.c:192:d0 EVTCHNOP failure: domain 13, error -22
>> (XEN) event_channel.c:192:d0 EVTCHNOP failure: domain 13, error -22
>>
>>
> These sound related..
>
Yup. I can't believe event channels are generally broken though, as the
PV network driver works and the PV disk driver works once the system
comes up.
>> If I boot this same VM without disabling the IDE drivers, it mounts
>> filesystems using /dev/hd* and starts up successfully. Now here's the
>> baffling part: I can then mount / and /home using /dev/xvd*! So the
>> paravirt disk driver *is* working once the domU boots off /dev/hd*, but
>> for some reason it doesn't work when I try to boot from /dev/xvd* directly.
>>
>> So my questions are:
>>
>> 1. Why can't the initrd mount /dev/xvd* to / ?
>>
>>
> Because the devices don't work for some reason..
>
Well, yeah, I had worked that much out on my own. :) I can't imagine
what miracle is occurring between the time that initrd/nash tries to
mount /dev/xvda3 to / and when I do it manually from the command line
once I'm able to to log in, but such a miracle is in fact occurring!
>> 2. Why does this work with the RHEL 5 dom0 kernel but not with a
>> similarly-configured xen-unstable 2.6.18.8 dom0 kernel?
>>
>>
> That's a good question.
>
>
>> xen-unstable 2.6.18.8 dom0 kernel config attached. I've gotten the
>> Xen-related stuff as close to RHEL 5 as I can, but available CONFIG_*
>> defines vary somewhat between the two kernel source trees.
>>
>> Thanks for any help anyone can provide.
>>
>>
> So the dom0 kernel was the only thing you changed?
>
Yes.
> You used the same Xen hypervisor for both dom0 kernels? which hypervisor
> version?
>
Xen 3.4.2, rebuilt from gitco source RPM w/oprofile active domains patch
applied:
$ sudo xm info
host : <snip>
release : 2.6.18.8-xen
version : #7 SMP Wed Mar 17 18:36:46 UTC 2010
machine : x86_64
nr_cpus : 8
nr_nodes : 1
cores_per_socket : 4
threads_per_core : 1
cpu_mhz : 2500
hw_caps :
bfebfbff:20100800:00000000:00000140:000ce3bd:00000000:00000001:00000000
virt_caps : hvm
total_memory : 8190
free_memory : 7266
node_to_cpu : node0:0-7
node_to_memory : node0:7266
xen_major : 3
xen_minor : 4
xen_extra : .2
xen_caps : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32
hvm-3.0-x86_32p hvm-3.0-x86_64
xen_scheduler : credit
xen_pagesize : 4096
platform_params : virt_start=0xffff800000000000
xen_changeset : unavailable
cc_compiler : gcc version 4.1.2 20080704 (Red Hat 4.1.2-44)
cc_compile_by : root
cc_compile_domain : <snip>
cc_compile_date : Wed Jan 27 00:36:30 UTC 2010
xend_config_format : 4
thanks,
-Andrew
_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users
|
Home