Re: [Xen-devel] 2.6.32.16: hvm domU boot - oops at add_pin_to_irq_node_nopanic

[Bruce Edge <bruce.edge@xxxxxxxxx>]

On Fri, Jul 9, 2010 at 3:37 AM, Stefano Stabellini <stefano.stabellini@xxxxxxxxxxxxx> wrote:

On Thu, 8 Jul 2010, Bruce Edge wrote:
> On Thu, Jul 8, 2010 at 10:04 AM, Stefano Stabellini <stefano.stabellini@xxxxxxxxxxxxx> wrote:
> � � � On Thu, 8 Jul 2010, Bruce Edge wrote:
> � � � > On Thu, Jul 8, 2010 at 9:55 AM, Stefano Stabellini <stefano.stabellini@xxxxxxxxxxxxx> wrote:
> � � � > � � � On Thu, 8 Jul 2010, Bruce Edge wrote:
> � � � > � � � > This is the same problem as this post:
> � � � > � � � > http://lists.xensource.com/archives/html/xen-devel/2010-06/msg01114.html
> � � � > � � � >
> � � � > � � � > Stefano suggested I retry after Jeremy's pull.
> � � � > � � � >
> � � � > � � � > Sync'd with 2.6.32.16 last night and it looks like the identical problem.
> � � � > � � � >
> � � � > � � � > I can boot Ubuntu 10.04 with it's own native kernel, but not with a pv-ops kernel.
> � � � > � � � >
> � � � > � � � > This is with # CONFIG_XEN_PLATFORM_PCI is not set, .config is attached.
> � � � > � � � >
> � � � >
> � � � > I think that Jeremy didn't merge the up-to-date PV on HVM branch in
> � � � > stable-2.6.32.x but only in next.
> � � � >
> � � � >
> � � � > I thought that the PV on HVM code was only used if�CONFIG_XEN_PLATFORM_PCI=y.�
> � � � > Is that not the case?
> � � � >
>
> For the most part yes, however the pv timer can still work even without
> the platform pci driver and with the pv timer enabled you need another
> patch to prevent the ioapic initialization code from breaking. This last
> patch is missing from stable-2.6.32.x.
>
>
> Could you give me a pointer to this patch? I'd like to test it to confirm whether it is indeed the problem.
>

you can checkout my tree from here (vanilla 2.6.34 + pv on hvm patches):

git://xenbits.xen.org/people/sstabellini/linux-pvhvm.git

branch name 2.6.34-pvhvm-v4.

The patches that you need to have are:

- "__setup_vector_irq: handle NULL chip_data"
32b5ffd98391cb47c3cc9459c445d9490a523622

- "Fix possible NULL pointer dereference in print_IO_APIC"
a57e90eaf9c55da55e3de09ebda99cb2f8a00e40

- "Fix find_unbound_irq in presence of ioapic irqs."
d7fc454bce7b63aab43ef0e11d97b206082e4740

The above patch set didn't all apply to xen/stable-2.6.32x Some were already applied and others needed additional patches:

BTW I have a new version of the branch (2.6.34-pvhvm-v6) without those
patches because I was able to fix the issue in a better way, but I
haven't sent it to the list yet. Jeremey's PV on HVM branch is based on
v4.

So I tried the same hvm boot using the following 3 branches.�

�� �xen/next-2.6.32

Stefano's

�� �2.6.34-pvhvm-v4�

�� �2.6.34-pvhvm-v6

The base xen 4.0.1rc3 did not have the patch Stefano posted later in this thread.

The boot summary is inline and the complete bootlogs are attached. It has a few extra bits required by the .34 kernel, but is essentially the same one for all 3 cases.

The gist of it being:

CONFIG_XEN=y

CONFIG_XEN_MAX_DOMAIN_MEMORY=32

CONFIG_XEN_SAVE_RESTORE=y

# CONFIG_XEN_DEBUG_FS is not set

CONFIG_XEN_BLKDEV_FRONTEND=m

CONFIG_NETXEN_NIC=m

CONFIG_XEN_NETDEV_FRONTEND=m

CONFIG_XEN_KBDDEV_FRONTEND=m

CONFIG_HVC_XEN=y

CONFIG_XEN_FBDEV_FRONTEND=m

CONFIG_XEN_BALLOON=y

CONFIG_XEN_SCRUB_PAGES=y

CONFIG_XEN_DEV_EVTCHN=m

CONFIG_XENFS=m

CONFIG_XEN_COMPAT_XENFS=y

CONFIG_XEN_SYS_HYPERVISOR=y

# CONFIG_XEN_PLATFORM_PCI is not set

The .config is also attached

1) xen/next-2.6.32 - Hangs at the end of the following sequence:

[ � �0.464686] ACPI: bus type pci registered^M

[ � �0.467109] PCI: Using configuration type 1 for base access^M

[ � �0.470495] bio: create slab <bio-0> at 0^M

[ � �0.523248] ACPI: Interpreter enabled^M

[ � �0.525249] ACPI: (supports S0 S3 S4 S5)^M

[ � �0.527628] ACPI: Using IOAPIC for interrupt routing^M

[ � �0.624323] ACPI: No dock devices found.^M

[ � �0.626607] ACPI: PCI Root Bridge [PCI0] (0000:00)^M

[ � �0.635170] * Found PM-Timer Bug on the chipset. Due to workarounds for a bug,^M

[ � �0.635170] * this clock source is slow. Consider trying other clock sources^M

[ � �0.641158] pci 0000:00:01.3: quirk: region 1f40-1f7f claimed by PIIX4 ACPI^M

[ � �1.115247] ACPI: PCI Interrupt Link [LNKA] (IRQs *5 10 11)^M

[ � �1.118966] ACPI: PCI Interrupt Link [LNKB] (IRQs 5 *10 11)^M

[ � �1.121346] ACPI: PCI Interrupt Link [LNKC] (IRQs 5 10 *11)^M

[ � �1.125128] ACPI: PCI Interrupt Link [LNKD] (IRQs *5 10 11)^M

<hang>

2)�2.6.34-pvhvm-v4 - boots, with a lot of call traces on the serial console:

[ � �1.880528] WARNING: at arch/x86/xen/mmu.c:1953 xen_hvm_exit_mmap+0x45/0x50()^M

[ � �1.884168] Hardware name: HVM domU^M

[ � �1.885967] Modules linked in:^M

[ � �1.887638] Pid: 48, comm: init Not tainted 2.6.34 #1^M

[ � �1.890384] Call Trace:^M

[ � �1.891721] �[<ffffffff8105cefb>] warn_slowpath_common+0x7b/0xc0^M

[ � �1.894804] �[<ffffffff8105cf54>] warn_slowpath_null+0x14/0x20^M

[ � �1.897820] �[<ffffffff81005015>] xen_hvm_exit_mmap+0x45/0x50^M

[ � �1.900941] �[<ffffffff8110f7b0>] exit_mmap+0x50/0x190^M

[ � �1.903569] �[<ffffffff8105a5e2>] mmput+0x42/0x110^M

[ � �1.906059] �[<ffffffff811443a3>] flush_old_exec+0x463/0x620^M

[ � �1.908995] �[<ffffffff8118436e>] load_elf_binary+0x38e/0x1d20^M

[ � �1.912375] �[<ffffffff81108ba6>] ? follow_page+0x2d6/0x350^M

[ � �1.915210] �[<ffffffff81108ba6>] ? follow_page+0x2d6/0x350^M

[ � �1.918055] �[<ffffffff8110d7cf>] ? __get_user_pages+0x10f/0x430^M

[ � �1.921309] �[<ffffffff8114386c>] ? get_arg_page+0x5c/0xc0^M

[ � �1.924127] �[<ffffffff8114528d>] search_binary_handler+0xed/0x310^M

[ � �1.927336] �[<ffffffff811458e3>] do_execve+0x333/0x410^M

[ � �1.930170] �[<ffffffff812dff58>] ? strncpy_from_user+0x38/0x60^M

[ � �1.933261] Clockevents: could not switch to one-shot mode: lapic is not functional.^M

[ � �1.937318] Could not switch to high resolution mode on CPU 0^M

[ � �1.940399] �[<ffffffff810135da>] sys_execve+0x4a/0x80^M

[ � �1.943078] �[<ffffffff8100b48a>] stub_execve+0x6a/0xc0^M

[ � �1.945737] ---[ end trace 7264d4229303d265 ]---^M

but eventually gets to a login and appears functional, but the number of call traces coming out during the boot are not comforting.

3)�2.6.34-pvhvm-v6 - Same as v4

However the root fs of the domU was corrupted after this last test. Possibly the result of using partprobe to get at lvm block device partitions and mount them to install new kernels, but I had been doing that for a while with no ill effects.

Please advise, is there any other pv-ops kernel branch suitable for domU hvm operation or other patch set I should try?

Dom0 modules:

#> lsmod

Module � � � � � � � � �Size �Used by

nf_conntrack_ipv4 � � �12980 �0�

nf_defrag_ipv4 � � � � �1481 �1 nf_conntrack_ipv4

xt_state � � � � � � � �1490 �0�

nf_conntrack � � � � � 74126 �2 nf_conntrack_ipv4,xt_state

xt_physdev � � � � � � �1739 �0�

iptable_filter � � � � �2791 �0�

ip_tables � � � � � � �18422 �1 iptable_filter

x_tables � � � � � � � 22493 �3 xt_state,xt_physdev,ip_tables

ipmi_msghandler � � � �37147 �0�

nfs � � � � � � � � � 316736 �1�

xenfs � � � � � � � � �11748 �1�

xen_evtchn � � � � � � �5670 �1�

pci_stub � � � � � � � �1598 �0�

bridge � � � � � � � � 53280 �0�

stp � � � � � � � � � � 2203 �1 bridge

ppdev � � � � � � � � � 6407 �0�

lp � � � � � � � � � � �9336 �0�

parport_pc � � � � � � 30246 �1�

psmouse � � � � � � � �64320 �0�

parport � � � � � � � �37744 �3 ppdev,lp,parport_pc

ioatdma � � � � � � � �42609 �24�

serio_raw � � � � � � � 4982 �0�

joydev � � � � � � � � 11296 �0�

dca � � � � � � � � � � 6701 �1 ioatdma

usbhid � � � � � � � � 41404 �0�

hid � � � � � � � � � �83408 �1 usbhid

usb_storage � � � � � �50089 �0�

e1000e � � � � � � � �136653 �0�

floppy � � � � � � � � 63348 �0�

#> cat /proc/misc�

�53 xen/evtchn

�54 network_throughput

�55 network_latency

�56 cpu_dma_latency

�57 device-mapper

��1 psaux

200 tun

�58 pktcdvd

228 hpet

�59 blktap-control

�60 xen/gntdev

229 fuse

�61 ecryptfs

231 snapshot

227 mcelog

�62 rfkill

Let me know if there's anything else I can provide, or other test config to run.

Thanks.

-Bruce

screenlog.pvhvm-v4
Description: Binary data

screenlog.pvhvm-v6
Description: Binary data

screenlog.xen-next
Description: Binary data

linux-2.6-pvhvm-v6.config
Description: Binary data

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel