[Top] [All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Xen-devel] Re: Testing status of HVM (Intel VT) on 64bit XEN unstable c/s 11731

Sorry, missed the attachements

Ed Smith wrote:

Summary:
Changeset 11731
- NEW: Most of the shadow crashes appear to be fixed,
  mtest01 -p80" runs for much longer on a 32bit HVM SMP
  (RHEL4U2) guest before hitting:
  Error: shadow lock held by shadow_update_cr3 (failure.3)

Test Configuration:
Dell Precision WorkStation 380, Dual Core, 2GB, 3 SATA (Intel VT)
64bit XEN RELEASE Build Hypervisor on a RHEL4U2 64bit root (/dev/sda)
32bit fully virtualized (HVM) guest RHEL4U2 256MB (/dev/sdb)
      pae=1(smp) pae=0(up), acpi=1, apic=1
      kernargs noapic
64bit fully virtualized (HVM) guest RHEL4U2 256MB (/dev/sdc)
      pae=1, acpi=1, apic=1
      kernargs noapic

Boot Tests:
Boot a fully virtualized (HVM) guest to the login prompt
Results are marked Pass|Fail where (n) points to a failure description

Regression Tests:
852 tests (851 ltp tests and one 30 minute user load test)
Tests are marked #Pass/#Fail where (n) points to a failure description

XEN 64bit 2 CPU Hypervisor (booted smp):
 ----------------------------------------------------------------------
| XEN      | Guest Kernel (SMP kernels booted with 2 CPUs)             |
| Changeset|-----------------------------------------------------------|
|          | 32bit UP     | 32bit SMP    | 64bit UP     | 64bit SMP    |
|          |--------------|--------------|--------------|--------------|
|          | Boot | Test  | Boot | Test  | Boot | Test  | Boot | Test  |
|----------|------|-------|------|-------|------|-------|------|-------|
| 11731    | Pass |       | Pass | 851/1 | Pass |       | Pass | 852/0 |
|          |      |       |      |(1,3)  |      |       |      |       |
|----------|------|-------|------|-------|------|-------|------|-------|
| 11690    | Pass |       | Pass | 849/3 | Pass |       | Pass | 852/0 |
|          |      |       |      |(1,3)  |      |       |      |       |
|----------|------|-------|------|-------|------|-------|------|-------|
| 11686    | Pass |       | Pass | 850/2 | Pass |       | Pass | 852/0 |
|          |      |       |      |(1,3,8)|      |       |      |       |
|----------|------|-------|------|-------|------|-------|------|-------|
| 11655    | Pass |       | Pass | 850/2 | Fail |       | Fail |       |
|          |      |       |      |(1,4)  | (7)  |       | (7)  |       |
|----------|------|-------|------|-------|------|-------|------|-------|
| 11616    | Pass |       | Fail |       | Fail |       | Fail |       |
|          |      |       | (6)  |       | (5)  |       | (5)  |       |
 ----------------------------------------------------------------------

Multiple Guest Boot Test
Test is a 30 minute user load on both Guests
XEN 64bit 2 CPU Hypervisor (booted smp):
 --------------------------------------------
| XEN      | Guest Kernel                    |
| Changeset|---------------------------------|
|          | 32bit 1CPU UP  | 32bit 2CPU SMP |
|          | 64bit 1CPU UP  | 64bit 2CPU SMP |
|          |----------------|----------------|
|          | Boot | Test    | Boot | Test    |
|----------|------|---------|------|---------|
| 11731    | Pass | Pass    | Pass | Pass    |
|          |      |         |      |         |
|----------|------|---------|------|---------|
| 11690    | Pass | Pass    | Pass | Pass    |
|          |      |         |      |         |
|----------|------|---------|------|---------|
| 11686    | Pass | Pass    | Pass | Pass    |
|          |      |         |      |         |
|----------|------|---------|------|---------|
| 11655    | Fail |         | Fail |         |
|          | (7)  |         | (7)  |         |
|----------|------|---------|------|---------|
| 11616    | Fail |         | Fail |         |
|          | (5)  |         | (5)  |         |
 --------------------------------------------

Failures:
 1. BUG 666: 32bit guests fail ltp gettimeofday02
    and nanosleep01/02 with clock problems
 2. NOT REPRODUCED: XEN crash on an xm destroy of a dead guest
    32bit SMP HVM guest:
    Fatal page fault - put_page_from_l1e+0x85/0x140
 3. XEN crash running ltp "mtest01 -p80" on 32bit SMP HVM guest:
    BUG at multi.c:2967 from sh_page_fault__shadow_3_guest_3
 4. FIXED: XEN crash running ltp "mtest01 -p80" on 32bit SMP HVM guest:
    BUG at multi.c:4082 from sh_clear_shadow_entry__shadow_3_guest_3
 8. FIXED: Domain crash running ltp "mtest01 -p80" on 32bit SMP
    HVM guest and then XEN locked up requiring a hard reset:
    domain_crash_sync called from private.h:495




File: failure.1

[root@tst110 ~]# xm info
host                   : tst110
release                : 2.6.16.13-xen
version                : #1 SMP Tue Sep 19 04:39:19 EDT 2006
machine                : x86_64
nr_cpus                : 2
nr_nodes               : 1
sockets_per_node       : 1
cores_per_socket       : 2
threads_per_core       : 1
cpu_mhz                : 2793
hw_caps                : 
bfebfbff:20100800:00000000:00000180:0000e43d:00000000:00000001
total_memory           : 1022
free_memory            : 0
xen_major              : 3
xen_minor              : 0
xen_extra              : -unstable
xen_caps               : xen-3.0-x86_64 hvm-3.0-x86_32 hvm-3.0-x86_32p 
hvm-3.0-x86_64
xen_pagesize           : 4096
platform_params        : virt_start=0xffff800000000000
xen_changeset          : Mon Sep 18 18:17:54 2006 +0100 11486:d8bceca5f07d
cc_compiler            : gcc version 4.0.0 20050519 (Red Hat 4.0.0-8)
cc_compile_by          : build
cc_compile_domain      : virtualiron.com
cc_compile_date        : Tue Sep 19 03:53:38 EDT 2006
xend_config_format     : 2
[root@tst110 ~]# ssh root@vs161
root@vs161's password:
Last login: Mon Sep 18 14:44:58 2006 from 10.1.2.13
[root@vs161 ~]# cd /qa/conductor/tests/ltp/testcases/bin/
[root@vs161 bin]# ./gettimeofday02
gettimeofday02    0  INFO  :  checking if gettimeofday is monotonous, takes 30s
gettimeofday02    1  FAIL  :  Time is going backwards (old 1158687174.446647 vs 
new 1158687174.446103!
[root@vs161 bin]# ./gettimeofday02
gettimeofday02    0  INFO  :  checking if gettimeofday is monotonous, takes 30s
gettimeofday02    1  PASS  :  gettimeofday monotonous in 30 seconds
[root@vs161 bin]# ./gettimeofday02
gettimeofday02    0  INFO  :  checking if gettimeofday is monotonous, takes 30s
gettimeofday02    1  PASS  :  gettimeofday monotonous in 30 seconds
[root@vs161 bin]# ./gettimeofday02
gettimeofday02    0  INFO  :  checking if gettimeofday is monotonous, takes 30s
gettimeofday02    1  FAIL  :  Time is going backwards (old 1158687256.569152 vs 
new 1158687256.569137!
[root@vs161 bin]# uname -a
Linux vs161 2.6.9-22.ELsmp #1 SMP Mon Sep 19 18:32:14 EDT 2005 i686 i686 i386 
GNU/Linux
[root@vs161 bin]# cat /proc/cpuinfo | grep -i processor
processor       : 0
processor       : 1
[root@vs161 bin]#                                                               
 

File: failure.3







tst110 login: 
Red Hat Enterprise Linux ES release 4 (Nahant Update 2)
Kernel 2.6.16.29-xen on an x86_64

tst110 login: root
Password:  
Last login: Wed Oct  4 17:08:44 on ttyS0
You have new mail.

[root@tst110 ~]#  xm info
host                   : tst110
release                : 2.6.16.29-xen
version                : #1 SMP Tue Oct 10 01:27:09 EDT 2006
machine                : x86_64
nr_cpus                : 2
nr_nodes               : 1
sockets_per_node       : 1
cores_per_socket       : 2
threads_per_core       : 1
cpu_mhz                : 2793
hw_caps                : 
bfebfbff:20100800:00000000:00000180:0000e43d:00000000:00000001
total_memory           : 1022
free_memory            : 0
xen_major              : 3
xen_minor              : 0
xen_extra              : -unstable
xen_caps               : xen-3.0-x86_64 hvm-3.0-x86_32 hvm-3.0-x86_32p 
hvm-3.0-x86_64
xen_pagesize           : 4096
platform_params        : virt_start=0xffff800000000000
xen_changeset          : Mon Oct 09 23:34:00 2006 +0100 11731:ab85c20d33e5
cc_compiler            : gcc version 4.0.0 20050519 (Red Hat 4.0.0-8)
cc_compile_by          : build
cc_compile_domain      : virtualiron.com
cc_compile_date        : Tue Oct 10 00:48:10 EDT 2006
xend_config_format     : 2
[root@tst110 ~]# xm list
Name                                      ID Mem(MiB) VCPUs State   Time(s)
Domain-0                                   0      721     2 r-----   3780.0
vs161                                      1      256     2 -b----  10878.7
[root@tst110 ~]#                         (XEN) Error: shadow lock held by 
shadow_update_cr3
(XEN) BUG at multi.c:2967
(XEN) ----[ Xen-3.0-unstable  x86_64  debug=n  Not tainted ]----
(XEN) CPU:    0
(XEN) RIP:    e010:[<ffff830000118364>] __bug+0x24/0x30
(XEN) RFLAGS: 0000000000010286   CONTEXT: hypervisor
(XEN) rax: 0000000000000000   rbx: 0000000000000b97   rcx: 00000000000021be
(XEN) rdx: 000000000000000a   rsi: 000000000000000a   rdi: ffff8300001beff4
(XEN) rbp: ffff830000198080   rsp: ffff8300001b7ab8   r8:  0000000000000000
(XEN) r9:  0000000000000020   r10: 0000000000000004   r11: ffff830000118a20
(XEN) r12: ffff83000017646d   r13: 0000000000000000   r14: ffff830000198be8
(XEN) r15: ffff830000198b98   cr0: 000000008005003b   cr4: 00000000000026f0
(XEN) cr3: 000000002941b000   cr2: 0000000000000000
(XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: 0000   cs: e010
(XEN) Xen stack trace from rsp=ffff8300001b7ab8:
(XEN)    ffff8300001b7cd8 ffff8300001a2080 ffff8300001b7cd8 ffff83000015820a
(XEN)    ffff8300001b7cd8 0000000000000000 000000000000006e ffff830000189280
(XEN)    ffff830010d89008 ffff8300001a2080 ffff830010d89008 ffff83000013e210
(XEN)    ffff83000013f010 ffff830000139ddb 0000000000000000 00000000c0117346
(XEN)    ffff8300001a2080 ffff8300001487e7 00000000c011733c ffff830010d89008
(XEN)    ffff8300001a2080 0000000000000004 0000000000000001 0000000000000000
(XEN)    ffff8300001a36b0 ffff83000013b7d9 0000000000029006 ffffffffffffffff
(XEN)    0000000000004008 ffff8300001b7c58 ffff830010d89008 ffff83000013b330
(XEN)    0000000100000004 ffff8300001b7c58 0000000000000001 00000000fee000b0
(XEN)    0000000000000000 ffff8300001b7c58 ffff8300001a3690 ffff83000013bd70
(XEN)    0000000000000000 0000000000016308 0000000000000004 ffff8300001a36b0
(XEN)    000000000000000a ffff83000013cee4 0000000000000000 0000000000000000
(XEN)    00000000fee000b0 00000000ffffd0b0 ffff830000198080 ffff8300001a2080
(XEN)    0000000000040006 ffff830000198080 ffff8300001b7f28 ffff8300001b7cd8
(XEN)    0000000000000000 ffff830000198be8 ffff830000198b98 ffff83000012fb40
(XEN)    ffff830000198080 ffff8300001a2080 ffff8300001b7f28 ffff830000000000
(XEN)    ffff830000198080 ffff830000198be8 ffff830000198b98 ffff830000161495
(XEN)    ffff830000198b98 ffff830000198be8 ffff830000198080 ffff830000000000
(XEN)    ffff8300001b7f28 ffff8300001a2080 0000000000000001 cccccccccccccccd
(XEN)    ffff828400377990 0000000000000000 ffff8140a0400000 0000000000000000
(XEN) Xen call trace:
(XEN)    [<ffff830000118364>] __bug+0x24/0x30
(XEN)    [<ffff83000015820a>] sh_page_fault__shadow_3_guest_3+0x6a/0xec0
(XEN)    [<ffff83000013e210>] vlapic_read+0x0/0x330
(XEN)    [<ffff83000013f010>] vlapic_write+0x0/0xa00
(XEN)    [<ffff830000139ddb>] hvm_mmio_intercept+0x11b/0x360
(XEN)    [<ffff8300001487e7>] vmx_load_cpu_guest_regs+0x307/0x310
(XEN)    [<ffff83000013b7d9>] send_mmio_req+0x1b9/0x230
(XEN)    [<ffff83000013b330>] hvm_io_assist+0xa10/0xa50
(XEN)    [<ffff83000013bd70>] mmio_operands+0xb0/0x160
(XEN)    [<ffff83000013cee4>] handle_mmio+0x10c4/0x1350
(XEN)    [<ffff83000012fb40>] do_page_fault+0x210/0x320
(XEN)    [<ffff830000161495>] handle_exception+0x45/0x6e
(XEN)    [<ffff830000156010>] sh_pae_recopy+0xb0/0x130
(XEN)    [<ffff83000014e181>] shadow_prealloc+0x1e1/0x310
(XEN)    [<ffff83000015751e>] sh_update_cr3__shadow_3_guest_3+0x18e/0x480
(XEN)    [<ffff83000014a7dc>] vmx_cr_access+0x6fc/0x1080
(XEN)    [<ffff83000014b8c0>] vmx_vmexit_handler+0x760/0x1150
(XEN)    [<ffff83000013fa7a>] cpu_has_apic_interrupt+0x1a/0x50
(XEN)    [<ffff83000013fae1>] cpu_has_pending_irq+0x31/0x50
(XEN)    [<ffff830000145648>] vmx_intr_assist+0xf8/0x410
(XEN)    [<ffff83000014c2d8>] vmx_asm_vmexit_handler+0x28/0x30
(XEN)    
(XEN) 
(XEN) ****************************************
(XEN) Panic on CPU 0:
(XEN) CPU0 FATAL TRAP: vector = 6 (invalid opcode)
(XEN) ****************************************
(XEN) 
(XEN) Reboot in five seconds...

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel

 

©2013 Xen Project, A Linux Foundation Collaborative Project. All Rights Reserved.
Linux Foundation is a registered trademark of The Linux Foundation.
Xen Project is a trademark of The Linux Foundation.

Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.