| |
|
 |
|
 |
|
|
|
| |
|
|
xen-devel
Re: [Xen-devel] [xen-unstable test] 2058: regressions - FAIL
On Wed, 2010-09-01 at 10:01 +0100, xen.org wrote:
> flight 2058 xen-unstable real
> http://www.chiark.greenend.org.uk/~xensrcts/logs/2058/
>
> Regressions :-(
>
> tests which did not succeed:
> test-amd64-amd64-pair 11 guest-start fail REGR. vs.
> 2053
2010-09-01 07:33:18 Z executing ssh ... root@xxxxxxxxxxxxx xm create
/etc/xen/debian.guest.osstest.cfg
Warning: Permanently added '10.80.249.102' (RSA) to the list of known
hosts.
Using config file "/etc/xen/debian.guest.osstest.cfg".
Started domain debian.guest.osstest (id=1)
2010-09-01 07:33:21 Z executing ssh ... root@xxxxxxxxxxxxx xm list
Warning: Permanently added '10.80.249.102' (RSA) to the list of known
hosts.
2010-09-01 07:33:21 Z guest debian.guest.osstest state is r
2010-09-01 07:33:21 Z guest debian.guest.osstest 5a:36:0e:0a:00:10 22
link/ip/tcp: waiting 100s...
2010-09-01 07:33:21 Z guest debian.guest.osstest 5a:36:0e:0a:00:10 22
link/ip/tcp: no entry in statedb::ips (waiting) ...
guest debian.guest.osstest 5a:36:0e:0a:00:10 22 link/ip/tcp: wait timed
out: no entry in statedb::ips.
It would be useful to timestamp this last entry, for a while I was
unsure if we'd actually waited 100s or not.
The guest log ends with:
[ 0.678464] Magic number: 1:252:3141
[ 0.680053] Freeing unused kernel memory: 600k freed
[ 0.680072] Write protecting the kernel read-only data: 8756k
Loading, please wait...
Begin: Loading essential drivers ... done.
Begin: Running /scripts/init-premount ... done.
Begin: Mounting root file system ... Begin: Running /scripts/local-top
... Volume group "bedbug.cam.xci-test.com" not found
done.
Begin: Waiting for root file system ...
I'm not sure if you said something in the past about the LVM thing being
a benign error due to using the same initrd everywhere (in which case I
think it would be better to arrange for the test suite to generate a
proper matching ramdisk for each kernel/VM, since it is quite confusing
and always sows the seeds of doubt in my mind that a problem might be a
ramdisk configuration issue)
The guest log doesn't appear to show blkfront firing up, although there
is no message about a VBD device without a driver. The xenstore listing
in the logs shows a VBD device in state 4 (connected) as expected, the
backend is blkback (over nbd, I think).
I don't see anything in the domain 0 logs pointing to the cause of the
hang. My guess would be that blkback has hung up reading from the nbd
device, probably while the guest was trying to read the partition table,
hence no messages about the disk in the guest log.
> test-amd64-amd64-pv 7 guest-start fail REGR. vs.
> 2053
Similar pattern to above. Are we confident that the underlying nbd
infrastructure is not flakey?
This specific test case appears to use NBD even though all guests are
local (there's only one host), perhaps we should avoid NBD except where
necessary to avoid spurious failures?
We should probably also add blktap on NFS to the tested combinations too
if it isn't already there, this would also cover the
migration-with-blktap case as well as helping to identify when failures
were NBD related.
Can we add the output of something like
ps wwwaxf -eo pid,tty,stat,time,nice,psr,pcpu,pmem,nwchan,wchan:25,args
from dom0 to the captured logs so we can see where processes are hung
up. In particular it would have been interesting to know where the qemu
was hung in the windows-install failure below.
Ian.
> test-amd64-amd64-xl 7 guest-start fail REGR. vs.
> 2053
> test-amd64-i386-pair 11 guest-start fail like
> 2053
> test-amd64-i386-pv 7 guest-start fail like
> 2053
> test-amd64-i386-xl 7 guest-start fail REGR. vs.
> 2053
> test-amd64-xcpkern-i386-win 5 windows-install fail REGR. vs.
> 2053
> test-amd64-xcpkern-i386-xl 10 guest-stop fail never
> pass
> test-i386-i386-pair 11 guest-start fail like
> 2053
> test-i386-i386-pv 7 guest-start fail like
> 2053
> test-i386-i386-xl 7 guest-start fail REGR. vs.
> 2053
> test-i386-xcpkern-i386-pair 12 guest-migrate/src_host/dst_host fail like
> 2053
> test-i386-xcpkern-i386-xl 10 guest-stop fail never
> pass
>
> version targeted for testing:
> xen 573ddf5cc145
> baseline version:
> xen 3c4c3d48a835
>
> jobs:
> build-amd64 pass
> build-amd64-oldkern pass
> build-i386 pass
> build-i386-oldkern pass
> build-i386-xcpkern pass
> test-amd64-amd64-pair fail
> test-amd64-amd64-pv fail
> test-amd64-amd64-win pass
> test-amd64-amd64-xl fail
> test-amd64-i386-pair fail
> test-amd64-i386-pv fail
> test-amd64-i386-win pass
> test-amd64-i386-xl fail
> test-amd64-xcpkern-i386-pair pass
> test-amd64-xcpkern-i386-pv pass
> test-amd64-xcpkern-i386-win fail
> test-amd64-xcpkern-i386-xl fail
> test-i386-i386-pair fail
> test-i386-i386-pv fail
> test-i386-i386-win pass
> test-i386-i386-xl fail
> test-i386-xcpkern-i386-pair fail
> test-i386-xcpkern-i386-pv pass
> test-i386-xcpkern-i386-win pass
> test-i386-xcpkern-i386-xl fail
>
> -------------------------------------------------------------------------------
> build-amd64:
> 1 host-install(1) pass
> 2 host-build-prep pass
> 3 xen-build pass
> linux 2968b258b1ca6bd16d75
> qemu 1734e63891573d4729c7
> xen 22078:573ddf5cc145
> -------------------------------------------------------------------------------
> build-amd64-oldkern:
> 1 xen-build pass
> linux 1029:9b1adfb8b0b3
> qemu 1734e63891573d4729c7
> xen 22078:573ddf5cc145
> -------------------------------------------------------------------------------
> build-i386:
> 1 host-install(1) pass
> 2 host-build-prep pass
> 3 xen-build pass
> linux 2968b258b1ca6bd16d75
> qemu 1734e63891573d4729c7
> xen 22078:573ddf5cc145
> -------------------------------------------------------------------------------
> build-i386-oldkern:
> 1 xen-build pass
> linux 1029:9b1adfb8b0b3
> qemu 1734e63891573d4729c7
> xen 22078:573ddf5cc145
> -------------------------------------------------------------------------------
> build-i386-xcpkern:
> 1 kernel-build pass
> linux 110879:32fc6955a6a5
> pq_linux 154:61499f9e9a87
> -------------------------------------------------------------------------------
> test-amd64-amd64-pair:
> 1 xen-build-check(1) pass
> 2 host-install/src_host(2) pass
> 3 host-install/dst_host(3) pass
> 4 xen-install/src_host pass
> 5 xen-install/dst_host pass
> 6 xen-boot/src_host pass
> 7 xen-boot/dst_host pass
> 8 debian-install/dst_host pass
> 9 debian-fixup/dst_host pass
> 10 guests-nbd-mirror pass
> 11 guest-start fail
> 12 capture-logs/src_host(12) pass
> 13 capture-logs/dst_host(13) pass
> -------------------------------------------------------------------------------
> test-amd64-amd64-pv:
> 1 xen-build-check(1) pass
> 2 host-install(2) pass
> 3 xen-install pass
> 4 xen-boot pass
> 5 debian-install pass
> 6 debian-fixup pass
> 7 guest-start fail
> 8 capture-logs(8) pass
> -------------------------------------------------------------------------------
> test-amd64-amd64-win:
> 1 xen-build-check(1) pass
> 2 host-install(2) pass
> 3 xen-install pass
> 4 xen-boot pass
> 5 windows-install pass
> 6 guest-saverestore pass
> 7 guest-localmigrate pass
> 8 guest-stop pass
> 9 capture-logs(9) pass
> -------------------------------------------------------------------------------
> test-amd64-amd64-xl:
> 1 xen-build-check(1) pass
> 2 host-install(2) pass
> 3 xen-install pass
> 4 xen-boot pass
> 5 debian-install pass
> 6 debian-fixup pass
> 7 guest-start fail
> 8 capture-logs(8) pass
> -------------------------------------------------------------------------------
> test-amd64-i386-pair:
> 1 xen-build-check(1) pass
> 2 host-install/src_host(2) pass
> 3 host-install/dst_host(3) pass
> 4 xen-install/src_host pass
> 5 xen-install/dst_host pass
> 6 xen-boot/src_host pass
> 7 xen-boot/dst_host pass
> 8 debian-install/dst_host pass
> 9 debian-fixup/dst_host pass
> 10 guests-nbd-mirror pass
> 11 guest-start fail
> 12 capture-logs/src_host(12) pass
> 13 capture-logs/dst_host(13) pass
> -------------------------------------------------------------------------------
> test-amd64-i386-pv:
> 1 xen-build-check(1) pass
> 2 host-install(2) pass
> 3 xen-install pass
> 4 xen-boot pass
> 5 debian-install pass
> 6 debian-fixup pass
> 7 guest-start fail
> 8 capture-logs(8) pass
> -------------------------------------------------------------------------------
> test-amd64-i386-win:
> 1 xen-build-check(1) pass
> 2 host-install(2) pass
> 3 xen-install pass
> 4 xen-boot pass
> 5 windows-install pass
> 6 guest-saverestore pass
> 7 guest-localmigrate pass
> 8 guest-stop pass
> 9 capture-logs(9) pass
> -------------------------------------------------------------------------------
> test-amd64-i386-xl:
> 1 xen-build-check(1) pass
> 2 host-install(2) pass
> 3 xen-install pass
> 4 xen-boot pass
> 5 debian-install pass
> 6 debian-fixup pass
> 7 guest-start fail
> 8 capture-logs(8) pass
> -------------------------------------------------------------------------------
> test-amd64-xcpkern-i386-pair:
> 1 xen-build-check(1) pass
> 2 host-install/src_host(2) pass
> 3 host-install/dst_host(3) pass
> 4 xen-install/src_host pass
> 5 xen-install/dst_host pass
> 6 xen-boot/src_host pass
> 7 xen-boot/dst_host pass
> 8 debian-install/dst_host pass
> 9 debian-fixup/dst_host pass
> 10 guests-nbd-mirror pass
> 11 guest-start pass
> 12 guest-migrate/src_host/dst_host pass
> 13 guest-migrate/dst_host/src_host pass
> 14 capture-logs/src_host(14) pass
> 15 capture-logs/dst_host(15) pass
> -------------------------------------------------------------------------------
> test-amd64-xcpkern-i386-pv:
> 1 xen-build-check(1) pass
> 2 host-install(2) pass
> 3 xen-install pass
> 4 xen-boot pass
> 5 debian-install pass
> 6 debian-fixup pass
> 7 guest-start pass
> 8 guest-saverestore pass
> 9 guest-localmigrate pass
> 10 guest-stop pass
> 11 capture-logs(11) pass
> -------------------------------------------------------------------------------
> test-amd64-xcpkern-i386-win:
> 1 xen-build-check(1) pass
> 2 host-install(2) pass
> 3 xen-install pass
> 4 xen-boot pass
> 5 windows-install fail
> 6 capture-logs(6) pass
> -------------------------------------------------------------------------------
> test-amd64-xcpkern-i386-xl:
> 1 xen-build-check(1) pass
> 2 host-install(2) pass
> 3 xen-install pass
> 4 xen-boot pass
> 5 debian-install pass
> 6 debian-fixup pass
> 7 guest-start pass
> 8 guest-saverestore pass
> 9 guest-localmigrate pass
> 10 guest-stop fail
> 11 capture-logs(11) pass
> -------------------------------------------------------------------------------
> test-i386-i386-pair:
> 1 xen-build-check(1) pass
> 2 host-install/src_host(2) pass
> 3 host-install/dst_host(3) pass
> 4 xen-install/src_host pass
> 5 xen-install/dst_host pass
> 6 xen-boot/src_host pass
> 7 xen-boot/dst_host pass
> 8 debian-install/dst_host pass
> 9 debian-fixup/dst_host pass
> 10 guests-nbd-mirror pass
> 11 guest-start fail
> 12 capture-logs/src_host(12) pass
> 13 capture-logs/dst_host(13) pass
> -------------------------------------------------------------------------------
> test-i386-i386-pv:
> 1 xen-build-check(1) pass
> 2 host-install(2) pass
> 3 xen-install pass
> 4 xen-boot pass
> 5 debian-install pass
> 6 debian-fixup pass
> 7 guest-start fail
> 8 capture-logs(8) pass
> -------------------------------------------------------------------------------
> test-i386-i386-win:
> 1 xen-build-check(1) pass
> 2 host-install(2) pass
> 3 xen-install pass
> 4 xen-boot pass
> 5 windows-install pass
> 6 guest-saverestore pass
> 7 guest-localmigrate pass
> 8 guest-stop pass
> 9 capture-logs(9) pass
> -------------------------------------------------------------------------------
> test-i386-i386-xl:
> 1 xen-build-check(1) pass
> 2 host-install(2) pass
> 3 xen-install pass
> 4 xen-boot pass
> 5 debian-install pass
> 6 debian-fixup pass
> 7 guest-start fail
> 8 capture-logs(8) pass
> -------------------------------------------------------------------------------
> test-i386-xcpkern-i386-pair:
> 1 xen-build-check(1) pass
> 2 host-install/src_host(2) pass
> 3 host-install/dst_host(3) pass
> 4 xen-install/src_host pass
> 5 xen-install/dst_host pass
> 6 xen-boot/src_host pass
> 7 xen-boot/dst_host pass
> 8 debian-install/dst_host pass
> 9 debian-fixup/dst_host pass
> 10 guests-nbd-mirror pass
> 11 guest-start pass
> 12 guest-migrate/src_host/dst_host fail
> 13 capture-logs/src_host(13) pass
> 14 capture-logs/dst_host(14) pass
> -------------------------------------------------------------------------------
> test-i386-xcpkern-i386-pv:
> 1 xen-build-check(1) pass
> 2 host-install(2) pass
> 3 xen-install pass
> 4 xen-boot pass
> 5 debian-install pass
> 6 debian-fixup pass
> 7 guest-start pass
> 8 guest-saverestore pass
> 9 guest-localmigrate pass
> 10 guest-stop pass
> 11 capture-logs(11) pass
> -------------------------------------------------------------------------------
> test-i386-xcpkern-i386-win:
> 1 xen-build-check(1) pass
> 2 host-install(2) pass
> 3 xen-install pass
> 4 xen-boot pass
> 5 windows-install pass
> 6 guest-saverestore pass
> 7 guest-localmigrate pass
> 8 guest-stop pass
> 9 capture-logs(9) pass
> -------------------------------------------------------------------------------
> test-i386-xcpkern-i386-xl:
> 1 xen-build-check(1) pass
> 2 host-install(2) pass
> 3 xen-install pass
> 4 xen-boot pass
> 5 debian-install pass
> 6 debian-fixup pass
> 7 guest-start pass
> 8 guest-saverestore pass
> 9 guest-localmigrate pass
> 10 guest-stop fail
> 11 capture-logs(11) pass
>
> ------------------------------------------------------------
> sg-report-flight on woking.cam.xci-test.com
> logs: /home/xc_osstest/logs
> images: /home/xc_osstest/images
>
> Logs, config files, etc. are available at
> http://www.chiark.greenend.org.uk/~xensrcts/logs
>
> Test harness code can be found at
> http://xenbits.xensource.com/gitweb?p=osstest.git;a=summary
>
>
> Not pushing.
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-devel
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel
|
|
|
| |
|
Home