WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-devel

[Xen-devel] changeset: 6884 update

To: Xen Devs <xen-devel@xxxxxxxxxxxxxxxxxxx>
Subject: [Xen-devel] changeset: 6884 update
From: Ted Kaczmarek <tedkaz@xxxxxxxxx>
Date: Fri, 16 Sep 2005 08:42:31 -0400
Delivery-date: Fri, 16 Sep 2005 12:40:17 +0000
Domainkey-signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:subject:from:to:content-type:date:message-id:mime-version:x-mailer:content-transfer-encoding; b=uQq7GaHlhajiWFOl6/8Nd0sJEpLcyfQglLe0PXJVtvcIH/ZkTestcAVeeuRZVGrRR90OOeXFpcComXQp/aRG4Ohue+hlIbmEF2T+DXeofgRBLCD+PXvVO17hG5NOrN/Bq8sUQIeduEOI7tgCs86PwkGYT9OqecdAFbSKkI4HQ0k=
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
FC4, Tyan 2462 SMP, 2 gig, most advanced networking modules built for
Dom0 and DomU.

On first attempt 
start xend manually

Create 1st DomU, it crashed, this is consistent even after reboot.


######################
DomU Kernel Crash CLI
######################

Freeing unused kernel memory: 148k freed
int3: 0000 [#1]
PREEMPT SMP
Modules linked in:
CPU:    1
EIP:    0061:[<c039a4bf>]    Not tainted VLI
EFLAGS: 00000246   (2.6.12-xenU)
EIP is at start_secondary+0x2f/0xd0
eax: 00000001   ebx: 00000000   ecx: 00000007   edx: c1540000
esi: 00000000   edi: 00000000   ebp: 00000000   esp: c1541fb8
ds: 007b   es: 007b   ss: 0069
Process swapper (pid: 0, threadinfo=c1540000 task=c009a530)
Stack: 00000000 00000000 00000000 00000000 00000000 00000000 00000000
00000000
       00000000 00000000 00000000 00000000 00000000 00000000 00000000
00000000
       00000000 00000000
Call Trace:
Code: cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc
cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc <cc> cc
cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc
<0>Kernel panic - not syncing: Attempted to kill the idle task!

###########
Log Snippet
###########

DEBUG (blkif:24) exception looking up device number for sda2: [Errno 2]
No such file or directory: '/dev/sda2'

Created second DomU with this one crashed, no problems.

Created first DomU, xm list now shows two instances

init 0 to second DomU, stop xend , restart xend , first DomU get created
with no problems.

Also some interesting messages

[2005-09-16 07:29:12 xend] DEBUG (blkif:24) exception looking up device
number for hda2: [Errno 2] No such file or directory: '/dev/hda2'
[2005-09-16 07:29:12 xend] INFO (XendRoot:141) EVENT>
xend.domain.unpause ['inyoureyes-domain2', 5]
[2005-09-16 07:29:25 xend] DEBUG (image:159) initDomain: cpu=-1
mem_kb=134144 ssidref=-1 dom=6
[2005-09-16 07:29:25 xend] DEBUG (XendDomainInfo:822) init_domain>
Created domain=6 name=inyoureyes-domain3 memory=131
[2005-09-16 07:29:25 xend] INFO (XendRoot:141) EVENT> xend.domain.create
['inyoureyes-domain3', 6]
[2005-09-16 07:29:25 xend] INFO (image:196) buildDomain os=linux dom=6
vcpus=2
[2005-09-16 07:29:25 xend] DEBUG (image:240) dom            = 6
[2005-09-16 07:29:25 xend] DEBUG (image:241) image
= /boot/vmlinuz-2.6.12-xenU
[2005-09-16 07:29:25 xend] DEBUG (image:242) store_evtchn   = 1
[2005-09-16 07:29:25 xend] DEBUG (image:243) console_evtchn = 2
[2005-09-16 07:29:25 xend] DEBUG (image:244) cmdline        =
root=/dev/hda3 ro selinux=0 3
[2005-09-16 07:29:25 xend] DEBUG (image:245) ramdisk        =
[2005-09-16 07:29:25 xend] DEBUG (image:246) flags          = 0
[2005-09-16 07:29:25 xend] DEBUG (image:247) vcpus          = 2
[2005-09-16 07:29:25 xend] DEBUG (blkif:24) exception looking up device
number for hda3: [Errno 2] No such file or directory: '/dev/hda3'
[2005-09-16 07:29:26 xend] INFO (XendRoot:141) EVENT>
xend.domain.unpause ['inyoureyes-domain3', 6]


inyoureyes-domain2 started fine

inyoureyes-domain3 got stuck at 
xen_net: Initialising virtual ethernet driver.
xen_net: Using grant tables.


For these tests I have most of the advanced networking modules being
built if that matters.

The good part is that xm destroy is able to kill inyoureyes-domain3.
>From what I have been seeing xm destroy problem is contained to crashed
and duped Domu's only, this appears to be consistent to what I read on
the list.

I restart inyoureyes-domain3 and xm list shows it as paused, issuing an
xm unpause it now shows blocked, but issuing xm console fails with 
xenconsole: Could not read tty from store: No such file or directory.
Than I issue xm list and inyoureyes-domain3 is no longer showing up.

Correlating log snippets

[2005-09-16 07:33:48 xend] DEBUG (XendDomain:472)
domain_restart_schedule> 6 halt 1
[2005-09-16 07:33:48 xend] INFO (XendDomain:485) Cancelling restart for
domain: name=inyoureyes-domain3 id=6
[2005-09-16 07:33:48 xend] INFO (XendDomain:514) Destroying domain:
name=inyoureyes-domain3
[2005-09-16 07:33:48 xend] INFO (XendRoot:141) EVENT>
xend.domain.destroy ['inyoureyes-domain3', 6]
[2005-09-16 07:33:48 xend] INFO (XendRoot:141) EVENT>
xend.domain.restart ['inyoureyes-domain3', 6, 'cancel']
[2005-09-16 07:33:48 xend] DEBUG (XendDomain:240) XendDomain>reap>
domain died name=?? id=1
[2005-09-16 07:33:48 xend] INFO (XendRoot:141) EVENT> xend.domain.exit
['??', 1, 'crash']
[2005-09-16 07:35:34 xend] DEBUG (image:159) initDomain: cpu=-1
mem_kb=134144 ssidref=-1 dom=7
[2005-09-16 07:35:34 xend] DEBUG (XendDomainInfo:822) init_domain>
Created domain=7 name=inyoureyes-domain3 memory=131
[2005-09-16 07:35:34 xend] INFO (XendRoot:141) EVENT> xend.domain.create
['inyoureyes-domain3', 7]
[2005-09-16 07:35:34 xend] INFO (image:196) buildDomain os=linux dom=7
vcpus=2
[2005-09-16 07:35:34 xend] DEBUG (image:240) dom            = 7
[2005-09-16 07:35:34 xend] DEBUG (image:241) image
= /boot/vmlinuz-2.6.12-xenU
[2005-09-16 07:35:34 xend] DEBUG (image:242) store_evtchn   = 1
[2005-09-16 07:35:34 xend] DEBUG (image:243) console_evtchn = 2
[2005-09-16 07:35:34 xend] DEBUG (image:244) cmdline        =
root=/dev/hda3 ro selinux=0 3
[2005-09-16 07:35:34 xend] DEBUG (image:245) ramdisk        =
[2005-09-16 07:35:34 xend] DEBUG (image:246) flags          = 0
[2005-09-16 07:35:34 xend] DEBUG (image:247) vcpus          = 2
[2005-09-16 07:35:34 xend] DEBUG (blkif:24) exception looking up device
number for hda3: [Errno 2] No such file or directory: '/dev/hda3'
[2005-09-16 07:35:35 xend] INFO (XendRoot:141) EVENT>
xend.domain.unpause ['inyoureyes-domain3', 6]
[2005-09-16 07:35:40 xend] INFO (XendRoot:141) EVENT> xend.domain.died
['inyoureyes-domain3', None]
[2005-09-16 07:35:53 xend] INFO (XendRoot:141) EVENT>
xend.domain.unpause ['inyoureyes-domain3', 7]
[2005-09-16 07:36:15 xend] DEBUG (XendDomain:240) XendDomain>reap>
domain died name=?? id=1
[2005-09-16 07:36:15 xend] DEBUG (XendDomain:240) XendDomain>reap>
domain died name=inyoureyes-domain3 id=7
[2005-09-16 07:36:15 xend] INFO (XendRoot:141) EVENT> xend.domain.exit
['??', 1, 'crash']
[2005-09-16 07:36:15 xend] INFO (XendDomain:514) Destroying domain:
name=inyoureyes-domain3
[2005-09-16 07:36:15 xend] INFO (XendRoot:141) EVENT> xend.domain.exit
['inyoureyes-domain3', 7, 'crash']
[2005-09-16 07:36:15 xend] INFO (XendRoot:141) EVENT>
xend.domain.destroy ['inyoureyes-domain3', 7]
[2005-09-16 07:36:42 xend] ERROR (XendDomainInfo:128) domain_getinfo(7)
failed, ignoring
Traceback (most recent call last):
  File "/usr/lib/python/xen/xend/XendDomainInfo.py", line 123, in
dom_get
    domlist = xc.domain_getinfo(dom, 1)
error: (3, 'No such process')
[2005-09-16 07:37:40 xend] ERROR (XendDomainInfo:128) domain_getinfo(7)
failed, ignoring
Traceback (most recent call last):
  File "/usr/lib/python/xen/xend/XendDomainInfo.py", line 123, in
dom_get
    domlist = xc.domain_getinfo(dom, 1)
error: (3, 'No such process')
[2005-09-16 07:37:40 xend] INFO (XendRoot:141) EVENT> xend.domain.died
['inyoureyes-domain3', None]



This one is easily repeatable at least for my setup.

ERROR: do_evtchn_op: HYPERVISOR_event_channel_op failed: -1
Exception in thread Watcher:
Traceback (most recent call last):
  File "/usr/lib/python2.4/threading.py", line 442, in __bootstrap
    self.run()
  File "/usr/lib/python2.4/threading.py", line 422, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/usr/lib/python/xen/xend/xenstore/xswatch.py", line 63, in
watchMain
    watch.fn(*watch.args, **watch.kwargs)
  File "/usr/lib/python/xen/xend/XendDomain.py", line 98, in
onReleaseDomain
    self.refresh(cleanup=True)
  File "/usr/lib/python/xen/xend/XendDomain.py", line 265, in refresh
    self.reap()
  File "/usr/lib/python/xen/xend/XendDomain.py", line 259, in reap
    self.final_domain_destroy(id)
  File "/usr/lib/python/xen/xend/XendDomain.py", line 522, in
final_domain_destroy
    raise XendError(str(ex))
XendError: (3, 'No such process')




Another Caveat
It appears that 

ERROR (process:37) [network-bridge] Cannot find device "xen-br0:1"

Stops happening after a few restarts of xend. 

But this message is consistent for any domains.

(blkif:24) exception looking up device number for sda2: [Errno 2] No
such file or directory: '/dev/sda2'

Seems not to matter as the DomU gets created though.

Also seeing some issue with memory, once the first DomU crashes I am
unable to recover the memory assigned to it. Stopping xend and purging
files in /var/lib/xenstored/store/domain/ no affect. Appears to
cooincide with this trace.

Exception in thread Watcher:
Traceback (most recent call last):
  File "/usr/lib/python2.4/threading.py", line 442, in __bootstrap
    self.run()
  File "/usr/lib/python2.4/threading.py", line 422, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/usr/lib/python/xen/xend/xenstore/xswatch.py", line 63, in
watchMain
    watch.fn(*watch.args, **watch.kwargs)
  File "/usr/lib/python/xen/xend/XendDomain.py", line 98, in
onReleaseDomain
    self.refresh(cleanup=True)
  File "/usr/lib/python/xen/xend/XendDomain.py", line 265, in refresh
    self.reap()
  File "/usr/lib/python/xen/xend/XendDomain.py", line 259, in reap
    self.final_domain_destroy(id)
  File "/usr/lib/python/xen/xend/XendDomain.py", line 522, in
final_domain_destroy
    raise XendError(str(ex))
XendError: (3, 'No such process')


Regards,
Ted







_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel

<Prev in Thread] Current Thread [Next in Thread>