WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-users

[Xen-users] VM dies spontaneously, how to diagnose?

To: xen-users@xxxxxxxxxxxxxxxxxxx
Subject: [Xen-users] VM dies spontaneously, how to diagnose?
From: Steve Wray <steve.wray@xxxxxxxxx>
Date: Wed, 07 Oct 2009 15:21:16 +1300
Delivery-date: Tue, 06 Oct 2009 19:22:11 -0700
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
List-help: <mailto:xen-users-request@lists.xensource.com?subject=help>
List-id: Xen user discussion <xen-users.lists.xensource.com>
List-post: <mailto:xen-users@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=unsubscribe>
Sender: xen-users-bounces@xxxxxxxxxxxxxxxxxxx
User-agent: Thunderbird 2.0.0.23 (Macintosh/20090812)
Hi there,

I have a Debian Lenny Xen server with a few VMs all running Debian Lenny.

One of these occasionally dies with no apparent reason. Theres nothing in its own logs that give any indication of what went wrong. I see nothing in the Xen logs that helps me directly.

I see entries like the one at the end of this mail (I'm going to put the log data at the end because it is a gigantic wall of text).


The config file for this domU contains:

<config>
on_poweroff = 'destroy'
on_reboot   = 'restart'
on_crash    = 'restart'
</config>

so I was expecting it would restart... however looking at the Xen logs it appears that when Xen tries to restart the crashed domU it comes to believe that its already running...

I'm wondering where I should start debugging this problem. It happens very intermittently.


<logentries>
[2009-10-05 20:34:54 3606] WARNING (XendDomainInfo:1258) Domain has crashed: name=monitoring-lenny id=36. [2009-10-05 20:34:54 3606] DEBUG (XendDomainInfo:1914) XendDomainInfo.destroyDomain(36)
[2009-10-05 20:34:54 3606] DEBUG (XendDomainInfo:1529) Destroying device model
[2009-10-05 20:34:54 3606] DEBUG (XendDomainInfo:1536) Releasing devices
[2009-10-05 20:34:54 3606] DEBUG (XendDomainInfo:1542) Removing vif/0
...
more 'destroyDevice' entries
...
[2009-10-05 20:34:54 3606] DEBUG (XendDomainInfo:1534) No device model
[2009-10-05 20:34:54 3606] DEBUG (XendDomainInfo:1536) Releasing devices
[2009-10-05 20:34:54 3606] DEBUG (XendDomainInfo:106) XendDomainInfo.create_from_dict({'vcpus_params': {'cap': 0, 'weight': 256}, 'PV_args': 'root=/dev/sda2 ro clocksource=jiffies', 'features': '', 'cpus': [], 'paused': 0, 'domid': 36, 'shutdown': 0, 'VCPUs_live': 1, 'PV_bootloader': '', 'actions_after_crash': 'restart', 'vbd_refs': ['8baad5aa-45c9-97f1-1a19-96615ef72229', '38a9f2bc-c8ee-8cfc-dac8-11c73eb130e4'], 'PV_ramdisk': '/boot/initrd.img-2.6.26-1-xen-686', 'is_control_domain': False, 'name_label': 'monitoring-lenny', 'VCPUs_at_startup': 1, 'HVM_boot_params': {}, 'platform': {}, 'PV_kernel': '/boot/vmlinuz-2.6.26-1-xen-686', 'console_refs': ['420f5516-d7bd-9d03-b840-d7c0d23e9963'], 'online_vcpus': 1, 'blocked': 0, 'on_xend_stop': 'ignore', 'memory_static_min': 0, 'HVM_boot_policy': '', 'shutdown_reason': 3, 'VCPUs_max': 3, 'start_time': 1249957930.470108, 'memory_static_max': 1073741824, 'actions_after_shutdown': 'destroy', 'on_xend_start': 'ignore', 'crashed': 0, 'memory_dynamic_max': 1073741824, 'actions_after_suspend': '', 'is_a_template': False, 'PV_bootloader_args': '', 'memory_dynamic_min': 1073741824, 'uuid': 'ac86ee9d-45b8-7359-b736-bdc74506068c', 'cpu_time': 482327.477346609, 'shadow_memory': 0, 'dying': 0, 'vcpu_avail': 7, 'notes': {'HV_START_LOW': 4118806528L, 'FEATURES': 'writable_page_tables|writable_descriptor_tables|auto_translated_physmap|pae_pgdir_above_4gb|supervisor_mode_kernel', 'VIRT_BASE': 3221225472L, 'GUEST_VERSION': '2.6', 'PADDR_OFFSET': 0, 'GUEST_OS': 'linux', 'HYPERCALL_PAGE': 3222278144L, 'LOADER': 'generic', 'SUSPEND_CANCEL': 1, 'PAE_MODE': 'yes', 'ENTRY': 3222274048L, 'XEN_VERSION': 'xen-3.0'}, 'other_config': {}, 'running': 0, 'actions_after_reboot': 'restart', 'vif_refs': ['ef714b6e-164c-5d7f-2708-bdc6a251221f', '0238b375-fb4d-7990-9644-5f2ec53c83ca'], 'vtpm_refs': [], 'devices': {'38a9f2bc-c8ee-8cfc-dac8-11c73eb130e4': ('vbd', {'uuid': '38a9f2bc-c8ee-8cfc-dac8-11c73eb130e4', 'bootable': 0, 'devid': 2049, 'driver': 'paravirtualised', 'dev': 'sda1', 'uname': 'phy:volumes/monitoring-lenny-swap', 'mode': 'w'}), '0238b375-fb4d-7990-9644-5f2ec53c83ca': ('vif', {'bridge': 'xenbr1', 'rate': '10MB/s', 'devid': 1, 'mac': '00:16:3E:12:69:5E', 'uuid': '0238b375-fb4d-7990-9644-5f2ec53c83ca'}), 'ef714b6e-164c-5d7f-2708-bdc6a251221f': ('vif', {'bridge': 'eth0', 'rate': '10MB/s', 'devid': 0, 'mac': '00:16:3E:C7:8A:C8', 'uuid': 'ef714b6e-164c-5d7f-2708-bdc6a251221f'}), '8baad5aa-45c9-97f1-1a19-96615ef72229': ('vbd', {'uuid': '8baad5aa-45c9-97f1-1a19-96615ef72229', 'bootable': 1, 'devid': 2050, 'driver': 'paravirtualised', 'dev': 'sda2', 'uname': 'phy:volumes/monitoring-lenny', 'mode': 'w'}), '420f5516-d7bd-9d03-b840-d7c0d23e9963': ('console', {'location': '2', 'devid': 0, 'protocol': 'vt100', 'uuid': '420f5516-d7bd-9d03-b840-d7c0d23e9963', 'other_config': {}})}}) [2009-10-05 20:34:54 3606] ERROR (XendDomainInfo:111) Domain construction failed
Traceback (most recent call last):
File "/usr/lib/xen-3.2-1/lib/python/xen/xend/XendDomainInfo.py", line 109, in create_from_dict
    vm.start()
File "/usr/lib/xen-3.2-1/lib/python/xen/xend/XendDomainInfo.py", line 444, in start
    raise XendError('VM already running')
XendError: VM already running
[2009-10-05 20:34:54 3606] DEBUG (XendDomainInfo:1897) XendDomainInfo.destroy: domid=36 [2009-10-05 20:34:54 3606] ERROR (XendDomainInfo:1425) Failed to restart domain 36.
</logentries>


--
Please remember that an email is just like a postcard; it is not confidential nor private nor secure and can be read by many other people than the intended recipient. A postcard can be read by anyone at the mail sorting office and expecting what is written on it to be private and secret is not realistic. Please hold no higher expectation of email.

If you need to send confidential information in an email you need to use encryption. PGP is Pretty good for this.

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users

<Prev in Thread] Current Thread [Next in Thread>
  • [Xen-users] VM dies spontaneously, how to diagnose?, Steve Wray <=