WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-users

Re: [Xen-users] Live migration issues

To: xen-users@xxxxxxxxxxxxxxxxxxx
Subject: Re: [Xen-users] Live migration issues
From: austin trask <austin.trask@xxxxxxxxxxxx>
Date: Fri, 21 Sep 2007 08:16:35 -0700
Delivery-date: Fri, 21 Sep 2007 08:17:20 -0700
Envelope-to: www-data@xxxxxxxxxxxxxxxxxx
In-reply-to: <46F2B515.7010507@xxxxxxxxxx>
List-help: <mailto:xen-users-request@lists.xensource.com?subject=help>
List-id: Xen user discussion <xen-users.lists.xensource.com>
List-post: <mailto:xen-users@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=unsubscribe>
References: <46F2AF3D.5010401@xxxxxxxxxxxx> <46F2B515.7010507@xxxxxxxxxx>
Sender: xen-users-bounces@xxxxxxxxxxxxxxxxxxx
User-agent: Thunderbird 1.5.0.13 (X11/20070824)
well I think I figured it out. It was a matter of having the xen images in different directories on the different hosts. for some reason it did not like it if the directories were:
/xen-images/xen-image.img   on xen900  and
/var/xen/xen-image.img on xen901 once I created similar directories on both machines live migration worked like a charm. IE
/var/xen/xen-image.img on both xen901 and xen900

thanks for xen
- austin




Mark Nielsen wrote:
looks to me like the destination can't see the guest's storage device

[2007-09-20 10:15:45 xend 6492] DEBUG (blkif:24) exception looking up device number for hda1: [Errno 2] No such file or directory: '/dev/hda1'

Mark



austin trask wrote:
I have been having a issue recently with the xm migrate feature. It seems to me that the migration begins to take place however somewhere in the middle of the migration things seem to fail.
I run this command..
root@xen900:/xen-images# xm migrate --live feisty_base 192.168.0.1


and to check and see if the system has made the connection I did
root@xen901:/var/log/xen# netstat -an|grep 8002
tcp 0 0 0.0.0.0:8002 0.0.0.0:* LISTEN tcp 0 0 192.168.0.1:8002 192.168.0.2:60671 ESTABLISHED
this is from the other xen host.

after a few moments the xm migrate command completes. Reviewing the logs it seems to be a issue on the receiving host. I am running the host machines with ubuntu 7.04.
there are a few logs on both machines.. from xen900
/var/log/xen/xen-hotplug.log reads
Nothing to flush.
Nothing to flush.
Nothing to flush.
Nothing to flush.
SIOCGIFFLAGS: No such device
xenstore-read: couldn't read path backend/vbd/5/769/node
xenstore-read: couldn't read path backend/vbd/5/770/node
Nothing to flush.
Nothing to flush.
xenstore-read: couldn't read path backend/vbd/7/769/node
xenstore-read: couldn't read path backend/vbd/7/770/node
Nothing to flush.
Nothing to flush.
xenstore-read: couldn't read path backend/vbd/9/770/node
xenstore-read: couldn't read path backend/vbd/9/769/node
Nothing to flush.
Nothing to flush.
(xen900)/var/xen/xend.log reads

[2007-09-20 10:16:32 xend 15789] INFO (XendCheckpoint:239) 4: sent 81, skipped 0, delta 9ms, dom0 100%, target 0%, sent 294Mb/s, dirtied 294Mb/s $ [2007-09-20 10:16:32 xend 15789] INFO (XendCheckpoint:239) Total pages sent= 130884 (0.98x) [2007-09-20 10:16:32 xend 15789] INFO (XendCheckpoint:239) (of which 0 were fixups) [2007-09-20 10:16:32 xend 15789] INFO (XendCheckpoint:239) All memory is saved [2007-09-20 10:16:32 xend 15789] INFO (XendCheckpoint:239) Save exit rc=0 [2007-09-20 10:16:32 xend.XendDomainInfo 15789] DEBUG (XendDomainInfo:1466) XendDomainInfo.destroyDomain(11) [2007-09-20 10:16:32 xend.XendDomainInfo 15789] INFO (XendDomainInfo:1564) Dev 769 still active, looping... [2007-09-20 10:16:32 xend.XendDomainInfo 15789] INFO (XendDomainInfo:1564) Dev 770 still active, looping... [2007-09-20 10:16:32 xend.XendDomainInfo 15789] INFO (XendDomainInfo:1564) Dev 769 still active, looping... [2007-09-20 10:16:32 xend.XendDomainInfo 15789] INFO (XendDomainInfo:1564) Dev 770 still active, looping... [2007-09-20 10:16:32 xend.XendDomainInfo 15789] INFO (XendDomainInfo:1564) Dev 769 still active, looping... [2007-09-20 10:16:32 xend.XendDomainInfo 15789] INFO (XendDomainInfo:1564) Dev 770 still active, looping...


(xen900)/var/xen/xend-debug.log reads(these logs are not timestamped so I am not sure if they are from this run)

Traceback (most recent call last):
 File "SocketServer.py", line 464, in process_request_thread
   self.finish_request(request, client_address)
 File "SocketServer.py", line 254, in finish_request
   self.RequestHandlerClass(request, client_address, self)
 File "SocketServer.py", line 522, in __init__
   self.handle()
 File "BaseHTTPServer.py", line 316, in handle
   self.handle_one_request()
 File "BaseHTTPServer.py", line 310, in handle_one_request
   method()
File "/usr/lib/python2.5/site-packages/xen/util/xmlrpclib2.py", line 66, in do_POST
   self.send_response(200)
 File "BaseHTTPServer.py", line 368, in send_response
   (self.protocol_version, code, message))
 File "socket.py", line 262, in write
   self.flush()
 File "socket.py", line 249, in flush
   self._sock.sendall(buffer)
error: (32, 'Broken pipe')



on xen901 /var/xen/xend.log reads

[2007-09-20 10:15:45 xend.XendDomainInfo 6492] DEBUG (XendDomainInfo:265) XendDomainInfo.restore(['domain', ['domid', '11'], ['uuid', '9e934f3b-1$ [2007-09-20 10:15:45 xend.XendDomainInfo 6492] DEBUG (XendDomainInfo:296) parseConfig: config is ['domain', ['domid', '11'], ['uuid', '9e934f3b-1$ [2007-09-20 10:15:45 xend.XendDomainInfo 6492] DEBUG (XendDomainInfo:395) parseConfig: result is {'shadow_memory': 0, 'uuid': '9e934f3b-10b2-a40f$ [2007-09-20 10:15:45 xend.XendDomainInfo 6492] DEBUG (XendDomainInfo:1262) XendDomainInfo.construct: None [2007-09-20 10:15:45 xend.XendDomainInfo 6492] DEBUG (XendDomainInfo:713) Storing VM details: {'shadow_memory': '0', 'uuid': '9e934f3b-10b2-a40f-$ [2007-09-20 10:15:45 xend 6492] DEBUG (DevController:110) DevController: writing {'backend-id': '0', 'mac': '00:16:3e:5b:e6:66', 'handle': '0', '$ [2007-09-20 10:15:45 xend 6492] DEBUG (DevController:112) DevController: writing {'bridge': 'xenbr0', 'domain': 'feisty_base', 'handle': '0', 'sc$ [2007-09-20 10:15:45 xend 6492] DEBUG (blkif:24) exception looking up device number for hda1: [Errno 2] No such file or directory: '/dev/hda1' [2007-09-20 10:15:45 xend 6492] DEBUG (DevController:110) DevController: writing {'backend-id': '0', 'virtual-device': '769', 'device-type': 'dis$ [2007-09-20 10:15:45 xend 6492] DEBUG (DevController:112) DevController: writing {'domain': 'feisty_base', 'frontend': '/local/domain/6/device/vb$ [2007-09-20 10:15:45 xend 6492] DEBUG (blkif:24) exception looking up device number for hda2: [Errno 2] No such file or directory: '/dev/hda2' [2007-09-20 10:15:45 xend 6492] DEBUG (DevController:110) DevController: writing {'backend-id': '0', 'virtual-device': '770', 'device-type': 'dis$ [2007-09-20 10:15:45 xend 6492] DEBUG (DevController:112) DevController: writing {'domain': 'feisty_base', 'frontend': '/local/domain/6/device/vb$ [2007-09-20 10:15:45 xend.XendDomainInfo 6492] DEBUG (XendDomainInfo:748) Storing domain details: {'console/port': '2', 'name': 'feisty_base', 'c$ [2007-09-20 10:15:45 xend 6492] DEBUG (balloon:127) Balloon: 533004 KiB free; need 532480; done. [2007-09-20 10:15:45 xend 6492] DEBUG (XendCheckpoint:155) [xc_restore]: /usr/lib/xen/bin/xc_restore 18 6 133120 1 2 [2007-09-20 10:15:45 xend 6492] INFO (XendCheckpoint:239) xc_linux_restore start: max_pfn = 20800 [2007-09-20 10:15:45 xend 6492] INFO (XendCheckpoint:239) Increased domain reservation by 82000 KB [2007-09-20 10:15:45 xend 6492] INFO (XendCheckpoint:239) Reloading memory pages: 0% [2007-09-20 10:16:30 xend 6492] INFO (XendCheckpoint:239) Received all pages (0 races)
[2007-09-20 10:16:30 xend 6492] INFO (XendCheckpoint:239) ^H^H^H^H100%
[2007-09-20 10:16:30 xend 6492] INFO (XendCheckpoint:239) Memory reloaded. [2007-09-20 10:16:30 xend 6492] INFO (XendCheckpoint:239) Decreased reservation by 2409 pages [2007-09-20 10:16:30 xend 6492] INFO (XendCheckpoint:239) Domain ready to be built. [2007-09-20 10:16:30 xend 6492] INFO (XendCheckpoint:239) Restore exit with rc=0 [2007-09-20 10:16:30 xend 6492] DEBUG (XendCheckpoint:209) store-mfn 565003 [2007-09-20 10:16:30 xend 6492] DEBUG (XendCheckpoint:209) console-mfn 561602 [2007-09-20 10:16:31 xend 6492] DEBUG (DevController:143) Waiting for devices vif.
[2007-09-20 10:16:31 xend 6492] DEBUG (DevController:149) Waiting for 0.
[2007-09-20 10:16:31 xend 6492] DEBUG (DevController:464) hotplugStatusCallback /local/domain/0/backend/vif/6/0/hotplug-status. [2007-09-20 10:16:31 xend 6492] DEBUG (DevController:478) hotplugStatusCallback 1. [2007-09-20 10:16:31 xend 6492] DEBUG (DevController:143) Waiting for devices usb. [2007-09-20 10:16:31 xend 6492] DEBUG (DevController:143) Waiting for devices vbd. [2007-09-20 10:16:31 xend 6492] DEBUG (DevController:149) Waiting for 769. [2007-09-20 10:16:31 xend 6492] DEBUG (DevController:464) hotplugStatusCallback /local/domain/0/backend/vbd/6/769/hotplug-status. [2007-09-20 10:16:31 xend 6492] DEBUG (DevController:478) hotplugStatusCallback 2. [2007-09-20 10:16:31 xend.XendDomainInfo 6492] DEBUG (XendDomainInfo:1458) XendDomainInfo.destroy: domid=6 [2007-09-20 10:16:31 xend.XendDomainInfo 6492] DEBUG (XendDomainInfo:1466) XendDomainInfo.destroyDomain(6)
[2007-09-20 10:16:31 xend 6492] ERROR (__init__:1021) Restore failed
Traceback (most recent call last):
File "/usr/lib/python2.5/site-packages/xen/xend/XendDomain.py", line 263, in domain_restore_fd
   return XendCheckpoint.restore(self, fd)
File "/usr/lib/python2.5/site-packages/xen/xend/XendCheckpoint.py", line 165, in restore
   dominfo.waitForDevices() # Wait for backends to set up
File "/usr/lib/python2.5/site-packages/xen/xend/XendDomainInfo.py", line 1605, in waitForDevices
   self.waitForDevices_(c)
File "/usr/lib/python2.5/site-packages/xen/xend/XendDomainInfo.py", line 1099, in waitForDevices_
   return self.getDeviceController(deviceClass).waitForDevices()
File "/usr/lib/python2.5/site-packages/xen/xend/server/DevController.py", line 145, in waitForDevices
   return map(self.waitForDevice, self.deviceIDs())
File "/usr/lib/python2.5/site-packages/xen/xend/server/DevController.py", line 163, in waitForDevice
   (devid, self.deviceClass))
VmError: Device 769 (vbd) could not be connected. Backend device not found.


there is nothing notable in the other logs on xen901. _______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users



_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users

<Prev in Thread] Current Thread [Next in Thread>