hi,
i got another error on vm-migrate.
"signal -7" in the log seems intersting. does this ring your bell?
YAMAMOTO Takashi
+ date
Thu Jun 17 21:51:44 JST 2010
+ xe vm-migrate live=true uuid=23ecfa58-aa30-ea6a-f9fe-7cb2a5487592
host=67b8b07b-8c50-4677-a511-beb196ea766f
Lost connection to the server.
/var/log/messages:
Jun 17 21:51:40 s1 ovs-cfg-mod:
00007|cfg_mod|INFO|-port.vif4958.1.vm-uuid=23ecfa58-aa30-ea6a-f9fe-7cb2a5487592
Jun 17 21:51:41 s1 xapi: [ warn|s1|2416799 unix-RPC|VM.pool_migrate
R:832813c0722b|hotplug] Warning, deleting 'vif' entry from
/xapi/4958/hotplug/vif/1
Jun 17 21:51:41 s1 xapi: [error|s1|90 xal_listen|VM (domid: 4958) device_event
= ChangeUncooperative false D:0e953bd99071|event] device_event could not be
processed because VM record not in database
Jun 17 21:51:47 s1 xapi: [ warn|s1|2417066 inet_rpc|VM.pool_migrate
R:fee54e870a4e|xapi] memory_required_bytes = 1080033280 > memory_static_max =
1073741824; clipping
Jun 17 21:51:57 s1 xenguest: Determined the following parameters from xenstore:
Jun 17 21:51:57 s1 xenguest: vcpu/number:1 vcpu/affinity:0 vcpu/weight:0
vcpu/cap:0 nx: 0 viridian: 1 apic: 1 acpi: 1 pae: 1 acpi_s4: 0 acpi_s3: 0
Jun 17 21:52:43 s1 scripts-vif: Called as "add vif" domid:4959 devid:1
mode:vswitch
Jun 17 21:52:44 s1 scripts-vif: Called as "online vif" domid:4959 devid:1
mode:vswitch
Jun 17 21:52:46 s1 scripts-vif: Adding vif4959.1 to xenbr0 with address
fe:ff:ff:ff:ff:ff
Jun 17 21:52:46 s1 ovs-vsctl: Called as br-to-vlan xenbr0
Jun 17 21:52:49 s1 ovs-cfg-mod: 00001|cfg|INFO|using "/etc/ovs-vswitchd.conf"
as configuration file, "/etc/.ovs-vswitchd.conf.~lock~" as lock file
Jun 17 21:52:49 s1 ovs-cfg-mod: 00002|cfg_mod|INFO|configuration changes:
Jun 17 21:52:49 s1 ovs-cfg-mod: 00003|cfg_mod|INFO|+bridge.xenbr0.port=vif4959.1
Jun 17 21:52:49 s1 ovs-cfg-mod:
00004|cfg_mod|INFO|+port.vif4959.1.net-uuid=9ca059b1-ac1e-8d3f-ff19-e5e74f7b7392
Jun 17 21:52:49 s1 ovs-cfg-mod:
00005|cfg_mod|INFO|+port.vif4959.1.vif-mac=2e:17:01:b0:05:fb
Jun 17 21:52:49 s1 ovs-cfg-mod:
00006|cfg_mod|INFO|+port.vif4959.1.vif-uuid=271f0001-06ca-c9ca-cabc-dc79f412d925
Jun 17 21:52:49 s1 ovs-cfg-mod:
00007|cfg_mod|INFO|+port.vif4959.1.vm-uuid=23ecfa58-aa30-ea6a-f9fe-7cb2a5487592
Jun 17 21:52:51 s1 xapi: [ info|s1|0 thread_zero||watchdog] received signal -7
Jun 17 21:52:51 s1 xapi: [ info|s1|0 thread_zero||watchdog] xapi watchdog
exiting.
Jun 17 21:52:51 s1 xapi: [ info|s1|0 thread_zero||watchdog] Fatal: xapi died
with signal -7: not restarting (watchdog never restarts on this signal)
Jun 17 21:55:11 s1 python: PERFMON: caught IOError: (socket error (111,
'Connection refused')) - restarting XAPI session
Jun 17 22:00:02 s1 python: PERFMON: caught socket.error: (111 Connection
refused) - restarting XAPI session
Jun 17 22:04:48 s1 python: PERFMON: caught socket.error: (111 Connection
refused) - restarting XAPI session
Jun 17 22:09:58 s1 python: PERFMON: caught socket.error: (111 Connection
refused) - restarting XAPI session
Jun 17 22:14:52 s1 python: PERFMON: caught socket.error: (111 Connection
refused) - restarting XAPI session
Jun 17 22:19:38 s1 python: PERFMON: caught socket.error: (111 Connection
refused) - restarting XAPI session
> hi,
>
> thanks. i'll take a look at the log if it happens again.
>
> YAMAMOTO Takashi
>
>> This is usually the result of a failure earier on. Could you grep through
>> the logs to get the whole trace of what went on? Best thing to do is grep
>> for VM.pool_migrate, then find the task reference (the hex string beginning
>> with 'R:' immediately after the 'VM.pool_migrate') and grep for this string
>> in the logs on both the source and destination machines.
>>
>> Have a look through these, and if it's still not obvious what went wrong,
>> post them to the list and we can have a look.
>>
>> Cheers,
>>
>> Jon
>>
>>
>> On 16 Jun 2010, at 07:19, YAMAMOTO Takashi wrote:
>>
>>> hi,
>>>
>>> after making my sr driver defer the attach operation as you suggested,
>>> i got migration work. thanks!
>>>
>>> however, when repeating live migration between two hosts for testing,
>>> i got the following error. it doesn't seem so reproducable.
>>> do you have any idea?
>>>
>>> YAMAMOTO Takashi
>>>
>>> + xe vm-migrate live=true uuid=23ecfa58-aa30-ea6a-f9fe-7cb2a5487592
>>> host=67b8b07b-8c50-4677-a511-beb196ea766f
>>> An error occurred during the migration process.
>>> vm: 23ecfa58-aa30-ea6a-f9fe-7cb2a5487592 (CentOS53x64-1)
>>> source: eea41bdd-d2ce-4a9a-bc51-1ca286320296 (s6)
>>> destination: 67b8b07b-8c50-4677-a511-beb196ea766f (s1)
>>> msg: Caught exception INTERNAL_ERROR: [
>>> Xapi_vm_migrate.Remote_failed("unmarshalling result code from remote") ] at
>>> last minute during migration
>>>
>>>> hi,
>>>>
>>>> i'll try deferring the attach operation to vdi_activate.
>>>> thanks!
>>>>
>>>> YAMAMOTO Takashi
>>>>
>>>>> Yup, vdi activate is the way forward.
>>>>>
>>>>> If you advertise VDI_ACTIVATE and VDI_DEACTIVATE in the 'get_driver_info'
>>>>> response, xapi will call the following during the start-migrate-shutdown
>>>>> lifecycle:
>>>>>
>>>>> VM start:
>>>>>
>>>>> host A: VDI.attach
>>>>> host A: VDI.activate
>>>>>
>>>>> VM migrate:
>>>>>
>>>>> host B: VDI.attach
>>>>>
>>>>> (VM pauses on host A)
>>>>>
>>>>> host A: VDI.deactivate
>>>>> host B: VDI.activate
>>>>>
>>>>> (VM unpauses on host B)
>>>>>
>>>>> host A: VDI.detach
>>>>>
>>>>> VM shutdown:
>>>>>
>>>>> host B: VDI.deactivate
>>>>> host B: VDI.detach
>>>>>
>>>>> so the disk is never activated on both hosts at once, but it does still
>>>>> go through a period when it is attached to both hosts at once. So you
>>>>> could, for example, check that the disk *could* be attached on the
>>>>> vdi_attach SMAPI call, and actually attach it properly on the
>>>>> vdi_activate call.
>>>>>
>>>>> Hope this helps,
>>>>>
>>>>> Jon
>>>>>
>>>>>
>>>>> On 7 Jun 2010, at 09:26, YAMAMOTO Takashi wrote:
>>>>>
>>>>>> hi,
>>>>>>
>>>>>> on vm-migrate, xapi attaches a vdi on the migrate-to host
>>>>>> before detaching it on the migrate-from host.
>>>>>> unfortunately it doesn't work for our product, which doesn't
>>>>>> provide a way to attach a volume to multiple hosts at the same time.
>>>>>> is VDI_ACTIVATE something what i can use as a workaround?
>>>>>> or any other suggestions?
>>>>>>
>>>>>> YAMAMOTO Takashi
>>>>>>
>>>>>> _______________________________________________
>>>>>> Xen-devel mailing list
>>>>>> Xen-devel@xxxxxxxxxxxxxxxxxxx
>>>>>> http://lists.xensource.com/xen-devel
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Xen-devel mailing list
>>>>> Xen-devel@xxxxxxxxxxxxxxxxxxx
>>>>> http://lists.xensource.com/xen-devel
>>>>
>>>> _______________________________________________
>>>> Xen-devel mailing list
>>>> Xen-devel@xxxxxxxxxxxxxxxxxxx
>>>> http://lists.xensource.com/xen-devel
>>
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@xxxxxxxxxxxxxxxxxxx
>> http://lists.xensource.com/xen-devel
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel
|
Home