WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-devel

Re: [Xen-devel] [PATCH] Avoid race when moving cpu between cpupools

To: Juergen Gross <juergen.gross@xxxxxxxxxxxxxx>
Subject: Re: [Xen-devel] [PATCH] Avoid race when moving cpu between cpupools
From: Andre Przywara <andre.przywara@xxxxxxx>
Date: Fri, 25 Feb 2011 15:25:30 +0100
Cc: George Dunlap <George.Dunlap@xxxxxxxxxxxxx>, "xen-devel@xxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxx>, Keir Fraser <keir@xxxxxxx>, "Diestelhorst, Stephan" <Stephan.Diestelhorst@xxxxxxx>
Delivery-date: Fri, 25 Feb 2011 06:28:00 -0800
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <AANLkTikSiJKLH=ginoEgO4Tx0-Z1AC2bwP4qBDjVSfAg@xxxxxxxxxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
References: <5485071c8b0a6a49f65b.1298541625@nehalem1> <4D666678.1000301@xxxxxxx> <AANLkTikSiJKLH=ginoEgO4Tx0-Z1AC2bwP4qBDjVSfAg@xxxxxxxxxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
User-agent: Thunderbird 2.0.0.23 (X11/20090820)
George Dunlap wrote:
Looks good -- thanks Juergen.

Acked-by: George Dunlap <george.dunlap@xxxxxxxxxxxxx>

 -George

On Thu, Feb 24, 2011 at 2:08 PM, Andre Przywara <andre.przywara@xxxxxxx> wrote:
Juergen Gross wrote:
Moving cpus between cpupools is done under the schedule lock of the moved
cpu.
When checking a cpu being member of a cpupool this must be done with the
lock
of that cpu being held.
I have reviewed and tested the patch. It fixes my problem. My script has
been running for several hundred iterations without any Xen crash, whereas
without the patch the hypervisor crashed mostly at the second iteration.

Juergen,

can you rule out that this code will be triggered on two CPUs trying to switch to each other? As Stephan pointed out: the code looks like as this could trigger a possible dead-lock condition, where:
1) CPU A grabs lock (a) while CPU B grabs lock (b)
2) CPU A tries to grab (b) and CPU B tries to grab (a)
3) both fail and loop to 1)
A possible fix would be to introduce some ordering for the locks (just the pointer address) and let the "bigger" pointer yield to the "smaller" one. I am not sure if this is really necessary, but I now see strange hangs after running the script for a while (30min to 1hr). Sometimes Dom0 hangs for a while, loosing interrupts (sda or eth0) or getting spurious ones, on two occasions the machine totally locked up.

I am not 100% sure whether this is CPUpools related, but I put some load on Dom0 (without messing with CPUpools) for the whole night and it ran fine.

Sorry for this :-(
I will try to further isolate this.

Anyway, it works much better with the fix than without and I will try to trigger this with the "reduce number of Dom0 vCPUs" patch.

Regards,
Andre.


Thanks Juergen and George for the persistent work!

Hot-unplugging of physical cpus might encounter the same problems, but
this
should happen only very rarely.

Signed-off-by: juergen.gross@xxxxxxxxxxxxxx
Acked-by: Andre Przywara <andre.przywara@xxxxxxx>

Keir, please apply for 4.1.0.



--
Andre Przywara
AMD-OSRC (Dresden)
Tel: x29712


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel