[Xen-devel] Ticket spinlocks and MP guests

[Espen Skoglund <espen.skoglund@xxxxxxxxxxxxx>]

I noticed that Linux has the "Ticket spinlock" patch now merged for
2.6.25 (see http://lwn.net/Articles/267968/).  Basically, the new
spinlock ensures that lock contenders will acquire the lock in FIFO
order.  This is good news if you're concerned with latency guarantees
or fairness for larger scale SMP systems.

However, spinlocks can have some rather unpleasant effects on
multi-processor guests.  You don't want to preempt a vCPU holding a
spinlock because other vCPUs trying to acquire the lock will then
spend their processing time busy-waiting for the usually shortly held
lock.  Gang-scheduling gets around the problem by ensuring that lock
contenders are not busy-waiting while the lock-holder is preempted,
but often leads to wasted processing resources.

The bad news is that the lock-holder preemption problem is aggravated
when one introduces ticket spinlocks.  Not only should one try to
prevent preempting lock-holders.  Since the lock contenders are
granted the lock in FIFO order one had better also make sure that the
waiting vCPUs are scheduled in the apropriate order.  With regular
spinlocks any one of the waiting vCPUs can acquire the lock and be
done with it quickly.  This is not an option with ticket spinlocks.

I have seen various references to 'bad preemption' avoidance in Xen,
but not any concrete description of what this actually means.  Has
some lock-holder preemption mechanism been implelemted or was it
deemed unnecessary.  A few years ago we did some measurements of the
effects of lock-holder preemptions in SMP guests [1].  However, the
experiments were performed using a 2.4.20 kernel.  The picture is
probably a bit different today.

Do people feel that the new ticket spinlocks should raise any
concerns, or do typical Xen SMP guest workloads remain largely
unaffected by the new locking scheme?

        eSk


[1] http://l4ka.org/publications/paper.php?docid=1086

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel