This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
Home Products Support Community News


Re: [Xen-devel] [PATCH] Fix locking bug in vcpu_migrate

To: Keir Fraser <keir.xen@xxxxxxxxx>
Subject: Re: [Xen-devel] [PATCH] Fix locking bug in vcpu_migrate
From: John Weekes <lists.xen@xxxxxxxxxxxxxxxxxx>
Date: Fri, 22 Apr 2011 15:33:26 -0700
Cc: "xen-devel@xxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxx>, George Dunlap <george.dunlap@xxxxxxxxxx>
Delivery-date: Fri, 22 Apr 2011 15:34:22 -0700
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <C9D78ADC.16C2F%keir.xen@xxxxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
References: <C9D78ADC.16C2F%keir.xen@xxxxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
User-agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv: Gecko/20110303 Thunderbird/3.1.9
On 4/22/2011 11:43 AM, Keir Fraser wrote:
It's odd that it seemed to lead to such a big difference for me, then.
>  I'll do some further tests -- maybe I changed something else to cause
>  the behavior, or the problem is more random than I thought and just
>  hasn't occurred for me yet in all the new tests.

I did further testing and determined that my domU was starting properly because I had only tested once or twice with Debian Squeeze after applying the patch; I had then done more extensive testing only under a Win2k3 domU. It seems that Win2k3 domUs don't have the same issue.

Back on the Squeeze domU, I am reliably seeing the BUG again, with either configuration.

I have rolled back schedule.c to pre-22948, when it was much simpler, and that seems to have resolved this particular bug.

Now, a different credit2 bug has occurred, though only once for me so far; with the other bug, I was seeing a panic with every 1 or 2 domU startups, but I have seen the new bug on one test out of 15.

Specifically, I have triggered the BUG_ON in csched_domcntl. The line number is not the standard one because I have added further debugging, but the BUG_ON is:

BUG_ON(svc->rqd != RQD(ops, svc->vcpu->processor));

The bt being:

(XEN)    [<ffff82c480119578>] csched_dom_cntl+0x11a/0x185
(XEN)    [<ffff82c48011f24d>] sched_adjust+0x102/0x1f9
(XEN)    [<ffff82c480102ee5>] do_domctl+0xb25/0x1250
(XEN)    [<ffff82c4801ff0e8>] syscall_enter+0xc8/0x122

Also, in three of those last 15 startups, my domU froze three times (consuming no CPU and seemingly doing nothing), somewhere in this block of code in ring_read in tools/firmware/hvmloader/xenbus.c -- I added debug information that allowed me to narrow it down. This function is being called when it is writing the SMBIOS tables. I can't tell whether this is related to the credit2 problem. (The domU can be "destroyed" to get out of it).

        /* Don't overrun the producer pointer */
        while ( (part = MASK_XENSTORE_IDX(rings->rsp_prod -
                                          rings->rsp_cons)) == 0 )
        /* Don't overrun the end of the ring */
if ( part > (XENSTORE_RING_SIZE - MASK_XENSTORE_IDX(rings->rsp_cons)) )
            part = XENSTORE_RING_SIZE - MASK_XENSTORE_IDX(rings->rsp_cons);
        /* Don't read more than we were asked for */
        if ( part > len )
            part = len;

Note that I am using stubdoms.

I would be happy to temporarily turn over the reins on this machine to you or George, if you'd like to debug any of these issues directly. I may not be able to continue experimenting in the short term here myself due to time constraints.


Xen-devel mailing list