This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
Home Products Support Community News


Re: [Xen-devel] [timer/ticks related] dom0 hang during boot on large 1TB

To: Keir Fraser <keir.fraser@xxxxxxxxxxxxx>
Subject: Re: [Xen-devel] [timer/ticks related] dom0 hang during boot on large 1TB system
From: Mukesh Rathor <mukesh.rathor@xxxxxxxxxx>
Date: Mon, 21 Dec 2009 11:52:52 -0800
Cc: "kurt.hackel@xxxxxxxxxx" <kurt.hackel@xxxxxxxxxx>, Dan Magenheimer <dan.magenheimer@xxxxxxxxxx>, "Xen-devel@xxxxxxxxxxxxxxxxxxx" <Xen-devel@xxxxxxxxxxxxxxxxxxx>, Jeremy Fitzhardinge <jeremy@xxxxxxxx>, Jan Beulich <JBeulich@xxxxxxxxxx>
Delivery-date: Mon, 21 Dec 2009 11:53:39 -0800
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <C75577FB.4E62%keir.fraser@xxxxxxxxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
References: <b014278a-1553-4cab-b8b5-9c63c1caeaab@default> <C75577FB.4E62%keir.fraser@xxxxxxxxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
On Mon, 21 Dec 2009 19:07:39 +0000
Keir Fraser <keir.fraser@xxxxxxxxxxxxx> wrote:

> On 21/12/2009 18:20, "Dan Magenheimer" <dan.magenheimer@xxxxxxxxxx>
> wrote:
> > Not to say the problem can't or shouldn't be fixed in Xen.
> > Keir, would bad things happen if construct_dom0 is done after
> > scrub_heap_pages()?  Other than some time wastage because
> > dom0's memory would get scrubbed just before it gets
> > overwritten (which is admittedly a much bigger problem
> > when dom0_mem is not specified in the Xen boot line
> > on a machine with ginormous memory).
> The problem is more likely that Xen system time started ticking some
> time earlier during boot process. I doubt it is to do with ordering of
> construct_dom0 versus boot-time scrubbing.
>  -- Keir

The problem is exactly how Dan described it. 'delta' for first interrupt
in dom0->timer_interrupt() goes up proportionately with amount of memory
on system. On this box, it appears more than 600GB causes delta to be 
large enough to wrap jiffies.

 1TB delta: 940b7d68a4
32GB delta: 02ae56eadb

xen->send_guest_vcpu_virq() ----> dom0->handle_IRQ() -> timer_interrupt()

timer_interrupt will call do_timer delta/NS_PER_TICK number of times.

Linux initializes jiffies to -5 minutes to catch problems from jiffies 
wrap early on. But like Dan said, dom0->calibrate_delay_direct() on 
baremetal starts running right away and is guaranteed to run in less 
than 5 minutes. We could let that assumption be true by moving page
scrub before xen->construct_dom0(), in which case the first timer
interrupt in dom0 will come in lot sooner, or just fix the loop to
account for wrap. Since jiffies just represents lower 32bits of 
jiffies_64, and get_jiffies_64() is provided for the purpose of reading
64bit version, I just avail of that. 


Xen-devel mailing list

<Prev in Thread] Current Thread [Next in Thread>