WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-devel

[Xen-users] Re: [Xen-devel] Xen 4 TSC problems

To: Olivier Hanesse <olivier.hanesse@xxxxxxxxx>, Dan Magenheimer <dan.magenheimer@xxxxxxxxxx>
Subject: [Xen-users] Re: [Xen-devel] Xen 4 TSC problems
From: Keir Fraser <keir.xen@xxxxxxxxx>
Date: Mon, 28 Feb 2011 15:39:48 +0000
Cc: Jeremy Fitzhardinge <jeremy@xxxxxxxx>, xen-devel@xxxxxxxxxxxxxxxxxxx, Keir Fraser <keir@xxxxxxx>, Jan Beulich <JBeulich@xxxxxxxxxx>, Xen Users <xen-users@xxxxxxxxxxxxxxxxxxx>, Mark Adams <mark@xxxxxxxxxxxxxxxxxx>
Delivery-date: Mon, 28 Feb 2011 07:41:11 -0800
Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:user-agent:date:subject:from:to:cc:message-id :thread-topic:thread-index:in-reply-to:mime-version:content-type :content-transfer-encoding; bh=4OJ9Gwjvpp6bYHAqjLzrLEIEfSR2UIaDYe1cdvbFrUQ=; b=To/Ea5wNlH2MXHVt/IDVgk4odRhMY0ZSpl2YBhDNxTvOntBG1xsAmr8OjZO/0KDk7f rYzr3/POVe4cWrefNGWutpR8e2cJT0wZawreciPqa7cY1xkydxomXxT0aLfJd+2TytGq Y7KfCx1EKPWuJv43/ikKlQU4I8BOVUbFWptxs=
Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=user-agent:date:subject:from:to:cc:message-id:thread-topic :thread-index:in-reply-to:mime-version:content-type :content-transfer-encoding; b=wdIcdHShehhetm2Dzl+LQkxwHOqeSoQVq9N0HdGtICCj1OG9TpdWpEfie1LH9aVT3d wBZPUMuJkSB11VXiKLbGWZVSc8tAlq8RVVgZKTcqFbPfkeJVHg+JeXsGzRBW+8smC1sA OS+/aOWbMnBWroCmFQ/84XME3Ow7G2N75VkFo=
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <AANLkTikptc2POrKQgJuoVZRdwJTo64DJ_hm12KuPky4D@xxxxxxxxxxxxxx>
List-help: <mailto:xen-users-request@lists.xensource.com?subject=help>
List-id: Xen user discussion <xen-users.lists.xensource.com>
List-post: <mailto:xen-users@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=unsubscribe>
Sender: xen-users-bounces@xxxxxxxxxxxxxxxxxxx
Thread-index: AcvXXbsI77bY+VqPdUeJj4CGzXPNaQ==
Thread-topic: [Xen-devel] Xen 4 TSC problems
User-agent: Microsoft-Entourage/12.28.0.101117
On 28/02/2011 15:23, "Olivier Hanesse" <olivier.hanesse@xxxxxxxxx> wrote:

> Keir : 
> 
> Yes, it is "under progress". 
> To make this change, I had to reboot every server, so it is taking time
> (production server :()
> So i was hoping to find a quick method to mitigate this issue on domUs while
> rebooting servers.
> 
> As this bug happens once or twice per server since October, I can't say that
> right now that changing platform timer to PIT fixed it. I have to wait (I hope
> forever!) this bug to happen again on a 'patched' server ... 
> 
> But even with clcoksource=pit, I am seeing some warp=3000+ in debug message ?
> I guess it is not a good sign, is it ?

Better not to have it, but honestly you're very unlikely to see any problem
from it. It's totally unrelated to the 3000-second time jumps.

 -- Keir

> Jan : I was hoping to find a way to make the domU clocksource more
> "independent" like with xen3.2.
> 
> 
> 2011/2/28 Dan Magenheimer <dan.magenheimer@xxxxxxxxxx>
>> Hi Olivier ­
>>  
>> It is the Xen clocksource that you want to try to change, not the dom0
>> clocksource.  To do this, you need to specify ³clocksource=pit² on the Xen
>> boot line (and reboot), not the dom0 boot line.
>>  
>> I believe Mark Adams played with tsc_mode to see if it solved! his (similar?
>> identical?) problem last year, and it didn¹t make any difference.
>> 
>> Please try booting Xen with ³clocksource=pit² and ensure that ³Platform timer
>> is 1.19MHz PIT² appears in the Xen boot messages.  If the 50min jump does not
>> appear again, it would point to a problem in the hpet, either hardware or
>> software.
>>  
>> Thanks,
>> Dan
>>  
>> 
>> From: Olivier Hanesse [mailto:olivier.hanesse@xxxxxxxxx]
>> Sent: Monday, February 28, 2011 7:37 AM
>> To: Jeremy Fitzhardinge
>> Cc: Dan Magenheimer; Keir Fraser; Jan Beulich; Mark Adams;
>> xen-devel@xxxxxxxxxxxxxxxxxxx; Xen Users; Keir Fraser
>> 
>> 
>> Subject: Re: [Xen-devel] Xen 4 TSC problems
>>  
>> 
>> Hello,
>> 
>>  
>> It happened again twice this weekend.
>> 
>>  
>> 
>> What about setting "tsc_mode=2" for my vms ? Should this mode prevent this
>> bug (coming from a bad emulated tsc due to firmware issue ? is it possible ?)
>> from affecting time in domUs ?
>> 
>>  
>> 
>> Setting clocksource=pit, make 'tsc' available in
>> "/sys/devices/system/clocksource/clocksource0/available_clocksource"
>> (otherwise only xen is available, is it normal ? ). 
>> 
>>  
>> 
>> Should I bypass xen clocksource and use tsc as a clocksource for dom0/domU ?
>> or  will it be worsed ?
>> 
>>  
>> 
>> Regards
>> 
>>  
>> 
>> Olivier
>> 
>>  
>> 
>> 2011/2/24 Jeremy Fitzhardinge <jeremy@xxxxxxxx>
>> 
>> On 02/24/2011 09:43 AM, Dan Magenheimer wrote:
>>> Just a wild guess, but this in Olivier's posted output:
>>> 
>>> (XEN) Platform timer appears to have unexpectedly wrapped 10 or more times.
>>> 
>>> and the fact that a 32-bit HPET wrap is ~300 seconds and, with the
>>> "10 or more times", 10 * 300 seconds is 3000 seconds, might be a clue
>>> (or a complete red herring, but I thought it worth mentioning).
>>> 
>>> Mark and Olivier, it would be interesting to know if you are
>>> using the same processor/system.
>> It definitely seems like some kind of problem on the host system rather
>> than anything in the guests themselves. !  If the platform timer is
>> misbehaving, then Xen could be completely screwing up the pvclock
>> calibration which it then passes to guests.
>> 
>> Could it be one of those "platform clock stops in certain power states"
>> problems?
>> 
>> 
>>    J
>> 
>>>> -----Original Message-----
>>>> From: Keir Fraser [mailto:keir.xen@xxxxxxxxx]
>>>> Sent: Thursday, February 24, 2011 7:52 AM
>>>> To: Olivier Hanesse; Jan Beulich
>>>> Cc: Mark Adams; Jeremy Fitzhardinge; xen-devel@xxxxxxxxxxxxxxxxxxx; Xen
>>>> Users; Dan Magenheimer; Keir Fraser
>>>> Subject: Re: [Xen-devel] Xen 4 TSC problems
>>>> 
>>>> On 24/02/2011 14:20, "Olivier Hanesse" <olivier.hanesse@xxxxxxxxx>
>>>> wrote:
>>>> 
>>>>> Both dom0 and domUs are affected by this" jump".
>>>>> 
>>>>> I expect to see something like "TSC marked as reliable, warp = 0".
>>>>> I got this on newer hardware with same config/distros.
>>>> It depends on the CPU itself, older CPUs do not have the super-stable
>>>> TSC
>>>> features. But that should never cause a massive 3000s time jump.
>>>> 
>>>>> Is there a way to measure if it is a TSC warp ? to point out a cpu
>>>> tsc issue ?
>>>> 
>>>> The TSC warps or out-of-sync issues that we could reasonably expect
>>>> would be
>>>> on the order of microseconds. A 3000s warp is something else entirely.
>>>> Xen
>>>> is very confused and/or some TSC or platform timer has jumped a long
>>>> way
>>>> (indicating a hardware/firmware issue).
>>>> 
>>>>  -- Keir
>>>> 
>>> &gt! ;> 2011/2/24 Jan Beulich <JBeulich@xxxxxxxxxx>
>> 
>>>>>>>>> On 24.02.11 at 12:57, Olivier Hanesse <olivier.hanesse@xxxxxxxxx>
>>>> wrote:
>>>>>>> I tried to turn off cstates with max_cstate=0 without success
>>>> (still "not
>>>>>>> reliable").
>>>>>>> 
>>>>>>> With cpuidle=0, I also got :
>>>>>>> 
>>>>>>> (XEN) TSC has constant rate, deep Cstates possible, so not
>>>> reliable,
>>>>>>> warp=3022 (count=1)
>>>>>> This message by itself isn't telling much I believe.
>>>>>> 
>>>>>>> xm info | grep command
>>>>>>> xen_commandline        : dom0_mem=512M cpuidle=0 loglvl=all
>>>> guest_loglvl=all
>>>>>>> dom0_max_vcpus=1 dom0_vcp! us_pin console=vga,com1 com1=19200,8n1
>> 
>>>>>>> 
>>>>>>> Keir :
>>>>>>> 
>>>>>>> Using clocksource=pit :
>>>>>>> 
>>>>>>> (XEN) Platform timer is 1.193MHz PIT
>>>>>>> 
>>>>>>> I also got :
>>>>>>> 
>>>>>>> (XEN) TSC has constant rate, deep Cstates possible, so not
>>>> reliable,
>>>>>>> warp=3262 (count=2)
>>>>>> The question is whether any of this eliminates the time jumps seen
>>>>>> by your DomU-s (from your past mails I wasn't actually sure whether
>>>>>> Dom0 also experienced this problem, albeit it would be odd if it
>>>> didn't).
>>>>>> Jan
>>>>>> 
>>>>>> Jan
>>>>>> 
>>>>> 
>>>> 
>>  
> 
> 



_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users

<Prev in Thread] Current Thread [Next in Thread>