WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-devel

RE: [Xen-devel] TSC virtualization across different host frequency platf

To: "Tian, Kevin" <kevin.tian@xxxxxxxxx>, "Dong, Eddie" <eddie.dong@xxxxxxxxx>, xen-devel@xxxxxxxxxxxxxxxxxxx
Subject: RE: [Xen-devel] TSC virtualization across different host frequency platform migration
From: Dan Magenheimer <dan.magenheimer@xxxxxxxxxx>
Date: Thu, 23 Apr 2009 06:58:02 -0700 (PDT)
Cc: "Yang, Xiaowei" <xiaowei.yang@xxxxxxxxx>, Keir Fraser <keir.fraser@xxxxxxxxxxxxx>, "Zhang, Xiantao" <xiantao.zhang@xxxxxxxxx>
Delivery-date: Thu, 23 Apr 2009 06:59:00 -0700
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <0A882F4D99BBF6449D58E61AAFD7EDD6136777F7@xxxxxxxxxxxxxxxxxxxxxxxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
> that's always the case even w/o migration, since VM can be
> preempted at any time.

Not if it the vcpu is pinned, and that may often
be the case for database apps.

> That's really application specific, depending on the frequency
> of gettimeofday, e.g. in some database transation to stamp
> each record fastly. I had no exact data though. One of my
> colleagues (Xiaowei Yang) once solved one severe performance
> downgrade issue (orders of magnitude, >10%), when guest
> is observed falling back to use vHPET instead of TSC. The
> reason for fallback is not important here, but that actually inspires
> us to pay more attention here.

I'm suggesting that you measure and compare the cycle
cost of a TSC read and a vTSC read (and maybe also
a vHPET read), and also look at the code to see what
the bottleneck is for a vTSC read and if it can be made
faster.  And since your solution doesn't solve PV time,
maybe also measure a vTSC read in a PV guest.

I also agree with Keir that a tsc scaler might be a
good enhancement to request for future VT-x designs.

> -----Original Message-----
> From: Tian, Kevin [mailto:kevin.tian@xxxxxxxxx]
> Sent: Thursday, April 23, 2009 12:15 AM
> To: Dan Magenheimer; Dong, Eddie; xen-devel@xxxxxxxxxxxxxxxxxxx
> Cc: Keir Fraser; Zhang, Xiantao; Yang, Xiaowei
> Subject: RE: [Xen-devel] TSC virtualization across different
> host frequency platform migration
>
>
> >From: Dan Magenheimer [mailto:dan.magenheimer@xxxxxxxxxx]
> >Sent: 2009年4月23日 11:40
> >
> >OK, I think I understand, but I think you are
> >solving a very limited problem ("make sure that
> >a user/program that requests time-of-day gets
> >an answer which is close to accurate") but
> >not solving the broader class of time problems,
> >and you may be making them worse.  If your solution
> >is implemented and the OS or an application reads tsc,
> >the values obtained will be monotonically increasing
> >but will have large gaps, correct?
>
> that's always the case even w/o migration, since VM can be
> preempted at any time.
>
> >
> >If software-emulated tsc reads is really 10% loss
> >in system performance, I agree that this might be
> >the lesser of two evils.  But I don't believe the
> >10%.
>
> That's really application specific, depending on the frequency
> of gettimeofday, e.g. in some database transation to stamp
> each record fastly. I had no exact data though. One of my
> colleagues (Xiaowei Yang) once solved one severe performance
> downgrade issue (orders of magnitude, >10%), when guest
> is observed falling back to use vHPET instead of TSC. The
> reason for fallback is not important here, but that actually inspires
> us to pay more attention here.
>
> Thanks,
> Kevin
>
> >
> >> -----Original Message-----
> >> From: Tian, Kevin [mailto:kevin.tian@xxxxxxxxx]
> >> Sent: Wednesday, April 22, 2009 9:21 PM
> >> To: Dong, Eddie; Dan Magenheimer; xen-devel@xxxxxxxxxxxxxxxxxxx
> >> Cc: Keir Fraser; Zhang, Xiantao
> >> Subject: RE: [Xen-devel] TSC virtualization across different
> >> host frequency platform migration
> >>
> >>
> >> >From: Dong, Eddie
> >> >Sent: 2009年4月23日 9:29
> >> >
> >> >Dan Magenheimer wrote:
> >> >> Hi Eddie/Kevin --
> >> >>
> >> >> I'm sorry to be dense, but I don't understand the
> >> >> details of your solution.  I'm also not sure I
> >> >> understand the problem you are trying to solve.
> >> >> The problem description doesn't describe a problem,
> >> >> just an event.
> >> >>
> >> >> I'm guessing the problem is:  When a guest chooses
> >> >> TSC as its primary clocksource AND a migration later
> >> >> occurs to a host with a different TSC frequency
> >> >> THEN wallclock time (e.g. the "date" command)
> >> >> becomes incorrect.
> >> >
> >> >Mostly yes though don't know if guest wall clock depends on
> >> >TSC heavily.
> >> >
> >> >>
> >> >> I'm also guessing that you are NOT trying to solve
> >> >> the problem:  An application that uses TSC
> >> >> heavily to measure the passage of time AND
> >> >> calibrates TSC on host A AND invisibly
> >> >> migrates to host B with a different TSC
> >> >> frequency THEN will NOT be able to accurately
> >> >> measure the passage of time.  However, it will
> >> >> continue to be monotonically increasing.
> >> >
> >> >Yes, if we don't scale the TSC.
> >> >The proposal tries to solve the problem.
> >> >
> >> >We can use software trap and emulate to scale the TSC so
> >> >that guest TSC after migration is same with that before migration.
> >> >
> >> >But this is not optimial since the overhead may be too high. So we
> >> >propose to use smart scaling, which continuously use TSC_OFFSET,
> >> >but adjust the TSC_OFFSET value time to time (today it is fixed),
> >> >so that an application that uses TSC heavily to measure the
> >> >passage of time
> >> >can get correct time.
> >> >
> >>
> >> So we're really not trying to solve the micro-level accuracy if app
> >> tries to use TSC for that purpose, which always bias from bare
> >> metal as long as running in a VM. Here we're trying to ensure
> >> macro-level accuracy and thus ToD in guest after migration, with
> >> performance optimized if app in VM uses gettimeofday frequently.
> >> In that way, gap between trap vs. notrap would be orders of
> >> magnitude.
> >>
> >> Thanks
> >> Kevin
> >

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel