This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
Home Products Support Community News


[Xen-devel] Re: Xen's use of PAT and PV guests

To: Jeremy Fitzhardinge <jeremy@xxxxxxxx>
Subject: [Xen-devel] Re: Xen's use of PAT and PV guests
From: Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx>
Date: Tue, 30 Mar 2010 12:57:13 -0400
Cc: Xen-devel <xen-devel@xxxxxxxxxxxxxxxxxxx>, Keir Fraser <keir.fraser@xxxxxxxxxxxxx>, Jan Beulich <JBeulich@xxxxxxxxxx>
Delivery-date: Tue, 30 Mar 2010 10:00:15 -0700
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <4BB1476D.4060209@xxxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
References: <4BB1476D.4060209@xxxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
User-agent: Mutt/1.5.19 (2009-01-05)
On Mon, Mar 29, 2010 at 05:35:57PM -0700, Jeremy Fitzhardinge wrote:
> I'm looking again at what it will take to reconcile Xen's PAT setup with  
> the standard Linux one so that we can enable PAT use in pvops kernels.
> Just for reference, this is the Linux vs Xen vs default PAT setups:

And this LKML is good a primer:


> Index PTE flags       Linux   Xen     Default
> 0                     WB      WB      WB
> 1             PWT     WC      WT      WT
> 2         PCD         UC-     UC-     UC-
> 3         PCD PWT     UC      UC      UC
> 4     PAT             WB      WC      WB
> 5     PAT     PWT     WC      WP      WT
> 6     PAT PCD         UC-     UC      UC-
> 7     PAT PCD PWT     UC      UC      UC
> Originally I was thinking of a moderately complex scheme in which an ELF  
> node on the dom0 kernel could determine the system-wide Xen PAT MSR, and  
> then the kernel ELF notes on subsequent domains would determine whether  
> the PAT CPU feature flag is enabled or not.
> However this has several problems:
>   1. it is fairly complex
>   2. if dom0 sets the PAT configuration to something strange, it may
>      completely break other PV guests entirely (since it might
>      effectively change the meaning of PCD+PWT globally)

How does this work on pages shared across domains? Say Guest A makes the
page WC,Dom0 makes it WB and Xen puts it in WC, and Dom0 reads does a
Write/Read/Write, but in actuallity it is a Read/Write/Write. Or is
there no danger there since the grant table pages have UC set on them?

>   3. disabling the PAT CPU feature flag is meaningless, as its only
>      effect is to say "there's no PAT, so PCD/PWT have their default
>      behaviours", which is definitely not true in general
> Linux only uses the first 4 PAT entries, and repeats it, effectively  
> making the PAT pte flag a don't-care.  In those 4 entries, the Linux,  
> Xen and Default configurations are identical aside from Linux using WC  
> rather than WT.
> It therefore seems to me that if I make Linux:
>   1. never set the PAT flag (which it won't anyway),
>   2. check that the value written to IA32_PAT is as expected, but
>      otherwise ignore it, and
>   3. use WT rather than WC

That would make all writes synchronous. Why not write back?
> then it all should just work.  I'm not completely confident in the third  
> point though, since I'm not quite sure about the full set of differences  
> between WT and WC, and their respective interactions with the MTRR, and  
> whether that would break anything.  At first glance it seems pretty safe  
> though...

The graphics cards (and the XServer) are the ones that come in my mind
as heavy users of having this "just right".  But in most (all?) cases
they want it to be UC or better UC- so that shouldn't affect this.


Ah, but of course, there is an exception:

(i915_dma.c): gtt = ioremap_wc(pci_resource_start(dev->pdev, gtt_bar) + 
gtt_offset, gtt_size);

and then 'ttm_bo_ioremap' which does the ioremap_wc if  TTM_PL_FLAG_WC is set.

And it looks to be is set by the Radeon (if card is an AGP) and nouveau
on their first memory BAR.  Also the vmwgfx (VMWare) driver sets this, but we 
don't have
to worry about that.

So I think setting it to WC->WT would mean that graphics performance
would go down the drain. But then, there are some lingering issues with the
TTM/DRM infrastructure that need to tracked down and I believe 
Arvind and Michael are actively looking at that.

Xen-devel mailing list