This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
Home Products Support Community News


Re: [Xen-devel] Is there a way to map pv guest pseudo physical address i

To: Wei Liu <liuw@xxxxxxxxx>
Subject: Re: [Xen-devel] Is there a way to map pv guest pseudo physical address into dom0?
From: Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx>
Date: Mon, 27 Jun 2011 12:50:08 -0400
Cc: Tim Deegan <Tim.Deegan@xxxxxxxxxxxxx>, Ian Campbell <Ian.Campbell@xxxxxxxxxxxxx>, "xen-devel@xxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxx>, Anthony Perard <anthony.perard@xxxxxxxxxx>, Stefano Stabellini <stefano.stabellini@xxxxxxxxxxxxx>
Delivery-date: Mon, 27 Jun 2011 09:51:06 -0700
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <BANLkTikJPE8kQjFCsF4tbykUs-1ma7gGNg@xxxxxxxxxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
References: <BANLkTimW6NZRSR39WpSFrKfxCtDMCheMoA@xxxxxxxxxxxxxx> <1309165317.32717.160.camel@xxxxxxxxxxxxxxxxxxxxxx> <BANLkTikK6z-e-2oEXLA7UE4qHWxnB+ra8Q@xxxxxxxxxxxxxx> <1309166253.32717.167.camel@xxxxxxxxxxxxxxxxxxxxxx> <BANLkTi=KEr3zv_S=ng2EF=-uOMupCWwmXw@xxxxxxxxxxxxxx> <20110627093345.GH17634@xxxxxxxxxxxxxxxxxxxxxxx> <alpine.DEB.2.00.1106271209370.12963@kaball-desktop> <BANLkTikJPE8kQjFCsF4tbykUs-1ma7gGNg@xxxxxxxxxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
User-agent: Mutt/1.5.21 (2010-09-15)
On Mon, Jun 27, 2011 at 09:35:24PM +0800, Wei Liu wrote:
> Thank you, Tim and Stefano.
> On Mon, Jun 27, 2011 at 7:23 PM, Stefano Stabellini
> <stefano.stabellini@xxxxxxxxxxxxx> wrote:
> > On Mon, 27 Jun 2011, Tim Deegan wrote:
> >> At 17:26 +0800 on 27 Jun (1309195596), Wei Liu wrote:
> >> > > You need to arrange for the Xen specific bit of the virtio frontend to
> >> > > do the p2m translation. All externally visible addresses from a PV 
> >> > > guest
> >> > > must be in MFN space.
> >> > >
> >> >
> >> > That's not a good approach. Dirvers reside on a higher level from the
> >> > "Xen specific bit". Doing this kind of translation will break their
> >> > generality. But there seems no other way to get this job done...
> >>
> >> This is a general problem with operating systems: bus addresses are not
> >> always the same as CPU memory addresses (even though in the easy case of
> >> a simple x86 PC they often are).  This is why OSes already have
> >> interfaces to allocate DMA-able memory and to translate between the
> >> address that CPU will map and the one the peripheral should use.  If
> >> your driver's not using the right address-space to program its hardware
> >> it is broken; the Xen PV model is just an extreme case of this.
> >
> > Indeed.
> >
> > When passing addresses to devices, drivers should use the DMA APIs to
> > make sure that proper address conversions take place (see
> > Documentation/DMA-API-HOWTO.txt).
> > On Xen the swiotlb implementation takes care of doing pfn to mfn
> > conversions as well.
> > For example, to allocate a consistent buffer that is addressable by both
> > a device and the cpu, the correct API is dma_alloc_coherent, that
> > returns two values: the virtual address which you can use to access it
> > from the CPU and dma_handle which you pass to the device.
> >
> This is the first time to notice swiotlb TBH. It seems to be a good
> start for me.
> After skimming the code in $KERNEL/drivers/xen/swiotlb-xen.c, I have
> further questions.
> It seems that swiotlb is used for PCI passthrough in PV, I'm wondering
> if paravirt drivers ever use it.

And when running in Dom0.
> > I can very well imagine that virtio drivers are not doing any of this
> > because they didn't need to, but it would be probably OK to submit a
> > patch to convert the drivers to use the proper APIs.
> >
> No, they are not doing this.
> Do you mean that I replace every allocation/deallocation of virtio
> drivers scatter buffer with DMA API?

Use the pci_map_sg calls and follow it with pci_sync calls.
The DMA API mentions how to do it properly. Countless drivers do it.

> Then, with 'swiotlb=force' in pv guest kernel command line, these
> DMA-API will call xen-swiotlb and get proper addresses?

Correct. Thought you don't need to use 'swiotlb=force'. You can
just do 'iommu=soft' and that will take care of it in PV case.

In Dom0 case there is no need for this argument.
> If I don't add 'swiotlb=force', which underlying implementation will
> be used? And, which DMA-API will be used in hvm case? will this change
> affect hvm guest?

For HVM if the machine has less than 4GB it will use the no_dma implementation.
Which is to say: phys_to_virt and virt_to_phys for all DMA calls.

If you have more than 4GB (or iommu=soft) it will use the native
SWIOTLB implementation.

(And the native SWIOTLB can't properly do PFN -> MFN or vice versa).
> Wei.
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-devel

Xen-devel mailing list