Re: [XenPPC] Xencomm for xen/ia64

[Hollis Blanchard <hollisb@xxxxxxxxxx>]

(CCed to xen-devel for completeness. ;)

On Wed, 2006-08-16 at 17:24 +0200, Tristan Gingold wrote:
> I am porting xen-ppc's xencomm to xen/ia64.
> Currently on xen/ia64 copy_from/to_guest uses guest virtual address.  This 
> works well as long as the virtual addresses are in the TLB.  When not in TLB 
> (or vTLB) the hypercall can't success without domain help.  The possible 
> solution is either to touch the memory areas before doing the hypercall 
> and/or restarting the hypercall.
> 
> Touching the memory area is an hack and we can't be sure it works.
> Restarting the hypercall is not always possible (some hypercalls are atomic: 
> DOM0_SHADOW_CONTROL_OP_CLEAN) or can result in a live-lock.
> 
> The most simple solution is to use guest physical addresses instead of 
> virtual 
> addresses.

Absolutely agreed.

> For hypercalls directly issued by the kernel, the translation is very easy.  
> For hypercalls (indirectly) issued by dom0 though the ioctl, kernel has to do 
> the translation.  Because it may not be linear in guest physical memory the 
> parameter is a pointer to a list of page (xencomm).
> 
> The pros of such approach is simplicity and reliability.
> 
> The main cons is maybe speed.  Hopefully the most frequent hypercalls (dom0vp 
> and eoi) either don't use in memory parameters (dom0vp) or may be modified so 
> that they pass parameters through registers (eoi).  IMHO speed won't be 
> affected.
> 
> Access to guest memory is also performed during vcpu_translate (to read vhpt) 
> or EFI/PAL/SAL calls.  We can either do not change that code (ie both 
> mechanisms are not exclusive) or change the code.  This point will be 
> postpone.

Right, by switching to the xencomm approach for copy_*_guest(), you lose
the ability to copy_*_guest() with arbitrary addresses, because
copy_*_guest() *requires* that the guest kernel has passed in a xencomm
structure.

x86 has a copy_*_user() implementation, which is used only in x86 code
and is independent of the copy_*_guest() API. You could create a similar
parallel API if you need to.

> If we agree on using xencomm we will have to work with xen/ppc people in 
> order 
> not to duplicate the code.  Hopefully it is rather small.  I have enhanced 
> the xencomm code so that the kernel may not use xencomm area but pass the 
> guest physical address with a flag to know the space is linear in memory.
> 
> At this time I can boot dom0 with xencomm.  I will publish the patch later.

I'll be very interested to see your patch. I guess the "flag" is a
reserved bit in the (physical) address passed from kernel to hypervisor?
Does that really gain much performance?

I guess you will need to do the same thing we need to with privcmd ioctl
handling, i.e. copy and modify the pointers in the dom0_op data
structures passed to the kernel. :(

We need to do one more thing though: we *also* need to change fix up the
size of longs and pointers in our code (since 32-bit userland is passing
structures to a 64-bit kernel). So perhaps these two fixup passes could
be split: we could share the xencomm conversion in common code, and PPC
arch code could contain the size munging.

This is why passing complex data structures as hcall parameters will
always be a bad thing.

-- 
Hollis Blanchard
IBM Linux Technology Center


_______________________________________________
Xen-ppc-devel mailing list
Xen-ppc-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-ppc-devel