| |
|
 |
|
 |
|
|
|
| |
|
|
xen-devel
[Xen-devel] Re: Improving hvm IO performance by using self IO emulator (
Hi Tristan,
Thanks for posting this.
Tristan Gingold wrote:
Summary: I am proposing a new method to improve hvm IO emulation: the IO
requests are reflected to the domain firmware which emulates the IO using PV
drivers. The pros of this method are minor hypervisor modifications, smooth
transition, performance improvement and convergence with PV model
Discussion:
The current IO emulator (ioemu process in dom-0) is a well known bottleneck
for hvm performance because IO requests travel is long and cross many rings.
I'm not quite sure that I agree this is the bottleneck. If IO latency
were the problem, then a major reduction in IO latency ought to
significantly improve performance right?
KVM has a pretty much optimal path from the kernel to userspace. The
overhead of going to userspace is roughly two syscalls (and we've
measured this overhead). Yet it makes almost no difference in IO
throughput.
The big problem with disk emulation isn't IO latency, but the fact that
the IDE emulation can only have one outstanding request at a time. The
SCSI emulation helps this a lot.
I don't know what the bottle neck is in network emulation, but I suspect
the number of copies we have in the path has a great deal to do with it.
Many ideas to improve the emulation have been proposed. None of them have
been adopted because their approach are too disruptive.
Based on my recent firmware experience I'd like to propose a new method.
The principle is rather simple: the hvm domain does all the work. IO requests
are simply reflected to the domain. When the hypervisor decodes an IO
request it sends it to the domain using a SMI(x86)/PMI(ia64)-like
interruption. This reflection saves some registers, put parameters (IO req)
into registers and call the firmware at defined address using a defined mode
(physical mode should be the best). The firmware handles the IO request like
ioemu does but use PV drivers (net, blk, fb...) to access to external
resources. It then resumes the domain execution through an hypercall which
restores registers and mode.
I think there are many pros to this approach:
* the changes in the hypervisor are rather small: only the code to do the
reflection has to be added. This is a well-known and light mechanism.
* the transition can be smooth: this new method can co-exist in several way
with the current method. First it can be used only when enabled. Then once
the reflection code is added in the hypervisor the firmware can just send the
IO request to ioemu like the hypervisor already does. The in domain IO
emulation can be added driver per driver (eg: IDE disk first, then network,
then fb).
This smooth transition is a major advantage to early evaluate this new method.
* Because all the emulation work is done in the domain the work in accounted
to this domain and not to another domain (dom0 today). This is good for
management and for security.
* From the hypervisor point of view such an hvm domain looks like a PV domain:
only the creation differs. This IO emulation method unifies the domain. This
will simplify save & restore and Xen in general.
* Performance should be improved compared to the current io emulation method:
the IO request travel is shorter. If we want to work on performance we could
later handle directly some IO requests in the hypervisor (I think of ports or
iomem which don't have side-effect).
I don't see a lot of cons, the major one is 'porting' ioemu code to
firmware code. This is the challenge. But qemu seems to be well structured.
Most of the files might be ported without changes, the core has of course to
be rewritten. The PV drivers should also be ported.
SMP can be first handled with a global lock and later concurrent accesses may
be allowed. This may improve performance compared to ioemu which is almost
single threaded.
There's a lot to like about this sort of approach. It's not a silver
bullet wrt performance but I think the model is elegant in many ways.
An interesting place to start would be lapic/pit emulation. Removing
this code from the hypervisor would be pretty useful and there is no
need to address PV-on-HVM issues.
Can you provide more details on how the reflecting works? Have you
measured the cost of reflection? Do you just setup a page table that
maps physical memory 1-1 and then reenter the guest?
Does the firmware get loaded as an option ROM or is it a special portion
of guest memory that isn't normally reachable?
Regards,
Anthony Liguori
I don't know yet how to use the PV-on-HVM drivers. There is currently only
one page to communicate with xenstore. We can try to share this page
between the firmware and the PV-on-HVM drivers or we may create a second
page.
I have thought of this new IO emulation method during my work on EFI gfw for
ia64. Recently I have looked more deeply into the sources. I can't see any
stopper yet. Unless someone has a strong point against this method I hope
I will be able to work on it shortly (ia64 first - sorry!)
Comments are *very* welcome.
Tristan.
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel
|
|
|
| |
|
Home