Hi Mats,
I once posted my questions about the behavior of Asynchronous I/O under Xen
which is also
directly related disk I/O performance. however I did not get any response. I
would appreciate
if you can advise about this.
As AIO can help improve better performance and Linux kernel keeps tuning the
AIO path, more and more IOs can be expected to take AIO path instead of
regular I/O path.
First Question:
If we consider Xen, do we need to do AIO both in the domain0 and guest
domains at the same? For example, considering two situations, let a full
virtualized guest domain still do regular I/O and domain0 (vbd back end
driver) do AIO; or let both full-virtualized guest domain and domain0 do
AIO. What is possible performance difference here?
Second Question:
Does Domain0 always wait till AIO data is available and then notify guest
domain? or Domain0 will issue an interrupt immediately to notify guest
domain0 when AIO is queued? If the first case is true, then all AIOs will
become synchronous.
Third Question:
Does Xen hypervisor change the behavior of Linux I/O scheduler more or less?
Four Question:
Will AIO have different performance impact on para-virtualized domain and
full-virtualized domain respectively?
Thanks,
Liang
----- Original Message -----
From: "Petersson, Mats" <Mats.Petersson@xxxxxxx>
To: "Mark Williamson" <mark.williamson@xxxxxxxxxxxx>;
<xen-users@xxxxxxxxxxxxxxxxxxx>
Cc: "Tom Horsley" <tomhorsley@xxxxxxxxxxxx>; "Goswin von Brederlow"
<brederlo@xxxxxxxxxxxxxxxxxxxxxxxxxxx>; "James Rivera" <jrivera@xxxxxxxxxxx>
Sent: Tuesday, January 16, 2007 10:22 AM
Subject: RE: [Xen-users] Getting better Disk IO
-----Original Message-----
From: xen-users-bounces@xxxxxxxxxxxxxxxxxxx
[mailto:xen-users-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of
Mark Williamson
Sent: 16 January 2007 17:07
To: xen-users@xxxxxxxxxxxxxxxxxxx
Cc: Tom Horsley; Goswin von Brederlow; James Rivera
Subject: Re: [Xen-users] Getting better Disk IO
> I've been hoping to see replies to this, but lacking good
information
> here is the state of my confusion on virtual machine disks:
>
> If you read the docs for configuring disks on domu and hvm machines,
> you'll find a gazillion or so ways to present the disks to
the virtual
> machine.
There are quite a lot of options, it's true ;-)
> One of those ways (who's name I forget) provides (if I
understand things,
> which I doubt :-), provides a special kind of disk
emulation designed to
> be driven by special drivers on the virtual machine side.
The combination
> gives near direct disk access speeds in the virtual machine.
>
> The catch is that you need those drives for the kernel on
the virtual
> machine side. They may already exist, you may have to build
them, and
> depending on the kernel version, they may be hard to build.
>
> Perhaps someone who actually understands this could elaborate?
Basically yes, that's all correct.
To summarise:
PV guests (that's paravirtualised, or Xen-native) use a
Xen-aware block device
that's optimised for good performance on Xen.
HVM guests (Hardware Virtual Machine, fully virtualised and
unaware of Xen)
use an emulated IDE block device, provided by Xen (actually,
it's provided by
the qemu-based device models, running in dom0).
The HVM emulated block device is not as optimised (nor does
it lend itself to
such effective optimisation) for high virtualised performance as the
Xen-aware device. Therefore a second option is available for
HVM guests: an
implementation of the PV guest device driver that is able to
"see through"
the emulated hardware (in a secure and controlled way) and
talked directly as
a Xen-aware block device. This can potentially give very
good performance.
The reason the emulated IDE controller is quite slow is a consequence of
the emulation. The way it works is that the driver in the HVM domain
writes to the same IO ports that the real device would use. These writes
are intercepted by the hardware support in the processor and a VMEXIT is
issued to "exit the virtual machine" back into the hypervisor. The HV
looks at the "exit reason", and sees that it's an IO WRITE operation.
This operation is then encoded into a small packet and sent to QEMU.
QEMU processes this packet and responds back to HV to say "OK, done
that, you may continue". HV then does a VMRUN (or VMRESUME in the Intel
case) to continue the guest execution, which is probably another IO
instruction to write to the IDE controller. There's a total of 5-6 bytes
written to the IDE controller per transaction, and whilst it's possible
to combine some of these writes into a single write, it's not always
done that way. Once all writes for one transaction are completed, the
QEMU ide emulation code will perform the requested operation (such as
reading or writing a sector). When that is complete, a virtual interrupt
is issued to the guest, and the guest will see this as a "disk done"
interrupt, just like real hardware.
All these steps of IO intercepts takes several thousand cycles, which is
a bit longer than a regular IO write operation would take on the real
hardware, and the system will still need to issue the real IO operations
to perform the REAL hardware read/write corresponding to the virtual
disk (such as reading a file, LVM or physical partition) at some point,
so this is IN ADDITION to the time used by the hypervisor.
Unfortunately, the only possible improvement on this scenario is the
type "virtual-aware" driver that is described below.
[Using a slightly more efficient model than IDE may also help, but
that's going to be marginal compared to the benefits of using a
virtual-aware driver].
--
Mats
I don't know if these drivers are included in any Linux
distributions yet, but
they are available in the Xen source tree so that you can
build your own, in
principle. Windows versions of the drivers are included in
XenSource's
products, I believe - including the free (as in beer)
XenExpress platform.
There are potentially other options being developed,
including an emulated
SCSI device that should improve the potential for higher
performance IO
emulation without Xen-aware drivers.
Hope that clarifies things!
Cheers,
Mark
--
Dave: Just a question. What use is a unicyle with no seat?
And no pedals!
Mark: To answer a question with a question: What use is a skateboard?
Dave: Skateboards have wheels.
Mark: My wheel has a wheel!
_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users
_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users
_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users
|
Home