This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
Home Products Support Community News


[Xen-devel] Re: [dm-devel] [RFC] dm-userspace

To: mingz@xxxxxxxxxxx
Subject: [Xen-devel] Re: [dm-devel] [RFC] dm-userspace
From: Dan Smith <danms@xxxxxxxxxx>
Date: Tue, 09 May 2006 16:02:09 -0700
Cc: device-mapper development <dm-devel@xxxxxxxxxx>, Xen Developers <xen-devel@xxxxxxxxxxxxxxxxxxx>, linux-kernel@xxxxxxxxxxxxxxx
Delivery-date: Tue, 09 May 2006 16:02:05 -0700
Envelope-to: www-data@xxxxxxxxxxxxxxxxxx
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
References: <87u08g553l.fsf@xxxxxxxxxxxxxxxxxxxxxxxxxx> <1146092129.14129.333.camel@xxxxxxxxxxxxxxxxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
User-agent: Gnus/5.110006 (No Gnus v0.6) Emacs/21.4 (gnu/linux)
(I'm including the xen-devel list on this, as things are starting to
get interesting).

MZ> do u have any benchmark results about overhead?

So, I've spent some time over the last week working to improve
performance and collect some benchmark data.

I moved to using slab caches for the request and remap objects, which
helped a little.  I also added a poll() method to the control device,
which improved performance significantly.  Finally, I changed the
internal remap storage data structure to a hash table, which had a
very large performance impact (about 8x).

Copying data to a device backed by dm-userspace presents a worst-case
scenario, especially with a small block-size like what qcow uses.  In
one of my tests, I copy about 20MB of data to a dm-userspace device,
backed by files hooked up to the loopback driver.  I compare this with
a "control" of a single loop-mounted image file (i.e., without
dm-userspace or CoW).  I measured the time to mount, copy, and unmount
the device, which (with the recent performance improvements) are

  Normal Loop:        1 seconds
  dm-userspace/qcow: 10 seconds

For comparison, before adding poll() and the hash table, the
dm-userspace number was over 70 seconds.

One of the most interesting cases for us, however, is providing a
CoW-based VM disk image, which is mostly used for reading, with a
small amount of writing for configuration data.  To test this, I used
Xen to compare a fresh FC4 boot (firstboot, where things like SSH keys
are generated and written to disk) that used an LVM volume as root to
using dm-userspace (and loopback-files) as the root.  The numbers are

  LVM root:          26 seconds
  dm-userspace/qcow: 27 seconds

Note that this does not yet include any read-ahead type behavior, nor
does it include priming the kernel module with remaps at create-time
(which results in a few initial compulsory "misses").  Also, I removed
the remap expiration functionality while adding the hash table and
have not yet added it back, so that may further improve performance
for large amounts of remaps (and bucket collisions).

Here is a link to a patch against


Here are links to the userspace library, as well as the cow daemon,
which provides qcow support:


(Note that the daemon is still rather rough, and the qcow
implementation has some bugs.  However, it works for light testing and
the occasional luck-assisted heavy testing)

As always, comments welcome and appreciated :)

Dan Smith
IBM Linux Technology Center
Open Hypervisor Team
email: danms@xxxxxxxxxx

Attachment: pgpeXmoeSmGRS.pgp
Description: PGP signature

Xen-devel mailing list
<Prev in Thread] Current Thread [Next in Thread>