This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
Home Products Support Community News


Re: [Xen-devel] Solution for problems with HyperSCSI and vbds ?

To: Ian.Pratt@xxxxxxxxxxxx
Subject: Re: [Xen-devel] Solution for problems with HyperSCSI and vbds ?
From: "Sven Kretzschmar" <sven.kretzschmar@xxxxxx>
Date: Wed, 15 Oct 2003 23:19:02 +0200
Cc: xen-devel@xxxxxxxxxxxxxxxxxxxxx
Delivery-date: Wed, 15 Oct 2003 22:20:47 +0100
Envelope-to: steven.hand@xxxxxxxxxxxx
In-reply-to: <E1A9pkt-0001lT-00@xxxxxxxxxxxxxxxxxxxx>
List-archive: <http://sourceforge.net/mailarchive/forum.php?forum=xen-devel>
List-help: <mailto:xen-devel-request@lists.sourceforge.net?subject=help>
List-id: List for Xen developers <xen-devel.lists.sourceforge.net>
List-post: <mailto:xen-devel@lists.sourceforge.net>
List-subscribe: <https://lists.sourceforge.net/lists/listinfo/xen-devel>, <mailto:xen-devel-request@lists.sourceforge.net?subject=subscribe>
List-unsubscribe: <https://lists.sourceforge.net/lists/listinfo/xen-devel>, <mailto:xen-devel-request@lists.sourceforge.net?subject=unsubscribe>
References: <E1A9pkt-0001lT-00@xxxxxxxxxxxxxxxxxxxx>
Reply-to: sven.kretzschmar@xxxxxx
Sender: xen-devel-admin@xxxxxxxxxxxxxxxxxxxxx
On 15.10.2003 at 18:48 Ian Pratt wrote:

>> >[Ian:]The main thing would be turning the VFR into more of an L2 switch
>> >than a router, with each domain having its own MAC[*]. We could then
>> >add a rule to grant a domain TX permission for a particular 802
>> >protocol number. HyperSCSI presumably has some high-level
>> >server-based authentication and privilege verification? If so, it

Yes, it has (even encryption, if needed).

>> >should be pretty straightforward.
>> This is much better, though more complicated too ;-)
>> However, I wouldn't do this based on protocols or routing HyperSCSI
>> ether packets or the need to use HyperSCSI kernel modules in
>> domains > 0 (Perhaps too complicated and only a special solution for this
>> problem).
>I still like my proposal ;-)

:)) Sorry for being so rude on it ;-)
..Besides my other points mentioned, I just want to avoid the
necessity to load a kernel module in domains > 0 in order to use
the /dev/sda device.
It should just be usable like a standard hardware device supported
by the kernel -- KISS-Principle, at least from the point of view of
domains > 0 or clients using domains >0.
(Yes, sometimes I am a very restrictive purist ;-).

>It's pretty straight forward to implement, is relatively clean,
>and will have good performance.

I would like to build up a "production strength" environment
with as high remote access disk performance (speed) as reasonably

But if I accept thinking about loading a kernel module in
domains > 0 in order to get HyperSCSI attached devices
to work somehow, then your proposal (VFR routing of ether packets
to and from domains > 0) is perhaps likely to result in better
performance than using enbd devices additionaly.
However, I somehow don't like the thought of 100+ domains
from e.g. 3 different physical servers to connect to the HyperSCSI
physical server directly.
Thinking about just 3 DOM0 HyperSCSI clients connecting
to the HyperSCSI-Server directly feels somehow more comfortable.
(e.g. a lot easier administration, less points of failiure.)
The 3 DOM0s in this example can then export the HyperSCSI
device(s) via whatever means to the domains > 0.

>However, if you're exporting a single disk from the HyperSCSI
>server its not much help.
>> The virtual block device driver mapps this to /dev/sda and forwards
>> the request to Xen (perhaps it also tags this request as a request
>> to a "special device" before forwarding the request to Xen).
>> Xen realizes that there is no physical device connected to /dev/sda
>> (or registered with Xen ? Maybe it can then also recognize that
>> the request was marked as targeting a "special device").
>> Because of that condition, it forwards this block device request
>> to DOM0 now in which a "request handler" kernel module will listen for
>> block device requests which may be forwarded to DOM0 from
>> Xen to be handled in DOM0 (It will need to register a callback
>> function with Xen in order to do so).
>I think your best solution is not to use Xen vbd's at all.  If
>you don't like NFS, how about having domains >0 using "enhanced
>network block devices" which talk to a simple server running in
>domain0. The storage for the nbd server can be files, partitions
>or logical volumes on /dev/sda.
>This should require writing no code, and will give pretty good
>performance. It gives good control over storage allocations etc.

Thanks a lot for pointing me to this solution !
I will look into it during the next days (especially performance ;-).

A propos:
Did you ever make benchmarks about the average or maximum
throughput of your VFR implementation in XEN ?
This would be interesting when routing enbd IP packets
from DOM0 to the other domains on the same
machine (in terms of a possible average/maximum reachable
Also, did you make some benchmarks about the amount
of performance degradation by using vbds/vds for disk access
compared with using the block device directly (test in DOM0)?
Could mounting /dev/sda via enbd be more performant or
at least nearly equally performant to using vds and vbds
because of the additional overhead of vd/vbd use... ??

>[It appears to work as a rootfs, but I haven't verified]

I'll try it....(initrd required, I think :-( ) ;-)

Best regards,


This SF.net email is sponsored by: SF.net Giveback Program.
SourceForge.net hosts over 70,000 Open Source Projects.
See the people who have HELPED US provide better services:
Click here: http://sourceforge.net/supporters.php
Xen-devel mailing list