Re: [Xen-devel] [RFC] Lightweight socket communication between domainsin a single machine

["kangho kim" <sunnykuz@xxxxxxxxx>]

Hi, there.
 

2007/5/8, Mark Williamson <mark.williamson@xxxxxxxxxxxx>:

Hi there,

> I attached a PPT file that has some note on the implementation, comparing
> to XenSocket.
> In each slide of the PPT, I wrote a few lines of slide note. Please read
> the note for more details.

So, to summarise, the main points of your approach are:

1) shared memory data transfers for streams of data - effectively a
Xen "device" channel per stream
2) use existing TCP syscall interface and protocol code to handle connection
setup and teardown (+ calls into additional setup / teardown code for Xway
itself, of course)

3) allow TCP-oriented applications to leverage Xway without any modification
whatsoever, using the same API as usual

It looks like a nice side effect of your strategy of leveraging TCP is that
you could perhaps modify your "Xway switch" to plumb the data stream either
through the TCP stack, or the Xway shared memory at will - allowing you to
fall back to TCP when migrating off a host, and allowing you to transparently
switch to Xway when migrating to be colocated with the server you're talking
to.

 

I agree with you that fortunately, it's easy for Xway to adjust the migration though we didn't consider the migration when designing it. The key point that Xway need to deal with for the migration is how Xway in each domain in a communication channel become aware that  your domain or the peer domain is migrated. Once Xway detects the migration, the rest is simple as you commented.

 

You seem to get some impressive throughput numbers when working on shared
memory ;-)  Do you think there are any further optimisations that you could
do here?

 

In the current design & implementation, Xway sends user data to the peer whenever user calls send request. The naive send machanism can be improved, so that we could expect higher bandwidth than the current. We have no idea above this improvement by now.

I'm curious as to how you handle addressing...  Is an IP-address->local domain
id lookup process happening somewhere?  How does this work?  Can any TCP
connection take advantage of Xway, or is some advance configuration required?

 

Xway maintains only IP addresses of domains that are in the same host, but does not IP-address->local domain id mapping. To know how Xway handles addressing, you need to know what is going on when a connection is established. I'll show you the detail steps concerning the connection establishment.

 

Suppose domains, D1 and D2 are in the same host, and socket application A1 in D1 attempts to connect socket appliation A2 in D2. Xway executes following tasks:

1) creates TCP session, C1, as if there were no Xway.

2) detects that the peer in the connection is in the same host by referencing IP address table in the current implementation.

3) creates another TCP session, C2,  between Xway in D1 and the helper daemon living in D2. C2 is supposed to bypass Xway module.

4) sends to D2 through C2, domain id of D1, grant references of shared memory and port number of event channel that D1 created.

4) receives from D2 through C2, domain id of D2, grant references of shared memory that D2 ceated.

5) creates Xway session which will be bound to C1, with the initial setup information.

6) destroys C2.

 

As you see in the steps above, Xway in D1 sends its domain id to D2, and become to know domain id of peer by querying the helper daemon in D2. This is IP-address->domain id lookup process that you want to know I guess.

> Since I don't know much about XenSocket I comapred only a few features that
> I'm interested in.

Perhaps the XenSocket folks can comment more on this.  It would be good to
identify how much commonality in code / purpose there are between the two
projects.

> I know that this summary slide is not enough to understand the whole things
> of Xway.
> However, let's start discussion on Xway as well as XenSocket.

--
Dave: Just a question. What use is a unicyle with no seat?  And no pedals!
Mark: To answer a question with a question: What use is a skateboard?
Dave: Skateboards have wheels.
Mark: My wheel has a wheel!

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel