WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-devel

RE: [Xen-devel] [RFC] Transcendent Memory ("tmem"): a new approach to ph

To: "Xen-Devel (E-mail)" <xen-devel@xxxxxxxxxxxxxxxxxxx>
Subject: RE: [Xen-devel] [RFC] Transcendent Memory ("tmem"): a new approach to physical memory management
From: Dan Magenheimer <dan.magenheimer@xxxxxxxxxx>
Date: Fri, 9 Jan 2009 00:00:54 +0000 (GMT)
Delivery-date: Thu, 08 Jan 2009 16:01:33 -0800
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <96f92226-38e2-4a9e-94bf-0ddd3310491f@default>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
For expediency, I've posted the Linux 2.6.28 patchset for tmem,
implementing precache and preswap, here:

http://oss.oracle.com/pipermail/tmem-devel/2009-January/000002.html

This being the first time I've used that mailing-list-er
myself, I had thought that the patches attached would be
inlined, but I see there were not and it will be necessary
to click on a URL for each.  Apologies... if this is
incovenient, please let me know and I will repost (to
xen-devel?).

It also appears that responders to messages on tmem-devel
require list membership.  Apologies again... please respond
to xen-devel with comments for now.

Thanks,
Dan

> -----Original Message-----
> From: Dan Magenheimer 
> Sent: Thursday, January 08, 2009 10:27 AM
> To: Xen-Devel (E-mail)
> Subject: [Xen-devel] [RFC] Transcendent Memory ("tmem"): a 
> new approach
> to physical memory management
> 
> 
> At last year's Xen North America Summit in Boston, I gave a talk
> about memory overcommitment in Xen.  I showed that the basic
> mechanisms for moving memory between domains were already present
> in Xen and that, with a few scripts, it was possible to roughly
> load-balance memory between domains.  During this effort, I
> discovered that "ballooning" had a lot of weaknesses, even
> though it is the foundation for "time-sharing" physical
> memory in every major virtualization system existing today.
> These weaknesses have led in many cases to unacceptable performance
> issues when VMs are densely packed; as a result, memory is becoming
> the bottleneck in many deployments of virtualization.
> 
> Transcendent Memory -- or "tmem" for short -- is phase II of that
> work and it essentially augments ballooning and "fixes" many of
> its weaknesses.  It requires paravirtualization, but the changes
> (to Linux) are fairly small and minimally-invasive.  The changes
> to Xen are larger, but also fairly non-invasive.  (No shell scripts
> this time! :-)  The concept and code is modular and may easily
> port to Windows, as well as KVM.  It may even be useful in
> containers and in a native physical operating system. And,
> yes, it is machine-independent so should be easily portable
> to ia64 too!
> 
> Basically, instead of moving the ownership of all physical memory
> between one domain and another, tmem instead collects system-wide
> underutilized memory into a "pool" in the hypervisor and provides
> indirect access to that memory so that it can serve the needs
> of domains without necessarily being directly addressible by the
> domains it serves.  It is implemented with a small set of
> (hyper)calls that enable pages to be copied between a domain
> and Xen, controlled by a carefully-crafted set of semantics that
> make it easy in most cases for memory to be easily reclaimed
> by Xen as memory needs vary (as they often do -- rapidly and
> unpredictably).  As a result, physical memory is utilized more
> efficiently, reducing unnecessary paging and the likelihood
> of thrashing and thus increasing performance and/or allowing
> greater VM density.
> 
> If you are interested in this topic, please see:
> 
> http://oss.oracle.com/projects/tmem
> (note, site is sometimes slow)
> 
> for more information.  This site will be updated frequently,
> with patches, documentation, and FAQs.  The site also
> supports mailing lists, though I'd prefer to have all
> Xen-related discussions start on xen-devel.
> 
> Linux patches based on 2.6.18-xen, 2.6.27-xen, and 2.6.28
> are available.  The Xen patch is currently-based on 3.3.0+
> and I am in the process of updating it and cleaning it up, so
> will post it in the near future, but can provide it to anyone
> who is very interested in seeing/trying it now.  I could
> use some help on the "control plane" python software,
> in performance evaluation, and in "porting".
> 
> Comments and questions welcome.  I also plan to submit an
> abstract for the upcoming Xen summit and, if accepted, give
> a talk about tmem there.
> 
> Thanks,
> Dan
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-devel
> 
>

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel