Hi Keir,
Attached please find my patch to allocate 1M pages at a time. With this patch
on 3.3.1 and unstable, I'm able to bring up the hypervisor with 512GB memory
and start a PV 64 guest (with my pud entry fix).
The only issue is performance related. As somewhat expected, after guest
shutdown it takes a while for pages to be freed and guest to disappear. Until
this time, system is fine, but xm will hang.
In my kdb debugger, I see cpu's in :
[0]ffff828c80110f52: page_scrub_softirq+22 test %al, %al
[1]ffff828c80110f52: page_scrub_softirq+22 test %al, %al
[2]ffff828c80110f52: page_scrub_softirq+22 test %al, %al
...
page_scrub_softirq():
/* free_heap_pages() does not parallelise well. Serialise this function. */
if ( !spin_trylock(&serialise_lock) )
I guess a future work item to look into parallelizing this some day....
Thanks a lot,
Mukesh
Keir Fraser wrote:
> On 18/02/2009 19:52, "Mukesh Rathor" <mukesh.rathor@xxxxxxxxxx> wrote:
>
>> This is a PV domain BTW, I guess xc_hvm_build.c wont' be involved. I'll
>> instrument libxc and debug more... meanwhile I thought I'd post this.
>
> Yes, my mistake, and this points us straight at the problem.
>
> You see the xc_domain_memory_populate_physmap() call at the end of
> libxc/xc_dom_x86.c:arch_setup_meminit()? That needs to be put in a for loop,
> allocating no more than, say, a million pages at a time.
>
> The patch should be very simple, so I'll leave the rest to you.
>
> -- Keir
>
Signed-off-by: Mukesh Rathor <mukesh.rathor@xxxxxxxxxx>
---
diff -r 746955812d23 tools/libxc/xc_dom_x86.c
--- a/tools/libxc/xc_dom_x86.c Wed Mar 11 17:47:56 2009 -0700
+++ b/tools/libxc/xc_dom_x86.c Wed Mar 18 14:31:07 2009 -0700
@@ -25,6 +25,7 @@
#include "xenctrl.h"
/* ------------------------------------------------------------------------ */
+#define MEM_CHUNKSZ 1024*1024
#define bits_to_mask(bits) (((xen_vaddr_t)1 << (bits))-1)
#define round_down(addr, mask) ((addr) & ~(mask))
@@ -694,7 +695,7 @@ int arch_setup_meminit(struct xc_dom_ima
int arch_setup_meminit(struct xc_dom_image *dom)
{
int rc;
- xen_pfn_t pfn;
+ xen_pfn_t pfn, pages, allocsz, *p2mp;
rc = x86_compat(dom->guest_xc, dom->guest_domid, dom->guest_type);
if ( rc )
@@ -712,10 +713,16 @@ int arch_setup_meminit(struct xc_dom_ima
for ( pfn = 0; pfn < dom->total_pages; pfn++ )
dom->p2m_host[pfn] = pfn;
- /* allocate guest memory */
- rc = xc_domain_memory_populate_physmap(dom->guest_xc, dom->guest_domid,
- dom->total_pages, 0, 0,
- dom->p2m_host);
+ /* allocate guest memory MEM_CHUNKSZ at a time */
+ for ( rc=0, p2mp=dom->p2m_host, pages=dom->total_pages; !rc && pages; )
+ {
+ allocsz = (pages <= MEM_CHUNKSZ) ? pages : MEM_CHUNKSZ;
+ rc = xc_domain_memory_populate_physmap(dom->guest_xc, dom->guest_domid,
+ allocsz, 0, 0, p2mp);
+ pages -= allocsz;
+ p2mp += allocsz;
+ }
+
return rc;
}
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel
|
Home