RE: [Xen-devel] [RFC][PATCH]Large Page Support for HAP

["Byrne, John (HP Labs)" <john.l.byrne@xxxxxx>]

Wei,

 

Sorry for being sluggish getting back to you, but my code was not working and I lost a week due to networking issues. (I probably could have debugged my code faster if I'd read your changes more carefully.) I have nothing so grand as a design; it is a hack to test 2M and 1G super-page performance on a random page-fault/TLB-miss benchmark. What I was hoping for was to have your code transparently support 1G pages on the assumption that their performance would be far better than 2M pages in this extreme case. Unfortunately for me, on the B1 rev CPU I have, I cannot see any difference between 2M and 1G pages. I saw something in one document about page splintering when the guest uses smaller pages than the NPT. Is this the issue? Do NPT super-pages not make any performance difference if they are larger than the guest pages? 

 

Thanks,

 

John Byrne

---

From: Huang2, Wei [mailto:Wei.Huang2@xxxxxxx]
Sent: Friday, November 16, 2007 9:54 AM
To: Byrne, John (HP Labs); xen-devel@xxxxxxxxxxxxxxxxxxx
Cc: Tim Deegan
Subject: RE: [Xen-devel] [RFC][PATCH]Large Page Support for HAP

John,

 

If you have a better design, share with us and I will be happy to work with you. :-) I agree that xc_hvm_build.c does not have to be modified, if memory.c is smart enough to scan all page_array information. But one concern is that sometimes Xen tools really want to create mapping at 4KB boundary instead of using large page. That requires extra information passed from tools (e.g., xc_hvm_build.c) to memory.c

 

-Wei

---

From: Byrne, John (HP Labs) [mailto:john.l.byrne@xxxxxx]
Sent: Friday, November 16, 2007 11:41 AM
To: Huang2, Wei; xen-devel@xxxxxxxxxxxxxxxxxxx
Cc: Tim Deegan
Subject: RE: [Xen-devel] [RFC][PATCH]Large Page Support for HAP

Wei,

 

I have been hacking at this, too,  since I am interested in trying 1GB pages to see what they can do. After I dug myself into a hole, I restarted from the beginning and am trying a different approach than modifying xc_hvm_build.c: modify populate_physmap() to opportunistically allocate large pages, if possible. I just thought I'd mention it.

 

John Byrne

 

---

From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx [mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of Huang2, Wei
Sent: Thursday, November 15, 2007 8:26 AM
To: xen-devel@xxxxxxxxxxxxxxxxxxx
Cc: Tim Deegan
Subject: [Xen-devel] [RFC][PATCH]Large Page Support for HAP

I implemented a preliminary version of HAP large page support. My testings showed that 32bit PAE and 64bit worked well. Also I saw decent performance improvement for certain benchmarks.

 

So before I go too far, I send this patch to community for reviews/comments. This patch goes with xen-unstable changeset 16281. I will redo it after collecting all ideas.

 

Thanks,

 

-Wei

 

============

DESIGN IDEAS:

1. Large page requests

- xc_hvm_build.c requests large page (2MB for now) while starting guests

- memory.c handles large page requests. If it can not handle it, falls back to 4KB pages.

 

2. P2M table

- P2M table takes page size order as a parameter; It builds P2M table (setting PSE bit, etc.) according to page size.

- Other related functions (such as p2m_audit()) handles the table based on page size too.

- Page split/merge

** Large page will be split into 4KB page in P2M table if needed. For instance, if set_p2m_entry() handles 4KB page but finds PSE/PRESENT bits are on, it will further split large page to 4KB pages.

** There is NO merge from 4KB pages to large page. Since large page is only used at the very beginning, guest_physmap_add(), this is OK for now.

 

3. HAP

- To access the PSE bit, L2 pages of P2M table is installed in linear mapping on SH_LINEAR_PT_VIRT_START. We borrow this address space since it was not used.

 

4. gfn_to_mfn translation (P2M)

- gfn_to_mfn_foreign() traverses P2M table and handles address translation correctly based on PSE bit.

- gfn_to_mfn_current() accesses SH_LINEAR_PT_VIRT_START to check PSE bit. If is on, we handle translation using large page. Otherwise, it falls back to normal RO_MPT_VIRT_START address space to access P2M L1 pages.

 

5. M2P translation

- Same as before, M2P translation still happens on 4KB level.

 

AREAS NEEDS COMMENTS:

1. Large page for 32bit mode

- 32bit use 4MB for large page. This is very annoying for xc_hvm_build.c. I don't want to create another 4MB page_array for it.

- Because of this, this area has not been tested very well. I expect changes soon.

 

2. Shadow paging

- This implementation will affect shadow mode, especially at xc_hvm_build.c and memory.c.

- Where and how to avoid affecting shadow?

 

3. Turn it on/off

- Do we want to turn this feature on/off through option (kernel option or anything else)?

 

4. Other missing areas?

===========

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel