WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-devel

[Xen-devel] [PATCH - proposed] XI Shadow Page Table Mechanism

To: xen-devel@xxxxxxxxxxxxxxxxxxx
Subject: [Xen-devel] [PATCH - proposed] XI Shadow Page Table Mechanism
From: Ben Thomas <bthomas@xxxxxxxxxxxxxxx>
Date: Mon, 26 Jun 2006 17:47:38 -0400
Delivery-date: Mon, 26 Jun 2006 14:48:06 -0700
Envelope-to: www-data@xxxxxxxxxxxxxxxxxx
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
User-agent: Mozilla Thunderbird 1.0.8-1.1.fc4 (X11/20060501)
A post last week contained the design document for a shadow page
table mechanism. This post contains more information, and a pointer
to the code.

As a recap, the "XI Shadow Mechanism" is a design for shadow
page table code for fully virtualized HVM domains running on a 64-bit
Xen hypervisor.

This work was undertaken to address a number of goals.  These are
enumerated in the document posted last week and include:

- ability to run fully virtualized 32, 32PAE and 64 bit guest
  domains concurrently on a 64-bit hypervisor

- start support of live migration of fully virtualized domains

- provide good performance and robustness

- limit size and scope of change to Xen hypervisor

This design has been coded, and tested on a variety of operating
systems. I've included this weekend's test report.

Of particular interest from the following table are the Average Time
comparisions (smaller is better):

pfault   10.56 vs 2.43 (XI)
sync_all 39.72 vs 33.51 (XI)
invl_pg  22.75 vs 4.00 (XI)

The patch is too large for the mailing list, and may be obtained from

http://s3.amazonaws.com/VIDownloads/xi-20060626.patch

The patch applies against changeset 10352 (and possibly others)

We look forward to any comments and suggestions.

Thanks,
-b



XI Shadow Test Report

This report shows a comparsion of test runs done on xen-unstable changeset
10352 with and without the XI patch applied.  It also compares pfault,
sync_all, and invl_pg performance.

With the XI patch 32bit SMP (PAE) guests now boot and pass the regression
tests, closing Bug #661.  With the XI patch 64bit SMP guests are much more
stable and are no longer experiencing SEGVs and GPFs, closing Bug #678.

 ----------------------------------------------------------------------
| XEN      | Guest Kernel SMP kernels booted with 2 CPUs               |
| Changeset|-----------------------------------------------------------|
|          | 32bit Non-XI | 32bit XI     | 64bit Non-XI | 64bit XI     |
|          |--------------|--------------|--------------|--------------|
|          | Boot | Test  | Boot | Test  | Boot | Test  | Boot | Test  |
|----------|------|-------|------|-------|------|-------|------|-------|
| 10352    | Fail |       | Pass | 852/1 | Pass | 852/0 | Pass | 852/0 |
|          | (1)  |       |      | (2)   |      |(3,4,5)|      |(3,4)  |
 ----------------------------------------------------------------------

1. BUG 661: 32bit SMP guest crashes on boot:
   "(XEN) __hvm_bug at vmx.c:2289"
2. BUG 666: 32bit UP guest fail ltp gettimeofday02:
    "Time is going backwards"
3. BUG 663: 64bit SMP guest report these messages while running:
    "(XEN) spurious IRQ irq got=-1"
4. BUG 662: 64bit SMP guest report this message in ltp pth_str01/2/3:
    "(XEN) <vlapic_accept_irq>level trig mode repeatedly for vector 252"
5. BUG 678: 64bit 2CPU SMP guest failed several ltp tests with SEGVs or GPF
    "sshd[23654] general protection rip:2a9555d426 rsp:7fbfffd000 error 0"

The XI patch can be enabled or disabled at build time and includes some
performance statistics that can be examined whether or not it's enabled.
This allows us to compare the performance of XI vs the existing shadow code.
You access the statistics from the XEN console (^a^a^a). Typing a 'Y'
clears the counters and a 'y' displays the counters since the last clear.
Here is a comparsion of XI vs Non-XI shadow code done on xen-unstable
changeset 10352. The guest is a RHEL4U2-64bit 2 CPU SMP 256MB Guest running
a usex -b24 load for 5 minutes.  Shadow statistics cleared at start of run.

(XEN) Non-XI shadow performance statistics for the last 303 seconds
(XEN) Op type                                   # of Operations                 
                    Total       Avg          Tick Profile
(XEN)              <2usec    <3usec    <4usec    <8usec   <16usec   <32usec   
<64usec  >=64usec  ops/msec      usec     In Op    Locked     Total
(XEN) pfault      6750685   1128922    624988   3249437   3833201   2863229   
1224908    315992  19991362     10.56         0         0         0
(XEN)               33.7%      5.6%      3.1%     16.2%     19.1%     14.3%     
 6.1%      1.5%    211175
(XEN) sync_all       8176       380       333    159011    167619    305470    
814598    298411   1753998     39.72         0         0         0
(XEN)                0.4%      0.0%      0.0%      9.0%      9.5%     17.4%     
46.4%     17.0%     69670
(XEN) invl_pg         219        17       114     23333     40087     21904     
24622      4639    114935     22.75         0         0         0
(XEN)                0.1%      0.0%      0.0%     20.3%     34.8%     19.0%     
21.4%      4.0%      2615

(XEN) XI shadow performance statistics for the last 302 seconds
(XEN) Op type                                   # of Operations                 
                    Total       Avg          Tick Profile
(XEN)              <2usec    <3usec    <4usec    <8usec   <16usec   <32usec   
<64usec  >=64usec  ops/msec      usec     In Op    Locked     Total
(XEN) pfault     23078414   4694139    485499    613419    396088   1427265     
11976      8228  30715028      2.43         0         0         0
(XEN)               75.1%     15.2%      1.5%      1.9%      1.2%      4.6%     
 0.0%      0.0%     74690
(XEN) sync_all       1734        97       169    305433    484991    174300    
150199    130405   1247328     33.51         0         0         0
(XEN)                0.1%      0.0%      0.0%     24.4%     38.8%     13.9%     
12.0%     10.4%     41810
(XEN) invl_pg       10169    302977    125270     44927      5385     17603     
  776      1656    508763      4.00         0         0         0
(XEN)                1.9%     59.5%     24.6%      8.8%      1.0%      3.4%     
 0.1%      0.3%      2037


--
------------------------------------------------------------------------
Ben Thomas                                         Virtual Iron Software
bthomas@xxxxxxxxxxxxxxx                            Tower 1, Floor 2
978-849-1214                                       900 Chelmsford Street
                                                   Lowell, MA 01851

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel

<Prev in Thread] Current Thread [Next in Thread>
  • [Xen-devel] [PATCH - proposed] XI Shadow Page Table Mechanism, Ben Thomas <=