[Xen-devel] Xen 3.0 Status update

["Ian Pratt" <m+Ian.Pratt@xxxxxxxxxxxx>]

At OLS we had a couple of "Xen Mini Summit" sessions. Although there
weren't any formal minutes, here's a rough summary of what we
discussed/concluded.

Status as of 24 July 2005
=========================

Summary: We have a couple of annoying bugs, but things are coming
together nicely.

x86_32p (PAE 16GB) and x86_64 ports are close to being feature complete
with x86_32. We should be able to ship 3.0.0 with the full feature set
for these ports. [IA64 and Power are not part of the 3.0.0 release, but
have actually made good progress anyhow].

We are going to need to support guest kernels that conform to the Xen
3.0 API for quite a few years to come, hence its important that we
ensure the API is extensible and as easy to make backward compatible as
possible. This has led to us wanting to work hard to push through a
couple of API changes that will make life easier in future:

 * New Time API. This is required for systems with unsyncronized TSCs
like the IBM Summit systems, and is also good for variable speed CPUs
(laptops)

 * Replacing control messages with XenBus/XenStore. Doing backwards
support of the old control message API would be a *huge* pain, so we're
really keen to get this incorporated before 3.0.0.

The new time API has been checked-in by Keir, and seems to be working
fine for most people, but there is at least one bug report of time
running fast. Please test.

Rolling-out XenBus/XenStore is a big project, but is making good
progress. A xen-tools@xxxxxxxxxxxxxxxxxxx mailing list has been created
to co-ordinate this work, and there's now a focussed effort to complete
it ASAP. 

The first phase of testing the 3.0 release can begin before the switch
to XenStore is complete -- there are plenty of platform related issues
that can be debugged completely independently. The plan is to do weekly
3.0-testing releases until we feel that we're on top of the  bugs being
found by the development community and would benefit from rolling a
3.0.0 release to get wider exposure. 

A very nice regression test infrastructure has been developed that
should be ready to go live in the next week. We also have a 'TestCD'
which can be used for automated testing. The aim is to get it run on as
a wide a variety of machines as possible. The results from both of these
test tools will appear in a results matrix on the web. The regression
test infrastructure is able to run sophisticated tests requiring
co-ordination between multiple virtual machines (e.g. SpecWEB etc). The
framework is easy to extend and add other tests (such as the ones
developed by Paul) and  it should be possible to develop it into a
comprehensive suite. It'll also be possible for others to run the
nightly tests on 'interesting machines' (e.g. wide SMP's) and have the
results automatically added to the web matrix.

The plan is to roll the first 3.0-testing release as soon as there are
no 'show stopper' bugs in the unstable tree. Unfortunately, the current
domU networking bug that a number of people have reported probably falls
into this category. Hopefully we can get a testing release out early
next week. More help to fix bugs (or isolate the changeset that
introduces them) would be *greatly* appreciated.

Although not strictly part of the 3.0 release, one of the most important
things we need to do is to get the arch-xen patch prepared into a form
that can be submitted upstream to Andrew/Linus. A couple of great
volunteers stepped forward, and we need to make this an absoloute
priority and help them as much as we can.

Looking at the various sections of the Xen code base, the following
paragraphs summarize the main issues:

Tools
=====

We need to complete the XenBus/XenStore switchover. Block devices are
basically done, but there's still work to do with net, console, balloon,
hotplug, shutdown etc.

There are a bunch of small outstanding tools issues we need to address:
 * sanitize all the xm commnds to give them consistent naming and
parameters
 * test error paths
 * split console from xend and replace control messages with XenBus (1st
part complete)
 * fix output of 'xm info'
 * (authentication of relocation connection)
 * (option to use xm-xend connection over SSL/TCP rather than just unix
domain socket)

Xen/Linux arch independent
==========================

The new time API needs more testing.

SEDF scheduler needs more testing.

Save/restore code needs to save state for multiple VCPUs rather than
just VCPU0.

Check-in AQ's patch to allow the number of CPUs and min memory used by
dom0 to be set from xend-config.sxp 

bug: xend start script looses networking on some machines. More testing
required for workaround suggested in bugzilla.

x86_32 Xen/Linux
================

needs: modify kmap to use update_va_mapping is an important optimization
for domains with more than 890MB (CONFIG_HIGHMEM4G).

bug: nasty networking issue reported in domU SLES9 guests.

x86_32p (PAE for >4GB) Xen/Linux
================================

[ compile with XEN_TARGET_X86_PAE=y ]

Nightly snapshot x86_32p install tarballs are now available from the
downloads page.  

Seems stable running dom0 and domU's, though not widely tested.
Needs particular testing on systems with >4GB RAM. Machines with dumb
SATA controllers with >4GB may be a particular problem.

needs:
 * 3-level shadow mode pagetable support to allow live relocation
 * save/restore support for 3-level pagetables
 * testing of NX/XD


x86_64 Xen/Linux
================

Seems stable running dom0 and domU's. Passes LTP plus other tests.
Needs particular testing on systems with >4GB RAM, particularly those
with dumb SATA controllers.

needs: 
 * SMP guest support (patch in progress)
 * writable pagetable support for fast fork/exit (patch in progress) 
 * save/restore support for 4 level pagetables
 * more testing of NX/XD


Roadmap
=======

Once the testing tree forks, the unstable tree will remain closed until
we have a stable 3.0 release shipping. 

The main development items slated for 3.1 are:
 * Pacifica / VT-x abstraction layer, plus improved IO emulation
 * Finish phase 3 of the tools project (split xend into lots of small
tools co-ordinated via XenStore)
 * Performance tuning and optimization -- less reliance on manual
configuration
 * Support for Infiniband/Smart NICs (direct guest IO access)


Ian



_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel