A couple of people have asked me how they could contribute to Xen
development.
I've done a cut ahd paste of various project ideas from the TODO
list, along with a list of suggestions I wrote in the hope of
getting some of our local undergraduates interested in working on
Xen.
Volunteers for any of these projects are greatly appreciated!
Thanks,
Ian
Xen Domain Control tool
=======================
The "xenctl" tool used for controling domains is still rather clunky
and not very user friendly. In particular, it should have an option to
create and start a domain with all the necessary parameters set from a
named xml file. Update: the 'xenctl script' functionality combined
with the '-i' option to 'domain new' sort of does this.
The java xenctl tool is really just a frontend for a bunch of C tools
named xi_* that do the actual work of talking to Xen and setting stuff
up. Some local users prefer to drive the xi_ tools directly, typically
from simple shell scripts. These tools are even less user friendly
than xenctl but its arguably clearer what's going on.
There's also a nice web based interface for controlling domains that
uses apache/tomcat. Unfortunately, this has fallen out of sync with
respect to the underlying tools, so is currently not built by default
and needs fixing. It shouldn't be hard to bring it up to date.
Virtual Consoles for domains >0
===============================
Another area that needs further work is the interface between Xen and
domain0 user space where the various XenoServer control daemons run.
The current interface is somewhat ad-hoc, making use of various
/proc/xeno entries that take a random assortment of arguments. We
intend to reimplement this to provide a consistent means of feeding
back accounting and logging information to the control daemon, and
enabling control instructions to be sent the other way (e.g. domain 3:
reduce your memory footprint to 10000 pages. You have 1s to comply.)
We should also use the same interface to provide domains with a
read/write virtual console interface. The current implemenation is
output only, though domain0 can use the VGA console read/write.
Resource Scheduling and Accounting
==================================
Xen contains resource schedulers to share out the physical
resources of the hardware among the different virtual machines
running on Xen. Currently, our schedulers for the CPU, network
interface and disk are fairly rudimentary, and geared toward
providing a "fair share" rather than providing controls to
guarantee (or restrict) a guest OS to particular pre-agreed
quota.
There are plenty of algorithms in the literature for schedulers
which do this. The goal of this project will be to select
appropriate algorithms, implement them, and perhaps enhance them
for the virtual machine environment. Previous experience with C
would be useful for this project. You'll have plenty of good data
structures and algorithms stuff to write up in the dissertation.
Logging, Auditing and Denial of Service prevention
==================================================
Xen is a key component of the XenoServers project, which has the
grand vision of creating an "Open Infrastructure for Global
Distributed Computing". We envisage a world in which Xenoserver
execution platforms will be scattered across the globe and
available for any member of the public to submit code for
execution. The sponsor of the code will be billed for all the
resources used or reserved during the course of execution. This
will serve to encourage load balancing, limit congestion, and
hopefully even make the platform self-financing.
One big problem we're going to have is that some (many?) users of
the XenoServer infrastructure may use the system for nefarious
purposes, for launching denial of service attacks, spamming, or
as a platform for hacking other systems. We need a way of
efficiently (in space and time) logging all the network activity
occurring on Xen along with the 'sponsor' that caused it to
occur, keeping it safe in case forensic analysis is required. We
also need to be able to detect and filter or rate limit
operations which are likely to be dubious. For example, "limit
outgoing connections to SMTP ports to be one per user per 10
seconds".
We can also quench a large class of denial of service attack by
looking at the symmetry of the number of packets received on a
connection to the number sent: for a TCP connection you should
receive an ACK every other packet. If not, chances are you're
hosing the remote host, and hence Xen should throttle the
connection until the symmetry improves. A similar technique can
be used to throttle attempts to do port scanning from
XenoServers.
The goal of this project is to implement some of the detection
and throttling mechanisms described above, and maybe think of a
few more.
Xen suspend/resume for migration
================================
One cool feature we'd like to add to Xen would be the ability to
"suspend to file" an operating system image running over Xen, and
then "resume from file" on another machine running Xen, hence
moving an operating system and along with all the applications
running over it! Because of the virtualization Xen provides, this
sort of thing isn't impossible, but it will require someone with
a decent knowledge of how the x86 handles page tables and such
like.
The code that does the spend/resume will run in another operating
system image, inspecting the memory footprint of the operating
system to be suspended, and then writing it to disk, along with a
load of meta information. The hard part is that when the OS is
'resumed', the physical memory pages that it was using are
unlikely to be the same ones it's been allocated this time round,
so its going to be necessary to re-write the page tables before
resuming the OS, based on the meta information that's been
stored.
As an extension, you could modify the guest operating system to
make suspend resume more efficient, perhaps by evicting the
entire buffer cache, hence minimising the suspended state.
Restartable Device Drivers
==========================
Like most operating systems, the Xen Virtual Machine Monitor runs
its device drivers in the most privileged execution level (ring 0
in x86 terminology), giving them unrestricted access to memory,
I/O addresses, privileged instructions etc. Unfortunately, many
device drivers are not written by programmers who actually
understand the subtleties of writing kernel code. Forgetting to
turn interrupts on, or accessing an un-mapped page are common
mistakes. Unfortunately, these simple errors can result in the
OS/VMM crashing or just hanging.
System reliability could be significantly improved if device
drivers were moved out of 'ring 0', and executed in a somewhat
more restricted environment where bugs can do less harm. For
example, only giving device drivers I/O access to the particular
piece of hardware they're supposed to be controlling, limiting
what memory they can write to, ensuring that interrupts aren't
turned off for long etc. If a device driver then causes a page
fault, or violates any of the assertions, it can then just be
stopped, and potentially restarted, hopefully clearing the fault
without any users even noticing.
Of course, some faults (like DMA'ing a received network packet to
a bad address) can't be protected against, but hopefully that
kind of error isn't common. To do this project you should have a
decent knowledge of device drivers, and a bit about x86 memory
protection.
FreeBSD 4.8 port
================
We really need a FreeBSD port. The aborted NetBSD port isn't a
very good starting point as Xen has evolved quite a bit since
then. Here's a step by step guide to doing the port:
1. learn how to build and boot a standard i686 FreeBSD kernel
2. Copy the i386 directory to i386-xeno and hack the makefiles
appropriately.
3. hack the kernel so that it doesn't use the top 64MB of virtual
address space. Close down the CS/DS segment descriptors and boot
the kernel on a normal i386 system to ensure its not using that
part of the VA space.
4. identify the 32 bit protected mode kernel entry point. Graft
the xen 'minimal OS' code on the front to set up stacks, trap
handlers etc.
5. Hack the user space domain builder so that it configures the
initial page tables how FreeBSD wants them (this is easier than
doing it in the guestOS startup code).
6. hack the equivalent of 'printk' so that it does a
HYPERVISOR_CONSOLE_WRITE so that we get debugging output early,
before console/syslog support is up in the guest
7. Grep the tree looking for privileged instructions. Turn them
into break points so you can tell when you hit them.
8. Use the domain builder to boot the xen-i386 kernel. See where
the guestos explodes (turn on debugging in Xen to get some useful
info about why Xen felt it necessary to terminate the guestos).
9. When it explodes, it'll probably be faulting on a privileged
instruction, or page table write. Fix the code to use the proper
Xen call. Be conservative and flush the Xen request queue after
every operation -- we can optimise latter.
10. Repeat steps 8 and 9 until you get "unable to mount root file
system". Have a celebratory beer, you've very close.
11 Next, port the Xen network and block device drivers from
Linux. This should be pretty easy.
12. You should then have a working FreeBSD system. Give it a good
workout by running some demanding Apps to generate a validation
test suite.
13. Optimise the port by removing the flush after every pte_queue
call. E.g. in linux it's only necessary to do a flush when
dropping the lock on a "vma", as this is sufficient to prevent
read-after-write races.
14. I buy you dinner.
-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxxx
https://lists.sourceforge.net/lists/listinfo/xen-devel
|
Home