Xen Summit 2025 - "x86 Nested Virtualization - Project Plan" design session notes

["Yann Sionneau" <yann.sionneau@xxxxxxxxxx>]

Hi all,

Here are my notes from this design session:
https://design-sessions.xenproject.org/uid/discussion/disc_ADNniRk2YcQ1wMOZeMXc/view

I apologize for it being partial and maybe containing errors, it was not
an easy task for a non-native speaker as well as someone who does not
know the topic of nested virt.

####################
Wednesday 17/09

Nested Virt It’s going to be a requirement for windows in a couple of years.
Let’s present the bigger picture plan, how to break it down and how
people can help
Also maybe prioritizing

*A break down in small pieces is shown on screen*

Feature prep :
Nested virt is a huge undertaking
Few bits are optional
In theory it’s just a couple of instructions
It’s easy to implement wrong and create security vuln
Idea is to try and have each small piece have testing (XTF?)

cpuid/msr it’s an old topic, we are struggling to finish this
we don’t have a good way of controlling the features in the hypervisor
There is a lot of code that assumes that all VMs are configured the same
way.

We need to control all nested features per domain

Lot of other work to do : hvm fixes.
Lot of work that is not actually nested virt related
Some is deleting code which is great.

Mostly it’s not integration with the emulator that causes most work
XenServer goal is to get VBS to work (Virtualization Based Security)
(Windows using HyperV underneath)
An issue is that Windows, if it sees that virt is available it could
turn anything on.
It makes assumptions that a group of features are available in one group.
To make windows happy it will demand quite a number of features available
making WSL2 working is smaller than making VBS working according to Andrew.
Viridian nested work has to be proven to work safely, this is a
performance improvement, it’s not a priority. However you really want it
because otherwise perf are bad.
=> Make existing code work towards our need
I’m not quite sure how much we can use existing code (Jan)
Some pieces will definitely need rewriting
NMI handling, breakpoint handling are broken
We have bugs to fix wrt to MSR: the check for the bitmap is in the
generic hook
We need to fix them (bugs) in order not to have security vuln in the
nested virt case

Maybe focusing on one vendor first (intel vs amd) to have it out more
quickly?
AMD has 50 controls, intel has ~500.
If we look at AMD first … it would help to have something out
90% of the work is common.
Teddy : Nested paging handling is not in the shown breakdown
How do we transition from the old way of nested virt to the new one?
Would it be ok to send a patch to ditch the old way?
Andrew : I don’t think we want to slowly move from the old to the new
Andrew : I think we should rewrite from scratch, and by someone actively
ignoring current code.
Andrew : we should remove code and put the new one in a single release
If we are building feature by feature it should be doable
Andrew : if we do it in 1 release we should put the removal of 1 feature
and the restoration in one patch series.
Running hypervisor that are not Xen , on top of Xen, should be part of
the testing
Today most likely only Xen on Xen works because their assumptions do align.
XAPI testing for instance uses a lot nested virt to just spin up lots of
Xen and do pool join etc

Conclusion: let’s do a first basic working solution running on 1st
generation hardware, from scratch.

####################

--


--
Yann Sionneau | Vates XCP-ng Developer

XCP-ng & Xen Orchestra - Vates solutions

web: https://vates.tech