Re: Cannot boot PVH dom0 with big initrd

[Jan Beulich <jbeulich@xxxxxxxx>]

On 13.02.2026 16:56, Roger Pau Monné wrote:
> On Fri, Feb 13, 2026 at 09:56:42AM +0100, Jan Beulich wrote:
>> On 13.02.2026 05:02, Marek Marczykowski-Górecki wrote:
>>> Hi,
>>>
>>> After fixing the xhci crash, I hit another issue - booting with 236MB
>>> initrd doesn't work, I get:
>>>
>>>     (XEN) [    3.151856] *** Building a PVH Dom0 ***
>>>     ...
>>>     (XEN) [    3.593940] Unable to allocate memory with order 0!
>>>     (XEN) [    3.597110] Failed to setup Dom0 physical memory map
>>>     (XEN) [    3.599884] 
>>>     (XEN) [    3.602482] ****************************************
>>>     (XEN) [    3.605272] Panic on CPU 0:
>>>     (XEN) [    3.607928] Could not construct d0
>>>     (XEN) [    3.610692] ****************************************
>>>     (XEN) [    3.613463] 
>>>     (XEN) [    3.616035] Reboot in five seconds...
>>>     (XEN) [    8.626565] Resetting with ACPI MEMORY or I/O RESET_REG.
>>>
>>> Full console log: 
>>> https://gist.github.com/marmarek/c9dbc87bf07b76f2899781755762f565
>>>
>>> If I skip initrd, then it boots just fine (but dom0 is not happy about
>>> that). 164MB initrd failed too, but 13MB started ok.
>>> Just in case, I tried skipping XHCI console, but it didn't change
>>> anything.
>>>
>>> Host has 16GB of memory, and there is no dom0_mem= parameter. Xen is
>>> started from GRUB, using MB2+EFI.
>>
>> Hmm, yes, there's an ordering issue: Of course we free initrd space (as used
>> for passing from the boot loader to Xen) only after copying to the designated
>> guest area. Yet dom0_compute_nr_pages(), intentionally, includes the space in
>> its calculation (adding initial_images_nrpages()'s return value). PV Dom0
>> isn't affected because to load huge initrd there, the kernel has to request
>> the initrd to not be mapped into the initial allocation.
> 
> Right, on PV dom0 we do not copy the image to a new set of pages, we
> simply assign the pages where the initrd resides to the domain.  We
> can't populate those pages in the p2m as-is, otherwise we would
> shatter super pages.
> 
> I think the fix below should do it, it's likely the best we can do.

That's at best a workaround imo. We definitely can do better, and the bigger
the initrd, the more important it may be that we actually do. One option may
be to similarly use the pages directly (i.e. by assigning rather than
copying), accepting that we may not be able to use large page mappings then
for the respective GFN range.

Jan

> Can you please give it a try Marek?
> 
> Thanks, Roger.
> ---
> diff --git a/xen/arch/x86/dom0_build.c b/xen/arch/x86/dom0_build.c
> index 0b467fd4a4fc..8e3cb5d0db76 100644
> --- a/xen/arch/x86/dom0_build.c
> +++ b/xen/arch/x86/dom0_build.c
> @@ -343,7 +343,7 @@ unsigned long __init dom0_compute_nr_pages(
>  
>      for_each_node_mask ( node, dom0_nodes )
>          avail += avail_domheap_pages_region(node, 0, 0) +
> -                 initial_images_nrpages(node);
> +                 is_pv_domain(d) ? initial_images_nrpages(node) : 0;
>  
>      /* Reserve memory for further dom0 vcpu-struct allocations... */
>      avail -= (d->max_vcpus - 1UL)
>