Re: Cannot boot PVH dom0 with big initrd

[Jan Beulich <jbeulich@xxxxxxxx>]

On 16.02.2026 09:40, Roger Pau Monné wrote:
> On Mon, Feb 16, 2026 at 09:11:29AM +0100, Jan Beulich wrote:
>> On 13.02.2026 16:56, Roger Pau Monné wrote:
>>> On Fri, Feb 13, 2026 at 09:56:42AM +0100, Jan Beulich wrote:
>>>> On 13.02.2026 05:02, Marek Marczykowski-Górecki wrote:
>>>>> Hi,
>>>>>
>>>>> After fixing the xhci crash, I hit another issue - booting with 236MB
>>>>> initrd doesn't work, I get:
>>>>>
>>>>>     (XEN) [    3.151856] *** Building a PVH Dom0 ***
>>>>>     ...
>>>>>     (XEN) [    3.593940] Unable to allocate memory with order 0!
>>>>>     (XEN) [    3.597110] Failed to setup Dom0 physical memory map
>>>>>     (XEN) [    3.599884] 
>>>>>     (XEN) [    3.602482] ****************************************
>>>>>     (XEN) [    3.605272] Panic on CPU 0:
>>>>>     (XEN) [    3.607928] Could not construct d0
>>>>>     (XEN) [    3.610692] ****************************************
>>>>>     (XEN) [    3.613463] 
>>>>>     (XEN) [    3.616035] Reboot in five seconds...
>>>>>     (XEN) [    8.626565] Resetting with ACPI MEMORY or I/O RESET_REG.
>>>>>
>>>>> Full console log: 
>>>>> https://gist.github.com/marmarek/c9dbc87bf07b76f2899781755762f565
>>>>>
>>>>> If I skip initrd, then it boots just fine (but dom0 is not happy about
>>>>> that). 164MB initrd failed too, but 13MB started ok.
>>>>> Just in case, I tried skipping XHCI console, but it didn't change
>>>>> anything.
>>>>>
>>>>> Host has 16GB of memory, and there is no dom0_mem= parameter. Xen is
>>>>> started from GRUB, using MB2+EFI.
>>>>
>>>> Hmm, yes, there's an ordering issue: Of course we free initrd space (as 
>>>> used
>>>> for passing from the boot loader to Xen) only after copying to the 
>>>> designated
>>>> guest area. Yet dom0_compute_nr_pages(), intentionally, includes the space 
>>>> in
>>>> its calculation (adding initial_images_nrpages()'s return value). PV Dom0
>>>> isn't affected because to load huge initrd there, the kernel has to request
>>>> the initrd to not be mapped into the initial allocation.
>>>
>>> Right, on PV dom0 we do not copy the image to a new set of pages, we
>>> simply assign the pages where the initrd resides to the domain.  We
>>> can't populate those pages in the p2m as-is, otherwise we would
>>> shatter super pages.
>>>
>>> I think the fix below should do it, it's likely the best we can do.
>>
>> That's at best a workaround imo. We definitely can do better, and the bigger
>> the initrd, the more important it may be that we actually do.
> 
> See the second patch I gave to Marek.  That one is slightly better by
> accounting for the initial images space as part of the reserved space.

Will check; didn't make it there, yet.

>> One option may
>> be to similarly use the pages directly (i.e. by assigning rather than
>> copying), accepting that we may not be able to use large page mappings then
>> for the respective GFN range.
> 
> Hm, there's always going to be a trade-off.  I think I would prefer
> having 1G pages in the p2m, rather than a bit more memory due to
> direct adding the initrd into the p2m.

"A bit more" may not get it, when considering e.g. a 2Gb initrd on a 4Gb
system. And of course "use pages directly" is only the simplest of the
possible approaches. We could "shift" the initrd some, or we could make
sure to allocate a 2Mb or 1Gb aligned region to hold it right from the
beginning.

Jan