Re: Intermittent kernel WARN in mm.h get_page() when forcefully removing a xen_netfront interface from xenstore

[Jürgen Groß <jgross@xxxxxxxx>]

On 15.07.26 13:02, Pedro Falcato wrote:
> +Cc Willy, Vlastimil
>
> On Tue, Jul 14, 2026 at 11:49:12PM -0400, Aaron Rainbolt wrote:
> > A user in the Qubes OS issue tracker reported that they could cause
> > kernel WARNs in some of their AppVMs when forcibly terminating a NetVM
> > that those AppVMs were connected to. [1] Typically, a message such as
> > `vif vif-0: xenbus: device forcefully removed from xenstore` appears,
> > then a "WARNING" appears immediately thereafter indicating an issue in
> > `xennet_disconnect_backend`, with the line of code throwing the warning
> > being somewhere in `include/linux/mm.h`. Originally it was thought that
> > the warning was simply normal behavior when a device is forcefully
> > removed, but it seemed very weird that memory management code would be
> > in charge of triggering that warning. Furthermore, I cannot reproduce
> > the issue reliably; for some users, it happens immediately after
> > shutting down the NetVM, whereas for me, I had to use multiple tricks
> > including pausing and unpausing a VM to trigger it. After some
> > fiddling, I was able to reproduce the issue using the latest stable
> > 7.1.3 kernel:
> >
> > 1. On a Whonix-Workstation 18 AppVM (such as anon-whonix), build kernel
> >     7.1.3 using the steps from
> >     
> > https://kernel-team.pages.debian.net/kernel-handbook/ch-common-tasks.html#s-common-size,
> >     ensuring that debug info is enabled. Use Debian's kernel config from
> >     the linux-base-7.1.3+deb14-amd64_7.1.3-1_amd64.deb package as the
> >     starting kernel config.
> > 2. Install the new kernel into a cloned Whonix-Workstation 18 template.
> > 3. Install pvgrub in dom0: `sudo qubes-dom0-update --action=install
> >     grub2-xen-pvh`
> > 4. Create a new AppVM named `anon-whonix-kerntest` based on this
> >     template.
> > 5. Set the kernel for `anon-whonix-kerntest` to `pvgrub2-pvh`.
> > 6. Start `anon-whonix-kerntest`.
> > 7. Run `uname -r` in `anon-whonix-kerntest`, ensure that the kernel
> >     version displayed is 7.1.3.
> > 8. Open a root QTerminal window in `anon-whonix-kerntest` by running
> >     the following in dom0: `qvm-run -u root anon-whonix-kerntest
> >     qterminal`.
> > 9. In the root QTerminal window, run `sysctl kernel.warn_limit=5` to
> >     prevent an immediate kernel panic when the kernel warn occurs.
> > 10. Open Tor Browser in `anon-whonix-kerntest`, and attempt to navigate
> >      to a fairly resource-intensive web page like Reddit's front page.
> > 11. While the page is still loading, run `qvm-pause
> >      anon-whonix-kerntest` in dom0 to freeze the AppVM.
> > 12. In dom0, run `qvm-kill sys-whonix`.
> > 13. Wait a few seconds, then run `qvm-unpause anon-whonix-kerntest` to
> >      unfreeze the AppVM.
> > 14. In the root QTerminal window of `anon-whonix-kerntest`, run `dmesg
> >      -w` to watch for kernel messages.
> > 15. Wait a few seconds, then start `sys-whonix`. Qubes OS will reattach
> >      the network when this is done (there is the equivalent of `xl
> >      network-attach` done by the Qubes code automatically). The following
> >      messages should appear in dmesg:
> >
> >      [  426.609004] vif vif-0: xenbus: device forcefully removed from xenstore
> >      [  426.609242] ------------[ cut here ]------------
> >      [  426.609254] WARNING: ./include/linux/mm.h:2097 at 
> > xennet_disconnect_backend+0x1c7/0x520 [xen_netfront], CPU#0: xenwatch/64
> >      [  426.609269] Modules linked in: snd_seq_dummy(E) snd_hrtimer(E) 
> > snd_seq(E) snd_seq_device(E) snd_timer(E) snd(E) soundcore(E) cfg80211(E) 
> > nls_ascii(E) nls_cp437(E) vfat(E) fat(E) rfkill(E) xenfs(E) nft_reject_inet(E) 
> > nf_reject_ipv4(E) nf_reject_ipv6(E) nft_reject(E) nft_ct(E) nf_conntrack(E) 
> > nf_defrag_ipv6(E) nf_defrag_ipv4(E) nf_tables(E) binfmt_misc(E) 
> > intel_rapl_msr(E) intel_rapl_common(E) intel_uncore_frequency_common(E) 
> > intel_pmc_ssram_telemetry(E) intel_vsec(E) aesni_intel(E) gf128mul(E) 
> > xen_netfront(E) xen_privcmd(E) xen_gntdev(E) xen_gntalloc(E) xen_blkback(E) 
> > drm(E) xen_evtchn(E) jitterentropy_rng(E) libsha3(E) loop(E) configfs(E) 
> > efi_pstore(E) nfnetlink(E) ip_tables(E) x_tables(E) autofs4(E) ext4(E) crc16(E) 
> > mbcache(E) jbd2(E) dm_snapshot(E) dm_bufio(E) dm_mod(E) xen_blkfront(E)
> >      [  426.609334] CPU: 0 UID: 0 PID: 64 Comm: xenwatch Tainted: G            
> > E       7.1.3 #5 PREEMPT(lazy)
> >      [  426.609340] Tainted: [E]=UNSIGNED_MODULE
> >      [  426.609343] RIP: 0010:xennet_disconnect_backend+0x1c7/0x520 
> > [xen_netfront]
> >      [  426.609349] Code: 94 dd a0 10 00 00 48 8b 4a 08 48 89 ce 83 e6 01 48 83 ee 01 
> > 48 09 f1 48 21 ca 0f b6 4a 33 81 f9 f5 00 00 00 0f 85 ea fe ff ff <0f> 0b 49 81 
> > ff 00 01 00 00 0f 82 f8 fe ff ff 4c 89 fe 48 c7 c7 e0
> >      [  426.609352] RSP: 0018:ffffce3d810c3c98 EFLAGS: 00010246
> >      [  426.609356] RAX: 0000000000000010 RBX: 0000000000000000 RCX: 
> > 00000000000000f5
> >      [  426.609358] RDX: fffffb0d80ca9800 RSI: 0000000000000000 RDI: 
> > ffffffff88c8a8a0
> >      [  426.609360] RBP: ffff8dec51df0000 R08: ffff8debf4dd5a80 R09: 
> > ffff8debf4dd5a80
> >      [  426.609362] R10: 0000000000000020 R11: 00000000000000bf R12: 
> > 0000000000000000
> >      [  426.609364] R13: ffff8dec8ef30b80 R14: ffff8dec197fcac0 R15: 
> > 0000000000000000
> >      [  426.609366] FS:  0000000000000000(0000) GS:ffff8ded2d129000(0000) 
> > knlGS:0000000000000000
> >      [  426.609369] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> >      [  426.609370] CR2: 00005faa1a7d3737 CR3: 00000000b420a003 CR4: 
> > 0000000000770ef0
> >      [  426.609377] PKRU: 55555554
> >      [  426.609379] Call Trace:
> This is due (or looks like it...) to
> static netdev_tx_t xennet_start_xmit(struct sk_buff *skb, struct net_device 
> *dev)
> [...]
> page = virt_to_page(skb->data); /* skb->data is a slab kmallocation */
> /* ... */
>         info.page = page;
>         first_tx = xennet_make_first_txreq(&info, offset, len);
> /* ... */
> queue->grant_tx_page[i] = page;
>
> and then later
>
> static void xennet_disconnect_backend(struct netfront_info *info)
> /* ... */
> xennet_release_tx_bufs
> get_page(queue->grant_tx_page[i]);
> gnttab_end_foreign_access(queue->grant_tx_ref[i],
>                           queue->grant_tx_page[i]);
>
> which may trigger get_page() on a slab page:
> static inline void get_page(struct page *page)
> {
>         struct folio *folio = page_folio(page);
>         if (WARN_ON_ONCE(folio_test_slab(folio)))
>                 return;
>         if (WARN_ON_ONCE(folio_test_large_kmalloc(folio)))
>                 return;
>         folio_get(folio);
> }
>
> and there's your WARN. The code as-is is wrong. get_page() does not stabilize
> slab allocations, and the skb is freed right after, which may very well free
> the data. This _can_ make sense for other page frags, but I would simply defer
> freeing the skb until either the full skb is xmitted, or you successfully
> cancel the whole transfer (what I understand is the "foreign access" in this
> code). The skb itself should pin the pages.
Thanks for the analysis and hint how to resolve this.

This will require quite some reworks, but should be possible.

Thanks,


Juergen

Attachment: OpenPGP_0xB0DE9DD628BF132F.asc
Description: OpenPGP public key

Attachment: OpenPGP_signature.asc
Description: OpenPGP digital signature