[PATCH v5 0/6] unmap_page_range optimisation

[Kevin Lampis <kevin.lampis@xxxxxxxxxx>]

https://lore.kernel.org/xen-devel/20260727150615.1373200-1-kevin.lampis@xxxxxxxxxx/

Previous discussion:
RFC: unmap_page_range optimisation (avoiding emulation faults during VM 
migration)
https://lore.kernel.org/xen-devel/16133EFF-88FF-467F-B78F-E96EB148C3A5@xxxxxxxxxx/

This series adds a new MMU_PT_UPDATE_SWAP sub-op to the mmu_update hypercall
which Linux can use from a new pte_get_and_clear pv-op to significantly improve
the performance of unmap_page_range.

A microbenchmark[1] which allocates a large number of pages and then clears
them shows a performance increase from 1100ms to 640ms using the
new pte_get_and_clear pv-op.

Further profiling the microbenchmark with bpftrace[2] shows that Linux calls
unmap_page_range() 23 times with an average execution time of 48ms
dropping to 28ms when using the new operation.

Changes in v5:
- New Xen patch x86: rearrange how mod_l*_entry() functions call UPDATE_ENTRY
- New XTF patch (XTF) Add test mmu-pt-update-swap
- Review comments

Changes in v4:
- Review comments

Changes in v3:
- Extend swap to work with l2/l3/l4 PTEs and any value not just 0
- Other review comments

Changes in v2:
- Add a sub-op to mmu_update instead of new hypercall
- Add Linux patch
- Other review comments

[1] microbenchmark
#include <err.h>
#include <sys/mman.h>
#include <time.h>
#include <stdint.h>
#include <stdio.h>
#include <unistd.h>

static uint64_t nsec(void)
{
    struct timespec ts;
    clock_gettime(CLOCK_MONOTONIC, &ts);
    return (uint64_t)ts.tv_sec * 1e9 + ts.tv_nsec;
}

int main(int argc, char **argv)
{
    const size_t len = 1024UL * 1024 * 1024 * 4;
    const long pagesz = sysconf(_SC_PAGESIZE);

    char *p = mmap(NULL, len,
                   PROT_READ | PROT_WRITE,
                   MAP_PRIVATE | MAP_ANONYMOUS,
                   -1, 0);
    if ( p == MAP_FAILED )
        err(1, "mmap");

    /* Fault every page in */
    for (size_t i = 0; i < len; i += pagesz)
        p[i] = 1;

    uint64_t start = nsec();

    if ( madvise(p, len, MADV_DONTNEED) )
        err(1, "madvise");

    uint64_t end = nsec();

    printf("MADV_DONTNEED on %zu MB took %.3f ms\n",
           len / 1024 / 1024,
           (end - start) / 1e6);

    munmap(p, len);
    return 0;
}

[2] bpftrace
bpftrace -e '
kprobe:unmap_page_range
/comm == "a.out"/
{
    @start[tid] = nsecs;
}

kretprobe:unmap_page_range
/@start[tid] && comm == "a.out"/
{
    $delta = nsecs - @start[tid];
    @count = count();
    @total = sum($delta);
    delete(@start[tid]);
}

interval:s:10
{
    printf("avg call time = %d ns (%d calls)\n",
           (@total / @count), (uint64)@count);
    exit();
}'


Kevin Lampis (6):
  x86: extend update_intpte() to support atomic get-and-update
  x86: rearrange how mod_l*_entry() functions call UPDATE_ENTRY
  x86: extend mod_l*_entry() to optionally return the old PTE value
  x86: extend do_mmu_update() to support returning the old PTE value
  (Linux) xen: Add new Xen pv-op pte_get_and_clear.
  (XTF) Add test mmu-pt-update-swap

 xen/arch/x86/mm.c             | 181 +++++++++++++++++++++++++---------
 xen/arch/x86/pv/mm.h          |  14 ++-
 xen/common/kernel.c           |   1 +
 xen/include/public/features.h |   3 +
 xen/include/public/xen.h      |   1 +
 5 files changed, 152 insertions(+), 48 deletions(-)

-- 
2.52.0