[PATCH v4 32/42] xen/riscv: remap interrupts to new IMSIC VS-file

[Oleksii Kurochko <oleksii.kurochko@xxxxxxxxx>]

Implement the first steps of migrating a vCPU to a different guest
interrupt file: take a new one on the vCPU's new pCPU with
imsic_vsfile_acquire(), which brings it to a clean state, maps it into
the guest's G-stage in place of the old one and records it in the
vCPU's IMSIC state. Then, if any interrupts at an APLIC are forwarded by
MSIs to the old interrupt file, reconfigure the APLIC to send them to
the new interrupt file.

Synchronize with the APLIC to ensure that no straggler MSIs will arrive
at the old interrupt file by running imsic_aplic_sync() on the pCPU
which owns that file, through the new imsic_call_on_cpu(): unlike the
new file, the old one belongs to another pCPU, which an IPI has to be
sent to.

The move maps the new file into the guest's G-stage, which takes the
p2m lock, and sends IPIs to the pCPU owning the old file, both of which
need IRQs enabled, while imsic_migrate_vcpu() is called with them
disabled. So do it in the new imsic_vsfile_move() and have
imsic_migrate_vcpu() enable IRQs around it, only when there is
something to move, which takes them to be disabled on entry, as an
ASSERT() checks.

Only a h/w file of another pCPU is moved: imsic_vsfile_needs_move()
keeps imsic_vsfile_move() from being reached for any other, as an
ASSERT() there checks.

Keep the BUG_ON("unimplemented") placeholder at the end of
imsic_vsfile_move() to guard against silent incorrect behaviour or
unexpected panics in guest VMs until the move is fully implemented.

Signed-off-by: Oleksii Kurochko <oleksii.kurochko@xxxxxxxxx>
---
Changes in v4:
 - Fold in "xen/riscv: prepare new IMSIC VS-file": with
   imsic_vsfile_acquire() already introduced by "xen/riscv: introduce
   IMSIC h/w interrupt file attaching to vcpu", taking and zeroing out
   the new file is a single call.
 - Move imsic_vsfile_acquire(), imsic_update_state(), the stubs of
   imsic_map_guest_file() and vgein_release() and the IOMMU TODO out of
   this patch, as they come with the attach now.
 - Move the file in the new imsic_vsfile_move(), which
   imsic_migrate_vcpu() runs with IRQs enabled, as imsic_migrate_vcpu()
   is now called with them disabled, by the vCPU itself on its way back
   to the guest.
 - Introduce imsic_call_on_cpu() here rather than in "xen/riscv:
   introduce IMSIC h/w interrupt file attaching to vcpu", as the old
   file is the first one to belong to another pCPU.
 - Don't panic() for a vCPU without a h/w file:
   imsic_vsfile_needs_move() keeps such a vCPU from being moved at all,
   so just ASSERT() it.
 - Update the commit message accordingly.
---
Changes in v3:
 - Update the commit message.
 - Move imsic_aplic_sync() and imsic_local_is_pending() to "xen/riscv:
   implement APLIC-IMSIC-hart sync barrier for vCPU migration" and only
   call the helper here.
 - compare old_vsfile_cpu against CPU_NONE instead of NR_CPUS.
 - Call vaplic_reconfigure_target() instead of aplic_reconfigure_target().
 - Update the comment at the start of the migration: at that point the new
   IMSIC VS-file is only being prepared, the interrupt producers are moved
   to it later.
 - Factor taking a h/w guest interrupt file (assign, zero out, map into
   G-stage and record in the per-vCPU IMSIC state) out of
   imsic_migrate_vcpu() into imsic_vsfile_acquire(), introduced here
   instead of in "xen/riscv: introduce IMSIC h/w interrupt file attaching
   to vcpu".
 - Crash the domain instead of BUG_ON()ing when no h/w guest interrupt
   file is free, and report which file could not be mapped.
 - Introduce the vgein_release() stub here, together with its first
   user: release the h/w guest interrupt file taken by
   imsic_vsfile_acquire() if mapping it into the guest fails. Drop the
   stray "\n" from its BUG_ON().
 - Clarify the comment ahead of imsic_vsfile_acquire() on why setting
   hstatus.VGEIN is left to the caller.
 - Implement imsic_update_state() here instead of leaving it a stub until
   "xen/riscv: introduce IMSIC h/w interrupt file attaching to vcpu": all
   it needs, vsfile_lock and CPU_NONE, is available already. The s/w
   IMSIC VS-file is recorded with vsfile_cpu == CPU_NONE rather than
   NR_CPUS.
 - Make imsic_update_state() static and drop its declaration from
   asm/imsic.h: imsic_vsfile_acquire() is its only user.
---
Changes in v2:
 - New patch.
---
---
 xen/arch/riscv/imsic.c | 103 +++++++++++++++++++++++++++++++++++++++--
 1 file changed, 99 insertions(+), 4 deletions(-)

diff --git a/xen/arch/riscv/imsic.c b/xen/arch/riscv/imsic.c
index 48214ab0d363..690a83fd6866 100644
--- a/xen/arch/riscv/imsic.c
+++ b/xen/arch/riscv/imsic.c
@@ -565,7 +565,9 @@ int cf_check vcpu_imsic_init(struct vcpu *v)
 
 /*
  * Arguments of the imsic_vsfile_local_*() helpers, which work on a guest
- * interrupt file of the pCPU they run on.
+ * interrupt file of the pCPU they run on. Those run on another pCPU go
+ * through imsic_call_on_cpu(), which passes a single pointer, hence the
+ * arguments are bundled.
  */
 struct imsic_vsfile_data {
     /* Guest interrupt file to work on, i.e. the hstatus.VGEIN selecting it. */
@@ -605,6 +607,33 @@ static void imsic_vsfile_deselect(const struct 
imsic_vsfile_ctx *ctx)
     csr_write(CSR_VSISELECT, ctx->vsiselect);
 }
 
+/*
+ * Execute func() on the pCPU which owns the IMSIC interrupt file func() is
+ * going to work with.
+ *
+ * An IMSIC VS-file is reachable only through hstatus.VGEIN of the hart the
+ * file belongs to, and a guest interrupt file index is meaningless on any
+ * other hart, so such work always has to be done by that very hart.
+ *
+ * The local case runs with IRQs disabled to provide func() with the same
+ * environment it is given when it is called from the function call IPI
+ * handler.
+ */
+static void imsic_call_on_cpu(unsigned int cpu, void (*func)(void *),
+                              struct imsic_vsfile_data *data)
+{
+    if ( cpu == smp_processor_id() )
+    {
+        unsigned long flags;
+
+        local_irq_save(flags);
+        func(data);
+        local_irq_restore(flags);
+    }
+    else
+        on_selected_cpus(cpumask_of(cpu), func, data, 1);
+}
+
 /*
  * Ensure that all the MSIs the APLIC has already generated for the hart this
  * runs on have really reached the hart's IMSIC.
@@ -616,7 +645,7 @@ static void imsic_vsfile_deselect(const struct 
imsic_vsfile_ctx *ctx)
  * any other hart, it has to be executed by the pCPU owning the interrupt file
  * the MSIs were being sent to.
  */
-static void __maybe_unused imsic_aplic_sync(void *unused)
+static void cf_check imsic_aplic_sync(void *unused)
 {
     imsic_local_eix_update(imsic_sync_id(&imsic_cfg), 1, true, false);
 
@@ -1077,7 +1106,8 @@ void imsic_vsfile_attach(struct vcpu *v)
  * A vCPU without a h/w file has none to move.
  *
  * No lock is needed: vsfile_cpu of a running vCPU is changed only by the vCPU
- * itself, when it acquires a file in imsic_vsfile_attach().
+ * itself, when it acquires a file in imsic_vsfile_attach() or
+ * imsic_migrate_vcpu().
  *
  * TODO: a s/w file, once supported, has to be moved too.
  */
@@ -1090,6 +1120,63 @@ static bool imsic_vsfile_needs_move(const struct vcpu *v)
     return cpu != CPU_NONE && cpu != smp_processor_id();
 }
 
+/*
+ * Move the h/w guest interrupt file of @v to the pCPU @v runs on, following
+ * "Migrating a virtual hart to a different guest interrupt file" in the AIA
+ * spec. Only called by imsic_migrate_vcpu(), see the requirements there.
+ */
+static void imsic_vsfile_move(struct vcpu *v)
+{
+    struct imsic_vsfile_data vsfile_data = {
+        .nr_eix = imsic_cfg.nr_eix,
+    };
+    struct vimsic_state *imsic_state = v->arch.vimsic_state;
+    unsigned long flags;
+    unsigned int old_vsfile_id;
+    unsigned int old_vsfile_cpu;
+
+    ASSERT(local_irq_is_enabled());
+    ASSERT(imsic_vsfile_needs_move(v));
+
+    read_lock_irqsave(&imsic_state->vsfile_lock, flags);
+    old_vsfile_id = imsic_state->guest_file_id;
+    old_vsfile_cpu = imsic_state->vsfile_cpu;
+    read_unlock_irqrestore(&imsic_state->vsfile_lock, flags);
+
+    /*
+     * At this point, all interrupt producers are still using the old IMSIC
+     * VS-file. Allocate and clear the new one before redirecting anything
+     * to it.
+     */
+
+    /* Zero-out, map and start to use the new IMSIC VS-file */
+    vsfile_data.hgei = imsic_vsfile_acquire(v);
+    if ( !vsfile_data.hgei )
+        return;
+
+    /*
+     * If any interrupts at an APLIC are forwarded by MSIs to the old interrupt
+     * file, reconfigure the APLIC to send them to the new interrupt file.
+     */
+    vaplic_reconfigure_target(v);
+
+    /*
+     * Synchronizing interactions between a hart and the APLIC.
+     *
+     * The MSIs to be flushed are the ones still on their way to the old
+     * interrupt file, so the barrier has to be done by the pCPU which owns
+     * that file.
+     */
+    imsic_call_on_cpu(old_vsfile_cpu, imsic_aplic_sync, NULL);
+
+    /*
+     * At this point, all interrupt producers have been moved
+     * to the new IMSIC VS-file.
+     */
+
+    BUG_ON("unimplemented");
+}
+
 /*
  * Make the h/w guest interrupt file of @v belong to the pCPU @v runs on.
  *
@@ -1101,9 +1188,17 @@ static bool imsic_vsfile_needs_move(const struct vcpu *v)
 void imsic_migrate_vcpu(struct vcpu *v)
 {
     ASSERT(v == current);
+    ASSERT(!local_irq_is_enabled());
 
     if ( !imsic_vsfile_needs_move(v) )
         return;
 
-    BUG_ON("unimplemented");
+    /*
+     * The move maps the new file into the guest's G-stage, which takes the
+     * p2m lock, and sends IPIs to the pCPU owning the old file: both need
+     * IRQs enabled.
+     */
+    local_irq_enable();
+    imsic_vsfile_move(v);
+    local_irq_disable();
 }
-- 
2.56.1