WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-changelog

[Xen-changelog] Grant tables: substantially more robust.

To: xen-changelog@xxxxxxxxxxxxxxxxxxx
Subject: [Xen-changelog] Grant tables: substantially more robust.
From: BitKeeper Bot <riel@xxxxxxxxxxx>
Date: Mon, 04 Apr 2005 20:22:17 +0000
Delivery-date: Tue, 05 Apr 2005 08:03:19 +0000
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
List-help: <mailto:xen-changelog-request@lists.xensource.com?subject=help>
List-id: BK change log <xen-changelog.lists.xensource.com>
List-post: <mailto:xen-changelog@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-changelog>, <mailto:xen-changelog-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-changelog>, <mailto:xen-changelog-request@lists.xensource.com?subject=unsubscribe>
Reply-to: Xen Development List <xen-devel@xxxxxxxxxxxxxxxxxxxxx>
Sender: xen-changelog-bounces@xxxxxxxxxxxxxxxxxxx
ChangeSet 1.1443.1.1, 2005/04/04 21:22:17+01:00, cwc22@xxxxxxxxxxxxxxxxxxxxxx

        Grant tables: substantially more robust.
        Block front and back drivers: support for using grant tables for 
interdomain communication.



 docs/misc/grant-tables.txt                                    |  325 +++++
 linux-2.4.29-xen-sparse/arch/xen/config.in                    |    1 
 linux-2.4.29-xen-sparse/arch/xen/defconfig-xen0               |    1 
 linux-2.4.29-xen-sparse/arch/xen/defconfig-xenU               |    1 
 linux-2.4.29-xen-sparse/arch/xen/drivers/blkif/frontend/vbd.c |    5 
 linux-2.4.29-xen-sparse/include/asm-xen/fixmap.h              |    4 
 linux-2.6.11-xen-sparse/arch/xen/Kconfig                      |   10 
 linux-2.6.11-xen-sparse/arch/xen/kernel/gnttab.c              |  121 +-
 linux-2.6.11-xen-sparse/drivers/xen/blkback/blkback.c         |  119 ++
 linux-2.6.11-xen-sparse/drivers/xen/blkfront/blkfront.c       |  122 +-
 linux-2.6.11-xen-sparse/drivers/xen/blkfront/block.h          |    4 
 linux-2.6.11-xen-sparse/drivers/xen/blkfront/vbd.c            |    5 
 linux-2.6.11-xen-sparse/include/asm-xen/asm-i386/fixmap.h     |    4 
 linux-2.6.11-xen-sparse/include/asm-xen/gnttab.h              |   35 
 xen/arch/x86/mm.c                                             |    8 
 xen/common/grant_table.c                                      |  581 ++++++----
 xen/include/public/grant_table.h                              |    8 
 xen/include/public/io/blkif.h                                 |   12 
 xen/include/xen/grant_table.h                                 |    7 
 19 files changed, 1116 insertions(+), 257 deletions(-)


diff -Nru a/docs/misc/grant-tables.txt b/docs/misc/grant-tables.txt
--- /dev/null   Wed Dec 31 16:00:00 196900
+++ b/docs/misc/grant-tables.txt        2005-04-05 04:03:22 -04:00
@@ -0,0 +1,325 @@
+********************************************************************************
+ A Rough Introduction to Using Grant Tables
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+                                              Christopher Clark, March, 2005.
+
+Grant tables are a mechanism for sharing and transferring frames between
+domains, without requiring the participating domains to be privileged.
+
+The first mode of use allows domA to grant domB access to a specific frame,
+whilst retaining ownership. The block front driver uses this to grant memory
+access to the block back driver, so that it may read or write as requested.
+
+ 1. domA creates a grant access reference, and transmits the ref id to domB.
+ 2. domB uses the reference to map the granted frame.
+ 3. domB performs the memory access.
+ 4. domB unmaps the granted frame.
+ 5. domA removes its grant.
+
+
+The second mode allows domA to accept a transfer of ownership of a frame from
+domB. The net front and back driver will use this for packet tx/rx. This
+mechanism is still being implemented, though the xen<->guest interface design
+is complete.
+
+ 1. domA creates an accept transfer grant reference, and transmits it to domB.
+ 2. domB uses the ref to hand over a frame it owns.
+ 3. domA accepts the transfer
+ 4. domA clears the used reference.
+
+
+********************************************************************************
+ Data structures
+ ~~~~~~~~~~~~~~~
+
+ The following data structures are used by Xen and the guests to implement
+ grant tables:
+
+ 1. Shared grant entries
+ 2. Active grant entries
+ 3. Map tracking
+
+ These are not the users primary interface to grant tables, but are discussed
+ because an understanding of how they work may be useful. Each of these is a
+ finite resource.
+
+ Shared grant entries
+ ~~~~~~~~~~~~~~~~~~~~
+
+ A set of pages are shared between Xen and a guest, holding the shared grant
+ entries. The guest writes into these entries to create grant references. The
+ index of the entry is transmitted to the remote domain: this is the
+ reference used to activate an entry. Xen will write into a shared entry to
+ indicate to a guest that its grant is in use.
+  sha->domid : remote domain being granted rights
+  sha->frame : machine frame being granted
+  sha->flags : allow access, allow transfer, remote is reading/writing, etc.
+
+ Active grant entries
+ ~~~~~~~~~~~~~~~~~~~~
+
+ Xen maintains a set of private frames per domain, holding the active grant
+ entries for safety, and to reference count mappings.
+  act->domid : remote domain being granted rights
+  act->frame : machine frame being granted
+  act->pin   : used to hold reference counts
+
+ Map tracking
+ ~~~~~~~~~~~~
+
+ Every time a frame is mapped, a map track entry is stored in the metadata of
+ the mapping domain. The index of this entry is returned from the map call,
+ and is used to unmap the frame. Map track entries are also searched whenever a
+ page table entry containing a foreign frame number is overwritten: the first
+ matching map track entry is then removed, as if unmap had been invoked.
+ These are not used by the transfer mechanism.
+  map->domid         : owner of the mapped frame
+  map->ref_and_flags : grant reference, ro/rw, mapped for host or device access
+
+********************************************************************************
+
+ Granting a foreign domain access to frames
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+ domA [frame]--> domB
+
+
+ domA:  #include <asm-xen/gnttab.h>
+        grant_ref_t gref[BATCH_SIZE];
+
+        for ( i = 0; i < BATCH_SIZE; i++ )
+            gref[i] = gnttab_grant_foreign_access( domBid, mfn, (readonly ? 1 
: 0) );
+
+
+ .. gref is then somehow transmitted to domB for use.
+
+
+ Mapping foreign frames
+ ~~~~~~~~~~~~~~~~~~~~~~
+
+ domB:  #include <asm-xen/hypervisor.h>
+        unsigned long       mmap_vstart;
+        gnttab_op_t         aop[BATCH_SIZE];
+        grant_ref_t         mapped_handle[BATCH_SIZE];
+
+        if ( (mmap_vstart = allocate_empty_lowmem_region(BATCH_SIZE)) == 0 )
+            BUG();
+
+        for ( i = 0; i < BATCH_SIZE; i++ )
+        {
+            aop[i].u.map_grant_ref.host_virt_addr =
+                                              mmap_vstart + (i * PAGE_SIZE);
+            aop[i].u.map_grant_ref.dom      = domAid;
+            aop[i].u.map_grant_ref.ref      = gref[i];
+            aop[i].u.map_grant_ref.flags    = ( GNTMAP_host_map | 
GNTMAP_readonly );
+        }
+
+        if ( unlikely(HYPERVISOR_grant_table_op(
+                        GNTTABOP_map_grant_ref, aop, BATCH_SIZE)))
+            BUG();
+
+        for ( i = 0; i < BATCH_SIZE; i++ )
+        {
+            if ( unlikely(aop[i].u.map_grant_ref.dev_bus_addr == 0) )
+            {
+                tidyup_all(aop, i);
+                goto panic;
+            }
+
+            phys_to_machine_mapping[__pa(mmap_vstart + (i * 
PAGE_SIZE))>>PAGE_SHIFT] =
+                FOREIGN_FRAME(aop[i].u.map_grant_ref.dev_bus_addr);
+
+            mapped_handle[i] = aop[i].u.map_grant_ref.handle;
+        }
+
+
+
+ Unmapping foreign frames
+ ~~~~~~~~~~~~~~~~~~~~~~~~
+
+ domB:
+        for ( i = 0; i < BATCH_SIZE; i++ )
+        {
+            aop[i].u.unmap_grant_ref.host_virt_addr = mmap_vstart + (i * 
PAGE_SIZE);
+            aop[i].u.unmap_grant_ref.dev_bus_addr   = 0;
+            aop[i].u.unmap_grant_ref.handle         = mapped_handle[i];
+        }
+        if ( unlikely(HYPERVISOR_grant_table_op(
+                        GNTTABOP_unmap_grant_ref, aop, BATCH_SIZE)))
+            BUG();
+
+
+ Ending foreign access
+ ~~~~~~~~~~~~~~~~~~~~~
+
+    Note that this only prevents further mappings; it does _not_ revoke access.
+    Should _only_ be used when the remote domain has unmapped the frame.
+    gnttab_query_foreign_access( gref ) will indicate the state of any mapping.
+
+ domA:
+        if ( gnttab_query_foreign_access( gref[i] ) == 0 )
+            gnttab_end_foreign_access( gref[i], readonly );
+
+        TODO: readonly yet to be implemented.
+
+
+********************************************************************************
+
+ Transferring ownership of a frame to another domain
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+ [ XXX: Transfer mechanism is alpha-calibre code, untested, use at own risk 
XXX ]
+ [ XXX: show use of batch operations below, rather than single frame XXX ]
+ [ XXX: linux internal interface could/should be wrapped to be tidier XXX ]
+
+
+ Prepare to accept a frame from a foreign domain
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+  domA:
+        if ( (p = alloc_page(GFP_HIGHUSER)) == NULL )
+        {
+            printk("Cannot alloc a frame to surrender\n");
+            break;
+        }
+        pfn = p - mem_map;
+        mfn = phys_to_machine_mapping[pfn];
+                                                                               
        
+        if ( !PageHighMem(p) )
+        {
+            v = phys_to_virt(pfn << PAGE_SHIFT);
+            scrub_pages(v, 1);
+            queue_l1_entry_update(get_ptep((unsigned long)v), 0);
+        }
+                                                                               
        
+        /* Ensure that ballooned highmem pages don't have cached mappings. */
+        kmap_flush_unused();
+
+        /* Flush updates through and flush the TLB. */
+        xen_tlb_flush();
+                                                                               
        
+        phys_to_machine_mapping[pfn] = INVALID_P2M_ENTRY;
+                                                                               
        
+        if ( HYPERVISOR_dom_mem_op(
+            MEMOP_decrease_reservation, &mfn, 1, 0) != 1 )
+        {
+            printk("MEMOP_decrease_reservation failed\n");
+            /* er... ok. free the page then */
+            __free_page(p);
+            break;
+        }
+                                                                               
        
+        accepting_pfn = pfn;
+        ref = gnttab_grant_foreign_transfer( (domid_t) args.arg[0], pfn );
+        printk("Accepting dom %lu frame at ref (%d)\n", args.arg[0], ref);
+                                                                               
        
+
+ Transfer a frame to a foreign domain
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+  domB:
+        mmu_update_t            update;
+        domid_t                 domid;
+        grant_ref_t             gref;
+        unsigned long           pfn, mfn, *v;
+        struct page            *transfer_page = 0;
+                                                                               
        
+        /* alloc a page and grant access.
+         * alloc page returns a page struct. */
+        if ( (transfer_page = alloc_page(GFP_HIGHUSER)) == NULL )
+            return -ENOMEM;
+
+        pfn = transfer_page - mem_map;
+        mfn = phys_to_machine_mapping[pfn];
+
+        /* need to remove all references to this page */
+        if ( !PageHighMem(transfer_page) )
+        {
+            v = phys_to_virt(pfn << PAGE_SHIFT);
+            scrub_pages(v, 1);
+            sprintf((char *)v, "This page (%lx) was transferred.\n", mfn);
+            queue_l1_entry_update(get_ptep((unsigned long)v), 0);
+        }
+#ifdef CONFIG_XEN_SCRUB_PAGES
+        else
+        {
+            v = kmap(transfer_page);

_______________________________________________
Xen-changelog mailing list
Xen-changelog@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-changelog

<Prev in Thread] Current Thread [Next in Thread>
  • [Xen-changelog] Grant tables: substantially more robust., BitKeeper Bot <=