Re: [Xen-devel] [PATCH 15/16] Infrastructure for manipulating 3-level event channel pages

[Ian Campbell <Ian.Campbell@xxxxxxxxxx>]

On Mon, 2013-02-04 at 09:23 +0000, Jan Beulich wrote:
> >>> On 31.01.13 at 15:43, Wei Liu <wei.liu2@xxxxxxxxxx> wrote:
> > +static long __map_l3_arrays(struct domain *d, xen_pfn_t *pending,
> > +                            xen_pfn_t *mask, int nr_pages)
> > +{
> > +    int rc;
> > +    void *mapping;
> > +    struct page_info *pginfo;
> > +    unsigned long gfn;
> > +    int pending_count = 0, mask_count = 0;
> > +
> > +#define __MAP(src, dst, cnt)                                    \
> > +    for ( (cnt) = 0; (cnt) < nr_pages; (cnt)++ )                \
> > +    {                                                           \
> > +        rc = -EINVAL;                                           \
> > +        gfn = (src)[(cnt)];                                     \
> > +        pginfo = get_page_from_gfn(d, gfn, NULL, P2M_ALLOC);    \
> > +        if ( !pginfo )                                          \
> > +            goto err;                                           \
> > +        if ( !get_page_type(pginfo, PGT_writable_page) )        \
> > +        {                                                       \
> > +            put_page(pginfo);                                   \
> > +            goto err;                                           \
> > +        }                                                       \
> > +        mapping = __map_domain_page_global(pginfo);             \
> > +        if ( !mapping )                                         \
> > +        {                                                       \
> > +            put_page_and_type(pginfo);                          \
> > +            rc = -ENOMEM;                                       \
> > +            goto err;                                           \
> > +        }                                                       \
> > +        (dst)[(cnt)] = mapping;                                 \
> > +    }
> > +
> > +    __MAP(pending, d->evtchn_pending, pending_count)
> > +    __MAP(mask, d->evtchn_mask, mask_count)
> > +#undef __MAP
> > +
> > +    rc = 0;
> > +
> > + err:
> > +    return rc;
> > +}
> 
> So this alone already is up to 16 pages per guest, and hence a
> theoretical maximum of 512k pages, i.e. 2G mapped space.

That's given a theoretical 32k guests? Ouch. It also ignores the need
for other global mappings.

on the flip side only a minority of domains are likely to be using the
extended scheme, and I expect even those which are would not be using
all 16 pages, so maybe we can fault them in on demand as we bind/unbind
evtchns.

Where does 16 come from? How many pages to we end up with at each level
in the new scheme?

Some levels of the trie are per-VCPU, did you account for that already
in the 2GB?

>  The
> global page mapping area, however, is only 1Gb in size on x86-64
> (didn't check ARM at all)...

There isn't currently a global page mapping area on 32-bit ARM (I
suppose we have avoided them somehow...) but obviously 2G would be a
problem in a 4GB address space.

On ARM we currently have 2G for domheap mappings which I suppose we
would split if we needed a global page map

These need to be global so we can deliver evtchns to VCPUs which aren't
running, right? I suppose mapping on demand (other than for a running
VCPU) would be prohibitively expensive.

Could we make this space per-VCPU (or per-domain) by saying that a
domain maps its own evtchn pages plus the required pages from other
domains with which an evtchn is bound? Might be tricky to arrange
though, especially with the per-VCPU pages and affinity changes?

Ian.


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel