This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
Home Products Support Community News


[Xen-devel] RE: [PATCH RFC] Per-CPU vector for Xen.

To: "Zhang, Xiantao" <xiantao.zhang@xxxxxxxxx>, Keir Fraser <keir.fraser@xxxxxxxxxxxxx>
Subject: [Xen-devel] RE: [PATCH RFC] Per-CPU vector for Xen.
From: "Zhang, Xiantao" <xiantao.zhang@xxxxxxxxx>
Date: Tue, 18 Aug 2009 13:45:08 +0800
Accept-language: en-US
Acceptlanguage: en-US
Cc: "xen-devel@xxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxx>, "Kay, Allen M" <allen.m.kay@xxxxxxxxx>, "Jiang, Yunhong" <yunhong.jiang@xxxxxxxxx>, "Dong, Eddie" <eddie.dong@xxxxxxxxx>, "Yang, Xiaowei" <xiaowei.yang@xxxxxxxxx>, "Li, Xin" <xin.li@xxxxxxxxx>
Delivery-date: Mon, 17 Aug 2009 22:47:57 -0700
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <706158FABBBA044BAD4FE898A02E4BC201C0553F3C@xxxxxxxxxxxxxxxxxxxxxxxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
References: <706158FABBBA044BAD4FE898A02E4BC201C0553F3C@xxxxxxxxxxxxxxxxxxxxxxxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
Thread-index: AcoeT7N2lTmmOpL/QJqSqmCyZrF+cQBXhdOA
Thread-topic: [PATCH RFC] Per-CPU vector for Xen.
Hi, Keir
   Another issue or limitation is also found when we tested Per-CPU vector 
patch at our side. Currenlty, Xen uses fixmap to access MSI-X resouce, (e.g. 
msi-x tables), 32 pages  512 pages are reserved for 32-pae and x86_64 
separately in fixmap section, but  each MSI-X capable device(regardless of real 
device or virtual function) at least needs one page to map its resource, so 
these pages may easily run out with these devices, espeically on 32-bit 
platforms. For 64-bit platforms, we can reserve more pages to fix the issue, 
but for 32-bit platforms, it is not always safe to increase the number of pages 
due to limited virtual address space.  We have one optional solution to fix the 
issue through dynamically map/unmap MSI-X tables when access it, but the 
concern is that it may cost much due to frequent access for the resource.   
What's suggestion or good idea to address the issue ?  Thanks!

Zhang, Xiantao wrote:
> Hi, Keir
>    To support more interrupt vectors in Xen for more devices, 
> especially for SR-IOV devices in a large system, we implemented
> per-cpu vector for Xen like Linux does. For SR-IOV devices, since
> each VF needs several separate vectors for interrupt delivery and
> global ~200 vectors in Xen is insufficient and easily run out after
> installing two or three such devices. Becaue SR-IOV devices are
> becoming popular now,  and from this point of view, we have to extend
> vector resource space to make these devices work. As linux does, we
> implemented per-cpu vector for Xen to extend and scale vector
> resource to nr_cpus x ~200 in a system.   BTW, the core logic of the
> patches is ported from upstream linux and then adapted for Xen.      
> Patch 0001:  Change nr_irqs to nr_irqs_gsi and make nr_irqs_gsi only
> used for GSI interrupts. 
> Patch 0002:  Modify Xen from vector-based interrupt infrastructure to
> IRQ-based one, and the big change is that one irq number is also
> allocated for MSI interrupt source, and the idea is same as Linux's.  
> Patch 0003:  Implement per-cpu vector for xen. Most core logic(such
> as vector allocation algorithm, IRQ migration logic...) is ported
> from upstream Linux.  
> About the patch quality, we have done enough testings against
> upstream, and no any regression found after applying this patchset. 
> Please help to review.  Comments are very appreicated!  Thanks!
> Signed-off-by : Xiantao Zhang <xiantao.zhang@xxxxxxxxx>
> Xiantao

Xen-devel mailing list

<Prev in Thread] Current Thread [Next in Thread>