WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-devel

RE: [Xen-devel] IRQ SMP affinity problems in domU with vcpus > 4 on HP P

To: "Zhang, Xiantao" <xiantao.zhang@xxxxxxxxx>, "Cinco, Dante" <Dante.Cinco@xxxxxxx>, "He, Qing" <qing.he@xxxxxxxxx>
Subject: RE: [Xen-devel] IRQ SMP affinity problems in domU with vcpus > 4 on HP ProLiant G6 with dual Xeon 5540 (Nehalem)
From: "Zhang, Xiantao" <xiantao.zhang@xxxxxxxxx>
Date: Thu, 22 Oct 2009 10:42:50 +0800
Accept-language: en-US
Acceptlanguage: en-US
Cc: "xen-devel@xxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxx>, Ian Jackson <Ian.Jackson@xxxxxxxxxxxxx>, Fraser <keir.fraser@xxxxxxxxxxxxx>
Delivery-date: Wed, 21 Oct 2009 19:44:45 -0700
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <706158FABBBA044BAD4FE898A02E4BC201C9BD914F@xxxxxxxxxxxxxxxxxxxxxxxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
References: <706158FABBBA044BAD4FE898A02E4BC201C9BD8CED@xxxxxxxxxxxxxxxxxxxxxxxxxxxx> <2B044E14371DA244B71F8BF2514563F503FC081B@xxxxxxxxxxxxxxxxx> <706158FABBBA044BAD4FE898A02E4BC201C9BD914F@xxxxxxxxxxxxxxxxxxxxxxxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
Thread-index: AcpOPi4Q9UlUI8ufS5qQ1CJSqDysQAAAM40gAAsf3SAABZW6UAAQxcbQAJQYlsAADK/qgAAEGtWgABRnJ9AAEEcpwAAiDlkgABCR9DAAApQ+oA==
Thread-topic: [Xen-devel] IRQ SMP affinity problems in domU with vcpus > 4 on HP ProLiant G6 with dual Xeon 5540 (Nehalem)
Zhang, Xiantao wrote:
> Dante,
>    Have you applied the two patches when you did the testing?  
> Without them, we can reproduce the issue you reported, but with them,
> the issue is gone.  The root-cause is that when program MSI, we have
> to mask the MSI interrupt source first, otherwise it may generate
> inconistent interrupts with incorrect destination and right vector or
> incorrect vector and right destination.     
> 
> For exmaple, if the old MSI interrupt info is 0.186 which means the
> destination id is 0 and the vector is 186, but when the IRQ migrates
> to another cpu(e.g.  Cpu 1), the MSI info should be changed to 1.194.
> When you programs MSI info to pci device, if not mask it first, it
> may generate the interrupt as 1.186 or 0.194. Obviously, ther
> interrupts with the info 1.186 and 0.194 doesn't exist, and according
> to the spec, any combination is possible. Since Xen writes addr field
> first, so it is likely to generate 1.186 instead of 0.194, so your
> pci devices may generate interrupt with new destination and old
> vector(1.186).    In my two patches, one is used to fix guest
> interrupt affinity issue(a race exists between guest eoi old vector
> and guest setting new vector), and another one is used to safely
> program MSI info to pci devices to avoid inconsistent interrupts
> generation.             
> 
>> (XEN) traps.c:1626: guest_io_write::pci_conf_write data=0x40ba
> 
> This should be written by dom0(likely to be Qemu).  And if it does
> exist, we may have to prohibit such unsafe writings about MSI in
> Qemu.  

Another issue may exist which leads to the issue.  Currenlty, both Qemu and 
hypervisor can program MSI but Xen lacks synchronization mechnism between them 
to avoid race.  As said in the last mail,  Qemu shouldn't be allowed to do the 
unsafe writing about MSI Info, and insteadly,  it should resort to hypervisor 
through hypercall for MSI programing, otherwise, Qemu may write staled MSI info 
to PCI devices  and leads to the strange issues.   
Keir/Ian
        What's your opinion about the potential issue ?  Maybe we need to add a 
lock between them or just allow hypervisor to do the writing ?    
Xiantao
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel

<Prev in Thread] Current Thread [Next in Thread>