This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
Home Products Support Community News


RE: [Xen-devel] [PATCH] passthru: Fix pci bar remapping for passthru dev

To: Wei Wang2 <wei.wang2@xxxxxxx>
Subject: RE: [Xen-devel] [PATCH] passthru: Fix pci bar remapping for passthru devices
From: "Jiang, Yunhong" <yunhong.jiang@xxxxxxxxx>
Date: Tue, 21 Jul 2009 17:06:21 +0800
Accept-language: en-US
Acceptlanguage: en-US
Cc: "xen-devel@xxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxx>
Delivery-date: Tue, 21 Jul 2009 02:07:00 -0700
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <200907201829.42618.wei.wang2@xxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
References: <200907201603.56725.wei.wang2@xxxxxxx> <200907201647.09059.wei.wang2@xxxxxxx> <E2263E4A5B2284449EEBD0AAB751098402CD29FF37@xxxxxxxxxxxxxxxxxxxxxxxxxxxx> <200907201829.42618.wei.wang2@xxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
Thread-index: AcoJV05TUgwcZj/3RPKKGtag3la84AAVf4rg
Thread-topic: [Xen-devel] [PATCH] passthru: Fix pci bar remapping for passthru devices
Thanks for your reply very much. BTW, I'm not object the patch , the idea of 
the patch is quite straightforward. 

In kernel side, the issue has been discussed a lot ( e.g 
http://kerneltrap.org/mailarchive/linux-kernel/2007/8/28/165632). The main 
challenge in kernel side is MMCFG (maybe because the MMIO for display is 
intercepted before MMCFG?? I'm not sure). So if the issue is caused by geust 
memory access, it may cause issue on native also.
I think the 64K mmio is ok because it will change mapping to 0xFFxxxxx, which 
is higher than APIC/HPET range. I suspect 32M is the minimal size that will 
cause problem.
Anyway, this is only for some clarification and should not matter much.

Yunhong Jiang

Wei Wang2 wrote:
> Hi Yunhong
> Thanks for the comment. Regarding the reason of guest hang, my
> guess is: guest
> OS might be trying to access guest memory at "0xfe000000"
> after remapping
> this address to mmio and guest memory may be corrupted after
> this access.
> For devices with large mmio, the complement code of mmio block
> size is more
> likely to be a real guest RAM. I saw the same issue when I
> assigned a graphic
> device with 256MB mmio to Linux guest. But devices with small
> mmio seems to
> work well. I did not see the problem on a broadcom NIC with 64K mmio.
> Thanks, Wei
> On Monday 20 July 2009 17:48:46 Jiang, Yunhong wrote:
>> Wei Wang2 wrote:
>>> Hi Yunhong
>>> My testing shows that Linux guests will try to probe pci bar with
>>> MMIO being enabled. When I assign a broadcom NIC with 32MB MMIO a
>>> Linux guest, guest will hang after remapping a guest address
>> Yes, I remember I saw this bug in kernel before.
>> I suspect the issue here is the local APIC address, which should not
>> be intercepted before MMIO/RAM address in native. i.e. the p2m table
>> should not cover the local APIC address as RAM, but I think your
>> change in the qemu side is more straightforward (otherwise, we may
>> need consider IOAPIC, HPET etc, which is complex). 
>> One thing left is, why it will hang, after all, guest will try to
>> restore the BAR address later, and at that time, the local apic
>> access can be intercepted again. 
>> --jyh
>>> "0xfe000000" to physical mmio. However, windows and BSD guests do
>>> not have this issue. They alway probe mmio size after disabling
>>> mmio. Thanks, Wei
>>> On Monday 20 July 2009 16:26:00 Jiang, Yunhong wrote:
>>>> I assume guest should disable the MMIO in PCI_COMMAND before
>>>> writing all "1"s to bar register. Otherwise, what will happen on
>>>> native if guest try to access 0xFFFFFFF0? And if we do update the
>>>> P2M, will it cause trouble to Xen HV? 
>>>> Thanks
>>>> Yunhong Jiang
>>>> xen-devel-bounces@xxxxxxxxxxxxxxxxxxx wrote:
>>>>> Hi,
>>>>> When guest code tries to get the block size of mmio, it will write
>>>>> all "1"s into pci bar register and then qemu will return all "0"s
>>>>> to the don't care bits in the emulated bar register to indicate
>>>>> the block size to guest code.
>>>>> In this case, we should not create p2m mapping in
>>>>> pt_bar_reg_write() and
>>>>> pt_exp_rom_bar_reg_write(). Attached patch fixes this issue,
>>>>> additional comment can be found in the patch.
>>>>> Thanks,
>>>>> Wei
>>>>> Signed-off-by: Wei Wang <wei.wang2@xxxxxxx>
>>>>> --
>>>>> AMD GmbH, Germany
>>>>> Operating System Research Center
>>>>> Legal Information:
>>>>> Advanced Micro Devices GmbH
>>>>> Karl-Hammerschmidt-Str. 34
>>>>> 85609 Dornach b. München
>>>>> Geschäftsführer: Jochen Polster, Thomas M. McCoy, Giuliano Meroni
>>>>> Sitz: Dornach, Gemeinde Aschheim, Landkreis München
>>>>> Registergericht München, HRB Nr. 43632
Xen-devel mailing list