WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-devel

[Xen-devel] Non-page aligned MMIO BAR error on dom-0 on Xen HVM setup

To: xen-devel@xxxxxxxxxxxxxxxxxxx
Subject: [Xen-devel] Non-page aligned MMIO BAR error on dom-0 on Xen HVM setup
From: Purna Chandar <purnachandar.midi@xxxxxxxxxxxxxxxxxx>
Date: Thu, 15 Oct 2009 13:00:31 +0530
Delivery-date: Thu, 15 Oct 2009 00:33:24 -0700
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
Hi,
I am using Neterion x3100  multi-function NIC adapter on stable Xen(2.6.18) in HVM mode. I think the problem I have (explained below) is a general one and has got nothing to do with neterion card in specific.
I am trying to assign one of the functions on the adapter to a guest OS(FC11 64 bit) in pass-through mode. When I try to start the guest OS (using xm create), the dom-0 throws  the error saying the pass-through device has a "non-page aligned MMIO BAR" and cannot boot. The function infact has a BAR (bar 4) whose size is 256 bytes and the resources are mapped on a page-aligned boundary. THe guest cannot be booted for some reason because of this.

To get around with this, I used the following kernel command line option:

reassign_resources  guestdev=<b:d.f>,<b:d,f>...

The BDFs of all the devices whose resources we want to re-assign to a page-boundary are given in guestdev parameter above. As these neterion functions are under a bridge, the bridge's BDF is also given in the guestdev option (without the bridge's BDF, I had issues with loading the neterion driver on guest OS).

With this kernel option, I am able to boot the guest OS but I see a different problem in the config space of the device in dom-0. The BAR regions are shown up as "disabled" in lspci output shown below. Without this option, the BARs are enabled.

01:00.2 Ethernet controller: Neterion Inc. X3100 Series 10 Gigabit Ethernet PCIe (rev 01)
        Subsystem: Neterion Inc. X3110 Single Port 10GBase-CR                           
        Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Interrupt: pin A routed to IRQ 16                                                                   
        Region 0: Memory at f9000000 (64-bit, prefetchable) [disabled] [size=8M]                            
        Region 2: Memory at fc404000 (64-bit, prefetchable) [disabled] [size=4K]                            
        Region 4: Memory at fc405000 (64-bit, prefetchable) [disabled] [size=256]                           
        [virtual] Expansion ROM at fc100000 [disabled] [size=512K]                      

What this "disabled" means is the resources are not treated as "memory mapped resources" at all (as bit-2 in COMMAND register is not set in config space).
Surprisingly, on the guest OS, the "disabled" flag is not seen on the BAR regions. I can successfully load the driver and do some data flow on pass-through device on the guest.

Things seem fine so far despite the problem above on dom-0.

But, when I trigger an FLR (function level reset) on the pass-through device/function on dom-0 (not on dom-u), I have problems on guest OS. The "disabled" bit appears in the lspci output on the device in the guest. The driver is not able to do any memory-mapped reads/writes and hence I am not able to load the driver. I triggered the FLR using following command:
# setpci -s 01:00.2 CAP_EXP+8.w=0xa814
(Before triggering FLR, value at CAP_EXP+8.w is 0x2814. As we have to set last bit in PCI Express capability for FLR, value used in above command is 0xa814).

Does anybody have similar issues? Is "reassign_resources" the correct kernel option to use? If not, how do I get around the "non-page alignment problem"

Any bit of help will be really helpful.

Thanks,
PurnaChandar M
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel
<Prev in Thread] Current Thread [Next in Thread>