[Xen-devel] pt_msix_init: Error: Can't map physical MSI-X table: Invalid argument

["Cinco, Dante" <Dante.Cinco@xxxxxxx>]

I get the above error message when I boot the domU (Linux 2.6.30.1). I'm using Xen 3.5-unstable (changeset 20303) with pv_ops kernel 2.6.31.1. I use pci-stub to hide the device so that I can assign/pass it through to domU. The error is coming from ioemu-xxx/hw/pt-msi.c. I think it is complaining about the table_off that is part of the mmap() call:

 

    dev->msix->phys_iomem_base = mmap(0, total_entries * 16,

                          PROT_WRITE | PROT_READ, MAP_SHARED | MAP_LOCKED,

                          fd, dev->msix->table_base + table_off);

 

When I printed table_off, its value is 0x4100 (not page-aligned) which is consistent with the BAR4 offset of the PCI device (see below). I think mmap() requires the last argument to be a multiple of the page size and results is a map error if it is not. Here's the partial output of "lspci -vv -s 0:07:0.1" from dom0:

 

07:00.0 Fibre Channel: PMC-Sierra Inc. Device 8032 (rev 05)

        Subsystem: Atto Technology Device 003c

        Latency: 0, Cache Line Size: 64 bytes

        Interrupt: pin A routed to IRQ 11

        Capabilities: [60] Message Signalled Interrupts: Mask- 64bit+ Queue=0/1 Enable+

                Address: 00000000fee11000  Data: 405e

        Capabilities: [70] Express (v2) Endpoint, MSI 01

                DevCap: MaxPayload 4096 bytes, PhantFunc 0, Latency L0s <4us, L1 unlimited

                        ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset-

        Capabilities: [b0] MSI-X: Enable- Mask- TabSize=9

                Vector table: BAR=4 offset=00004100

                PBA: BAR=4 offset=00004000

 

I'm able to workaround this problem by modifying pt-msi.c: force offset to zero in mmap() and on the next line, adjust the pointer by table_off:

 

    table_off_adjust = table_off & 0x0fff;

    dev->msix->phys_iomem_base = mmap(0, total_entries * 16 + table_off_adjust,

                          PROT_WRITE | PROT_READ, MAP_SHARED | MAP_LOCKED,

                          fd, dev->msix->table_base + table_off - table_off_adjust);

    dev->msix->phys_iomem_base = (void *)((char *)dev->msix->phys_iomem_base + table_off_adjust);

 

Ideally, I should only be applying this to my device but since I'm not passing any other device that use MSI, I don't see any negative effect. In addition, my device only uses MSI and not MSI-X.

 

Obviously, this is a custom hack but I was wondering if there is a cleaner solution for this. Is a non-page aligned BAR offset unusual? When I boot Linux 2.6.30.1 on bare metal (no Xen), I don't see this error.

 

Dante Cinco

 

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel