This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
Home Products Support Community News


Re: [Xen-devel] [PATCH] vt-d: ensure x2apic is not enabled accidently if

To: "Cui, Dexuan" <dexuan.cui@xxxxxxxxx>
Subject: Re: [Xen-devel] [PATCH] vt-d: ensure x2apic is not enabled accidently if no DRHD at all.
From: Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx>
Date: Fri, 5 Mar 2010 12:51:29 -0500
Cc: "xen-devel@xxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxx>, Keir Fraser <keir.fraser@xxxxxxxxxxxxx>, Jan Beulich <JBeulich@xxxxxxxxxx>, "Zhang, Xiantao" <xiantao.zhang@xxxxxxxxx>
Delivery-date: Fri, 05 Mar 2010 10:21:09 -0800
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <ED3036A092A28F4C91B0B4360DD128EABDE28AE3@xxxxxxxxxxxxxxxxxxxxxxxxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
References: <ED3036A092A28F4C91B0B4360DD128EABDE28AE3@xxxxxxxxxxxxxxxxxxxxxxxxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
User-agent: Mutt/1.5.19 (2009-01-05)
On Thu, Mar 04, 2010 at 12:18:19PM +0800, Cui, Dexuan wrote:
> Thank Jan Beulich for pointing this out

You need more than that.

Here is what I see

(XEN) Enabling APIC mode:  Phys.  Using 3 I/O APICs
(XEN) ACPI: HPET id: 0x8086a301 base: 0xfed00000
(XEN) [VT-D]dmar.c:637: Host address width 40
(XEN) [VT-D]dmar.c:646: found ACPI_DMAR_DRHD:
(XEN) [VT-D]dmar.c:379:   dmaru->address = fe710000
(XEN) [VT-D]dmar.c:336:   IOAPIC: 0:1f.7
(XEN) [VT-D]dmar.c:391:   flags: INCLUDE_ALL

So one DRHD for three IOAPICs. In Linux kernel it would turn off x2APIC
(XEN) [VT-D]dmar.c:650: found ACPI_DMAR_RMRR:
(XEN) [VT-D]dmar.c:331:   endpoint: 0:1d.0
(XEN) [VT-D]dmar.c:331:   endpoint: 0:1d.1
(XEN) [VT-D]dmar.c:331:   endpoint: 0:1d.2
(XEN) [VT-D]dmar.c:331:   endpoint: 0:1d.7
(XEN) [VT-D]dmar.c:331:   endpoint: 0:1a.0
(XEN) [VT-D]dmar.c:331:   endpoint: 0:1a.1
(XEN) [VT-D]dmar.c:331:   endpoint: 0:1a.2
(XEN) [VT-D]dmar.c:331:   endpoint: 0:1a.7
(XEN) [VT-D]dmar.c:540:   RMRR region: base_addr bf3f9000 end_address bf40c548
(XEN) PCI: MCFG configuration 0: base c0000000 segment 0 buses 0 - 255
(XEN) PCI: MCFG area at c0000000 reserved in E820
(XEN) iommu_supports_eim: 1 1 1
(XEN) iommu_supports_eim: 1 1 1
(XEN) Switched to APIC driver x2apic_cluster.

Whoopsie. Here it calls the routine you patched, which just checks to
see if there are DRHDs, but doesn't do a DRHD.count == IOAPCI.count.

(XEN) x2APIC mode enabled.
(XEN) Using ACPI (MADT) for SMP configuration information
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Initializing CPU#0
(XEN) Detected 2000.131 MHz processor.
(XEN) Initing memory sharing.
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 256K
(XEN) CPU: L3 cache: 8192K
(XEN) CPU: Physical Processor ID: 0
(XEN) CPU: Processor Core ID: 0
(XEN) VMX: Supported advanced features:
(XEN)  - APIC MMIO access virtualisation
(XEN)  - APIC TPR shadow
(XEN)  - Extended Page Tables (EPT)
(XEN)  - Virtual-Processor Identifiers (VPID)
(XEN)  - Virtual NMI
(XEN)  - MSR direct-access bitmap
(XEN) HVM: ASIDs enabled. 
(XEN) HVM: VMX enabled
(XEN) HVM: Hardware Assisted Paging detected.
(XEN) Intel machine check reporting enabled on CPU#0.
(XEN) CPU0: Thermal monitoring enabled (TM1)
(XEN) [VT-D]iommu.c:1072: drhd->address = fe710000
(XEN) [VT-D]iommu.c:1073: iommu->reg = ffff82c3fff57000
(XEN) Intel VT-d Snoop Control supported.
(XEN) Intel VT-d DMA Passthrough not supported.
(XEN) Intel VT-d Queued Invalidation supported.
(XEN) Intel VT-d Interrupt Remapping supported.
(XEN) [VT-D]iommu.c:1838:  apic:0x0/0x3, 0x0
(XEN) [VT-D]iommu.c:1838:  apic:0x1/0x3, 0x2
(XEN) [VT-D]iommu.c:1845: ioapic_to_iommu: ioapic 0x1 (id: 0x2) is NULL! Will 
not try to enable Interrupt Remapping.

.. Here later on, it does the check and turns iommu_intr off.
(XEN) Intel VT-d Snoop Control enabled.
(XEN) Intel VT-d DMA Passthrough disabled .
(XEN) Intel VT-d Queued Invalidation enabled.
(XEN) Intel VT-d Interrupt Remapping disabled .
(XEN) Intel VT-D turned on!???
(XEN) IOMMU rc: 0, 1
(XEN) I/O virtualisation enabled
(XEN) I/O virtualisation for PV guests disabled
(XEN) CPU0: Intel Genuine Intel(R) CPU           @ 0000 @ 2.00GHz stepping 02
(XEN) Booting processor 1/1 eip 88000
(XEN) Initializing CPU#1
(XEN) x2APIC mode enabled.
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K

Here is the little debug patch I used

diff -r b8d2a4134a68 xen/drivers/passthrough/iommu.c
--- a/xen/drivers/passthrough/iommu.c   Wed Mar 03 17:41:58 2010 +0000
+++ b/xen/drivers/passthrough/iommu.c   Fri Mar 05 12:50:36 2010 -0500
@@ -282,6 +282,7 @@
         rc = iommu_hardware_setup();
         iommu_enabled = (rc == 0);
+       printk("IOMMU rc: %d, %d\n", rc, iommu_enabled);
     if ( force_iommu && !iommu_enabled )
diff -r b8d2a4134a68 xen/drivers/passthrough/vtd/intremap.c
--- a/xen/drivers/passthrough/vtd/intremap.c    Wed Mar 03 17:41:58 2010 +0000
+++ b/xen/drivers/passthrough/vtd/intremap.c    Fri Mar 05 12:50:36 2010 -0500
@@ -128,9 +128,15 @@
     struct acpi_drhd_unit *drhd;
+    printk("%s: %d %d %d\n", __FUNCTION__,iommu_enabled, iommu_qinval, 
     if ( !iommu_enabled || !iommu_qinval || !iommu_intremap )
         return 0;
+    if ( list_empty(&acpi_drhd_units) )
+        return 0;
     for_each_drhd_unit ( drhd )
         if ( !ecap_queued_inval(drhd->ecap) ||
              !ecap_intr_remap(drhd->ecap) ||
diff -r b8d2a4134a68 xen/drivers/passthrough/vtd/iommu.c
--- a/xen/drivers/passthrough/vtd/iommu.c       Wed Mar 03 17:41:58 2010 +0000
+++ b/xen/drivers/passthrough/vtd/iommu.c       Fri Mar 05 12:50:36 2010 -0500
@@ -1834,6 +1834,8 @@
         int apic;
         for ( apic = 0; apic < nr_ioapics; apic++ )
+           dprintk(XENLOG_INFO VTDPREFIX," apic:0x%x/0x%x, 0x%x\n",
+                       apic, nr_ioapics, IO_APIC_ID(apic));
             if ( ioapic_to_iommu(IO_APIC_ID(apic)) == NULL )
                 iommu_intremap = 0;
@@ -1972,6 +1974,14 @@
     if ( init_vtd_hw() )
         goto error;
+#define P(p,s) printk("Intel VT-d %s %s.\n", s, (p)? "enabled" : "disabled ")
+    P(iommu_snoop, "Snoop Control");
+    P(iommu_passthrough, "DMA Passthrough");
+    P(iommu_qinval, "Queued Invalidation");
+    P(iommu_intremap, "Interrupt Remapping");
+#undef P
+    printk("Intel VT-D turned on!???\n");
     register_keyhandler('V', &dump_iommu_info_keyhandler);
     return 0;
@@ -1984,6 +1994,7 @@
     iommu_passthrough = 0;
     iommu_qinval = 0;
     iommu_intremap = 0;
+    printk("Intel VT-d is turned off due to errors.\n");
     return -ENOMEM;
> Thanks,
> -- Dexuan

> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-devel

Xen-devel mailing list

<Prev in Thread] Current Thread [Next in Thread>