WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-devel

RE: [Xen-devel] [PATCH]x86:x2apic: Disable x2apic on x86-32 permanently

To: Keir Fraser <keir@xxxxxxx>, "Zhang, Fengzhe" <fengzhe.zhang@xxxxxxxxx>, "xen-devel@xxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxx>
Subject: RE: [Xen-devel] [PATCH]x86:x2apic: Disable x2apic on x86-32 permanently
From: "Li, Xin" <xin.li@xxxxxxxxx>
Date: Sat, 22 Jan 2011 00:07:12 +0800
Accept-language: zh-CN, en-US
Acceptlanguage: zh-CN, en-US
Cc:
Delivery-date: Fri, 21 Jan 2011 08:12:23 -0800
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <C95F568B.12394%keir@xxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
References: <FC2FB65B4D919844ADE4BE3C2BB739AD34F6D1F3@xxxxxxxxxxxxxxxxxxxxxxxxxxxx> <C95F568B.12394%keir@xxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
Thread-index: Acu5ZcYz2vLckx0kQOSJvRcHzARMuwABpsUGAAOGZgAAAVVBWAABVfdQ
Thread-topic: [Xen-devel] [PATCH]x86:x2apic: Disable x2apic on x86-32 permanently
> Okay. I think after 4.1 is branched we should simply remove 32-bit host
> support entirely. There's really no good reason for it to persist imo.

You're much more aggressive than me, but I totally agree :)
Thanks!
-Xin

> On 21/01/2011 15:19, "Li, Xin" <xin.li@xxxxxxxxx> wrote:
> 
> > Below is original log.  We found Xen hypervisor needs to map
> > VT-d/x2apci/MSI/MSI-x resources using map_domain_page.  And on 32bit, the
> map
> > page pool is trending to be used up.  Maybe we can fix it, but for large
> > machines with many devices and CPUs, we probably will still meet the 32bit
> > limitation.  So for new features which need map page in Xen hypervisor, I 
> > tend
> > to enable on 64 bit only.  I'm not sure how important 32bit Xen hypervisor 
> > is,
> > but I think 64 bit is becoming more and more popular.  Simply disabling is 
> > not
> > technically elegant, but for longer term, I think it's the trend because it
> > makes things simpler :).  Comments?
> > Thanks!
> > -Xin
> >
> > root (hd0,2)
> >  Filesystem type is ext2fs, partition type 0x83
> > kernel (hd0,2)/boot/xen.gz dom0_mem=512M iommu=1 loglvl=all guest_loglvl=all
> > un
> > restricted_guest=1  console=com1 sync_console iommu=1 msi=1
> conring_size=4M
> >    [Multiboot-elf, <0x100000:0x14433c:0x34cc4>, shtab=0x279078,
> > entry=0x100000]
> > module (hd0,2)/boot/vmlinuz-2.6-xen ro root=/dev/sda3 console=hvc0
> > earlyprintk=
> > xen ignore_loglevel pci=assign-busses
> >    [Multiboot-module @ 0x27a000, 0x3ede40 bytes]
> > module (hd0,2)/boot/initrd-2.6-xen.img
> >    [Multiboot-module @ 0x668000, 0x369800 bytes]
> >
> >  __  __            _  _    _                    _        _     _
> >  \ \/ /___ _ __   | || |  / |   _   _ _ __  ___| |_ __ _| |__ | | ___
> >   \  // _ \ '_ \  | || |_ | |__| | | | '_ \/ __| __/ _` | '_ \| |/ _ \
> >   /  \  __/ | | | |__   _|| |__| |_| | | | \__ \ || (_| | |_) | |  __/
> >  /_/\_\___|_| |_|    |_|(_)_|   \__,_|_| |_|___/\__\__,_|_.__/|_|\___|
> >
> > (XEN) Xen version 4.1-unstable (build@xxxxxxxxxxxx) (gcc version 4.1.2
> > 20080704 (Red Hat 4.1.2-44)) Sat Sep 11 06:03:28 CST 2010
> > (XEN) Latest ChangeSet: Fri Sep 10 19:06:33 2010 +0100 22132:3985fea87987
> > (XEN) Console output is synchronous.
> > (XEN) Bootloader: GNU GRUB 0.97
> > (XEN) Command line: dom0_mem=512M iommu=1 loglvl=all guest_loglvl=all
> > unrestricted_guest=1  console=com1 sync_console iommu=1 msi=1
> conring_size=4M
> > (XEN) Video information:
> > (XEN)  VGA is text mode 80x25, font 8x16
> > (XEN)  VBE/DDC methods: none; EDID transfer time: 0 seconds
> > (XEN)  EDID info not retrieved because no DDC retrieval method detected
> > (XEN) Disc information:
> > (XEN)  Found 1 MBR signatures
> > (XEN)  Found 1 EDD information structures
> > (XEN) Xen-e820 RAM map:
> > (XEN)  0000000000000000 - 000000000009bc00 (usable)
> > (XEN)  000000000009bc00 - 00000000000a0000 (reserved)
> > (XEN)  00000000000e0000 - 0000000000100000 (reserved)
> > (XEN)  0000000000100000 - 0000000020050000 (usable)
> > (XEN)  0000000020050000 - 0000000020250000 (reserved)
> > (XEN)  0000000020250000 - 0000000040000000 (usable)
> > (XEN)  0000000040000000 - 0000000040200000 (reserved)
> > (XEN)  0000000040200000 - 000000007f553000 (usable)
> > (XEN)  000000007f553000 - 000000007f5e7000 (reserved)
> > (XEN)  000000007f5e7000 - 000000007f7e7000 (ACPI NVS)
> > (XEN)  000000007f7e7000 - 000000007f7ff000 (ACPI data)
> > (XEN)  000000007f7ff000 - 000000007f800000 (usable)
> > (XEN)  000000007f800000 - 0000000080000000 (reserved)
> > (XEN)  00000000f8000000 - 00000000fc000000 (reserved)
> > (XEN)  00000000fec00000 - 00000000fec01000 (reserved)
> > (XEN)  00000000fed10000 - 00000000fed14000 (reserved)
> > (XEN)  00000000fed18000 - 00000000fed1a000 (reserved)
> > (XEN)  00000000fed1c000 - 00000000fed20000 (reserved)
> > (XEN)  00000000fee00000 - 00000000fee01000 (reserved)
> > (XEN)  00000000ff980000 - 00000000ffc00000 (reserved)
> > (XEN)  00000000ffd80000 - 0000000100000000 (reserved)
> > (XEN) System RAM: 2032MB (2081724kB)
> > (XEN) ACPI: RSDP 000F0410, 0024 (r2  INTEL)
> > (XEN) ACPI: XSDT 7F7FDE18, 006C (r1  INTEL  SNB-CPT  6222004 MSFT
> 10013)
> > (XEN) ACPI: FACP 7F752D98, 00F4 (r4  INTEL  SNB-CPT  6222004 MSFT
> 10013)
> > (XEN) ACPI: DSDT 7F725018, EACA (r2  INTEL  SNB-CPT        0 INTL
> 20100331)
> > (XEN) ACPI: FACS 7F7E6D40, 0040
> > (XEN) ACPI: APIC 7F7FCF18, 0092 (r2  INTEL  SNB-CPT  6222004 MSFT
> 10013)
> > (XEN) ACPI: TCPA 7F79FD18, 0032 (r2                        0
> 0)
> > (XEN) ACPI: MCFG 7F79FC98, 003C (r1 INTEL  SNDYBRDG  6222004 MSFT
> 97)
> > (XEN) ACPI: HPET 7F79FC18, 0038 (r1 A M I   PCHHPET  6222004 AMI.
> 3)
> > (XEN) ACPI: SSDT 7F753018, 1068 (r1 TrmRef PtidDevc     1000 INTL 20100331)
> > (XEN) ACPI: BOOT 7F79FB98, 0028 (r1 INTEL  SNB-CPT   6222004 AMI
> 10013)
> > (XEN) ACPI: SSDT 7F751018, 0996 (r1  PmRef    CpuPm     3000 INTL
> 20100331)
> > (XEN) ACPI: DMAR 7F752F18, 00B0 (r1 INTEL      SNB         1 INTL
> 1)
> > (XEN) No NUMA configuration found
> > (XEN) Faking a node at 0000000000000000-000000007f800000
> > (XEN) Xen heap: 9MB (9716kB)
> > (XEN) Domain heap initialised
> > (XEN) found SMP MP-table at 000fcaa0
> > (XEN) DMI 2.6 present.
> > (XEN) Using APIC driver default
> > (XEN) ACPI: PM-Timer IO Port: 0x408
> > (XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[404,0], pm1x_evt[400,0]
> > (XEN) ACPI: 32/64X FACS address mismatch in FADT -
> 7f79df40/000000007f7e6d40,
> > using 32
> > (XEN) ACPI:                  wakeup_vec[7f79df4c], vec_size[20]
> > (XEN) ACPI: Local APIC address 0xfee00000
> > (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
> > (XEN) Processor #0 6:10 APIC version 21
> > (XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled)
> > (XEN) Processor #2 6:10 APIC version 21
> > (XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x04] enabled)
> > (XEN) Processor #4 6:10 APIC version 21
> > (XEN) ACPI: LAPIC (acpi_id[0x04] lapic_id[0x06] enabled)
> > (XEN) Processor #6 6:10 APIC version 21
> > (XEN) ACPI: LAPIC (acpi_id[0x05] lapic_id[0x01] enabled)
> > (XEN) Processor #1 6:10 APIC version 21
> > (XEN) ACPI: LAPIC (acpi_id[0x06] lapic_id[0x03] enabled)
> > (XEN) Processor #3 6:10 APIC version 21
> > (XEN) ACPI: LAPIC (acpi_id[0x07] lapic_id[0x05] enabled)
> > (XEN) Processor #5 6:10 APIC version 21
> > (XEN) ACPI: LAPIC (acpi_id[0x08] lapic_id[0x07] enabled)
> > (XEN) Processor #7 6:10 APIC version 21
> > (XEN) ACPI: LAPIC_NMI (acpi_id[0xff] high edge lint[0x1])
> > (XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
> > (XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23
> > (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
> > (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
> > (XEN) ACPI: IRQ0 used by override.
> > (XEN) ACPI: IRQ2 used by override.
> > (XEN) ACPI: IRQ9 used by override.
> > (XEN) Enabling APIC mode:  Flat.  Using 1 I/O APICs
> > (XEN) ACPI: HPET id: 0x8086a701 base: 0xfed00000
> > (XEN) Table is not found!
> > (XEN) Using ACPI (MADT) for SMP configuration information
> > (XEN) IRQ limits: 24 GSI, 1528 MSI/MSI-X
> > (XEN) Early fatal page fault at e008:ff1da1c6 (cr2=ffffffb8, ec=0000)
> > (XEN) Stack dump: ff279610 00000080 ff24bfb0 fffff000 00000000 ff24bdd4
> > ffffc000 00000000
> > ff1da1c6 0000e008 00010082 00000000 00000000 ff24bdd4 ff130e89 ff21828a
> > 00000006 ffbfec64 ff279610 00bde0a0 00000000 ff24be04 ff1384f7 0007e95c
> > 00000002 00000000 ff11d4ef 00000096 00000096 f73de0a0 ff279610 ff279638
> > ff2795b8 ff24be34 ff13c740 ff279568 00000004 c9008020 00000000 ff21ac1c
> > ff2795e4 00000096 ff279568 ff21a7bc 00000008 ff24be54 ff13e038 ff2795b8
> > 00010970 02fa0000 ff279910 ff279910 00000007 ff24be74 ff144b0a ff279910
> > ff24be74 ff232570 00000007 00000000 00100800 ff24bfa4 ff234bf7 ff21dc0c
> > 00000080 00000008 00000000 00000000 00000000 00000000 00000000
> 00000000
> > 00000000 00000000 00000000 00000000 00000000 0007bef0 00000000
> 00000000
> > ff220260 00000000 00000000 1f000000 00000000 00000000 ff07bc50 ff07bfb0
> > ff07bef0 00757640 00848000 00000015 ffffffff 00000000 00000000 00000000
> > 00000000 00000000 00000000 7f7ff000 00000000 40000000 00000000 00000000
> > ff07bef0 00000000 00000000 00000000 00000000 00000007 00000007 00000008
> > 000001a4 000001a4 ff07bee0 01000000 00000000 ffd80000 00000000 ffd80000
> > 00000000 00000015 ffffffff ff07bf24 0000097d 00000000 00000000 00000000
> > 00000008 0000006e 00000001 00000003 000002f8 00000000 00000000
> 00220d70
> > ff2737c0 00067ebc ff10006c 0007bfb0 00000000 00000000 00000000 00000000
> > 00000000 00000000 00000000 00000000 00000000 00000000 00000000
> 00000000
> > 00000000 00000000 00000000 00000000 00000000 00000000 fffff000 00000000
> >
> >> -----Original Message-----
> >> From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
> >> [mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of Keir Fraser
> >> Sent: Friday, January 21, 2011 9:09 PM
> >> To: Zhang, Fengzhe; xen-devel@xxxxxxxxxxxxxxxxxxx
> >> Subject: Re: [Xen-devel] [PATCH]x86:x2apic: Disable x2apic on x86-32
> >> permanently
> >>
> >> Could you give more info on the bug, such as a backtrace? I wonder whether
> >> this might be better fixed in some more elegant way. Also I can't eyeball
> >> where map_domain_page() is being called during x2apic setup.
> >>
> >>  Thanks,
> >>  Keir
> >>
> >> On 21/01/2011 12:21, "Zhang, Fengzhe" <fengzhe.zhang@xxxxxxxxx> wrote:
> >>
> >>> x86:x2apic: Disable x2apic on x86-32 permanently
> >>>
> >>> x2apic initialization on x86_32 uses vcpu pointer before it is 
> >>> initialized.
> >>> As
> >>> x2apic is unlikely to be used on x86_32, this patch disables x2apic
> >>> permanently on x86_32. It also asserts the sanity of vcpu pointer before
> >>> dereference to prevent further misuse.
> >>>
> >>> Signed-off-by: Fengzhe Zhang <fengzhe.zhang@xxxxxxxxx>
> >>>
> >>> diff -r 02c0af2bf280 xen/arch/x86/apic.c
> >>> --- a/xen/arch/x86/apic.c Mon Jan 17 18:05:52 2011 +0000
> >>> +++ b/xen/arch/x86/apic.c Wed Jan 19 03:24:16 2011 -0500
> >>> @@ -961,6 +961,22 @@
> >>>      if ( !cpu_has_x2apic )
> >>>          return;
> >>>
> >>> +#ifdef __i386__
> >>> +    clear_bit(X86_FEATURE_X2APIC, boot_cpu_data.x86_capability);
> >>> +    if (x2apic_enabled) {
> >>> +        uint64_t msr_content;
> >>> +
> >>> +        rdmsrl(MSR_IA32_APICBASE, msr_content);
> >>> +        msr_content &= ~(MSR_IA32_APICBASE_ENABLE |
> >> MSR_IA32_APICBASE_EXTD);
> >>> +        wrmsrl(MSR_IA32_APICBASE, msr_content);
> >>> +        msr_content |= MSR_IA32_APICBASE_ENABLE;
> >>> +        wrmsrl(MSR_IA32_APICBASE, msr_content);
> >>> +        x2apic_enabled = 0;
> >>> +    }
> >>> +    printk("x2APIC disabled permanently on x86_32.\n");
> >>> +    return;
> >>> +#endif
> >>> +
> >>>      if ( !opt_x2apic )
> >>>      {
> >>>          if ( !x2apic_enabled )
> >>> diff -r 02c0af2bf280 xen/arch/x86/x86_32/domain_page.c
> >>> --- a/xen/arch/x86/x86_32/domain_page.c Mon Jan 17 18:05:52 2011 +0000
> >>> +++ b/xen/arch/x86/x86_32/domain_page.c Wed Jan 19 03:24:16 2011 -0500
> >>> @@ -53,6 +53,8 @@
> >>>      perfc_incr(map_domain_page_count);
> >>>
> >>>      v = mapcache_current_vcpu();
> >>> +    /* Prevent vcpu pointer being used before initialize. */
> >>> +    ASSERT((unsigned long)v != 0xfffff000);
> >>>
> >>>      dcache = &v->domain->arch.mapcache;
> >>>      vcache = &v->arch.mapcache;
> >>> _______________________________________________
> >>> Xen-devel mailing list
> >>> Xen-devel@xxxxxxxxxxxxxxxxxxx
> >>> http://lists.xensource.com/xen-devel
> >>
> >>
> >>
> >> _______________________________________________
> >> Xen-devel mailing list
> >> Xen-devel@xxxxxxxxxxxxxxxxxxx
> >> http://lists.xensource.com/xen-devel
> 


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel