WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-devel

RE: [Xen-devel] dma_alloc_coherent issue with tg3 in x86_64 build

To: "Chris Wright" <chrisw@xxxxxxxx>, <xen-devel@xxxxxxxxxxxxxxxxxxx>
Subject: RE: [Xen-devel] dma_alloc_coherent issue with tg3 in x86_64 build
From: "Ian Pratt" <m+Ian.Pratt@xxxxxxxxxxxx>
Date: Thu, 28 Jul 2005 21:59:22 +0100
Delivery-date: Thu, 28 Jul 2005 20:57:50 +0000
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
Thread-index: AcWTq7aEVqSZUhmbS8uLlSd5PKODeQACaUxg
Thread-topic: [Xen-devel] dma_alloc_coherent issue with tg3 in x86_64 build
> During boot of dom0, I get the following.  Box is remote, so 
> I'm now waiting for proxy power button pusher ;-)  tg3 init 
> fails doing dma_alloc_coherent (I assume this is from 
> tg3_init_one -> tg3_test_dma -> pci_alloc_consistent -> 
> dma_alloc_coherent).  Looks like an issue with the amount of 
> memory in dom0.  Is this known?

Is this with a debug=y build of Xen? I can see how that might provoke
the situation more than other builds.

Do you know how large a contiguous chunk the tg3 needs? 

I guess we could hold onto a few large chunks before making the dom0
memory allocation.

Alternatively, if we get a failure, we should round up to the next power
of two and then balloon out that much memory. Repeat until success...

Ian



> thanks,
> -chris
> 
> root (hd0,0)
>  Filesystem type is ext2fs, partition type 0x83 kernel 
> /xen.gz com1=115200,8n1 console=com1,vga
>    [Multiboot-elf, <0x100000:0x7c3e0:0x29c20>, 
> shtab=0x1a6078, entry=0x100000] module /vmlinuz-2.6.12-xen0 
> root=/dev/VolGroup00/LogVol00 ro console=tty0,ttyS0
>    [Multiboot-module @ 0x1a7000, 0x6b43f8 bytes] module 
> /initrd-generic-2.6.img
>    [Multiboot-module @ 0x85c000, 0x2ab400 bytes]
> 
>  __  __            _____  ___         _                _ 
>  \ \/ /___ _ __   |___ / / _ \     __| | _____   _____| |
>   \  // _ \ '_ \    |_ \| | | |__ / _` |/ _ \ \ / / _ \ |
>   /  \  __/ | | |  ___) | |_| |__| (_| |  __/\ V /  __/ |
>  /_/\_\___|_| |_| |____(_)___/    \__,_|\___| \_/ \___|_|
>                                                          
>  http://www.cl.cam.ac.uk/netos/xen
>  University of Cambridge Computer Laboratory
> 
>  Xen version 3.0-devel (chrisw@xxxxxxxxxxxx) (gcc version 
> 3.4.3 20041125 (Red Hat 3.4.3-7)) Thu Jul 28 10:54:44 PDT 
> 2005  Latest ChangeSet: Thu Jul 28 05:34:45 2005 
> 501a70f3ae968e46e27b9003febf05253f4cf949
> 
> (XEN) Physical RAM map:
> (XEN)  0000000000000000 - 000000000009dc00 (usable)
> (XEN)  000000000009dc00 - 00000000000a0000 (reserved)
> (XEN)  00000000000d0000 - 0000000000100000 (reserved)
> (XEN)  0000000000100000 - 000000007ff60000 (usable)
> (XEN)  000000007ff60000 - 000000007ff72000 (ACPI data)
> (XEN)  000000007ff72000 - 000000007ff80000 (ACPI NVS)
> (XEN)  000000007ff80000 - 0000000080000000 (reserved)
> (XEN)  00000000fec00000 - 00000000fec00400 (reserved)
> (XEN)  00000000fee00000 - 00000000fee01000 (reserved)
> (XEN)  00000000fff80000 - 0000000100000000 (reserved)
> (XEN) System RAM: 2046MB (2096116kB)
> (XEN) Xen heap: 14MB (14632kB)
> (XEN) found SMP MP-table at 000f7270
> (XEN) DMI present.
> (XEN) Using APIC driver default
> (XEN) ACPI: RSDP (v002 PTLTD                                 
> ) @ 0x00000000000f7210
> (XEN) ACPI: XSDT (v001 PTLTD           XSDT   0x06040000  LTP 
> 0x00000000) @ 0x000000007ff6d5a7
> (XEN) ACPI: FADT (v003 AMD    HAMMER   0x06040000 PTEC 
> 0x000f4240) @ 0x000000007ff71d0f
> (XEN) ACPI: SRAT (v001 AMD    HAMMER   0x06040000 AMD  
> 0x00000001) @ 0x000000007ff71e03
> (XEN) ACPI: MADT (v001 PTLTD           APIC   0x06040000  LTP 
> 0x00000000) @ 0x000000007ff71ecb
> (XEN) ACPI: ASF! (v016    MBI     CETP 0x06040000 PTL  
> 0x00000001) @ 0x000000007ff71f59
> (XEN) ACPI: DSDT (v001 AMD-K8  AMDACPI 0x06040000 MSFT 
> 0x0100000e) @ 0x0000000000000000
> (XEN) ACPI: Local APIC address 0xfee00000
> (XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
> (XEN) Processor #0 15:5 APIC version 16
> (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled)
> (XEN) Processor #1 15:5 APIC version 16
> (XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1])
> (XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
> (XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
> (XEN) IOAPIC[0]: apic_id 2, version 17, address 0xfec00000, GSI 0-23
> (XEN) ACPI: IOAPIC (id[0x03] address[0xc0000000] gsi_base[24])
> (XEN) IOAPIC[1]: apic_id 3, version 17, address 0xc0000000, GSI 24-27
> (XEN) ACPI: IOAPIC (id[0x04] address[0xc0001000] gsi_base[28])
> (XEN) IOAPIC[2]: apic_id 4, version 17, address 0xc0001000, GSI 28-31
> (XEN) ACPI: IOAPIC (id[0x05] address[0xc0600000] gsi_base[32])
> (XEN) IOAPIC[3]: apic_id 5, version 17, address 0xc0600000, GSI 32-35
> (XEN) ACPI: IOAPIC (id[0x06] address[0xc0601000] gsi_base[36])
> (XEN) IOAPIC[4]: apic_id 6, version 17, address 0xc0601000, GSI 36-39
> (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 high edge)
> (XEN) ACPI: IRQ0 used by override.
> (XEN) ACPI: IRQ2 used by override.
> (XEN) Enabling APIC mode:  Flat.  Using 5 I/O APICs
> (XEN) Using ACPI (MADT) for SMP configuration information
> (XEN) Initializing CPU#0
> (XEN) Detected 1994.377 MHz processor.
> (XEN) Using scheduler: Simple EDF Scheduler (sedf)
> (XEN) CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 
> bytes/line)
> (XEN) CPU: L2 Cache: 1024K (64 bytes/line)
> (XEN) CPU0: AMD Opteron(tm) Processor 246 stepping 08
> (XEN) Booting processor 1/1 eip 90000
> (XEN) Initializing CPU#1
> (XEN) CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 
> bytes/line)
> (XEN) CPU: L2 Cache: 1024K (64 bytes/line)
> (XEN) CPU1: AMD Opteron(tm) Processor 246 stepping 08
> (XEN) Total of 2 processors activated.
> (XEN) ENABLING IO-APIC IRQs
> (XEN) ..TIMER: vector=0x31 pin1=2 pin2=-1
> (XEN) checking TSC synchronization across 2 CPUs: passed.
> (XEN) Platform timer is 1.193MHz PIT
> (XEN) Brought up 2 CPUs
> (XEN) mtrr: v2.0 (20020519)
> (XEN) mtrr: your CPUs had inconsistent fixed MTRR settings
> (XEN) mtrr: probably your BIOS does not setup all CPUs.
> (XEN) mtrr: corrected configuration.
> (XEN) *** LOADING DOMAIN 0 ***
> (XEN) Xen-ELF header found: 
> 'GUEST_OS=linux,GUEST_VER=2.6,XEN_VER=3.0,VIRT_BASE=0xffffffff
80000000,LOADER=generic'
> (XEN) PHYSICAL MEMORY ARRANGEMENT:
> (XEN)  Dom0 alloc.:   0000000020000000->0000000040000000 
> (382799 pages to be allocated)
> (XEN) VIRTUAL MEMORY ARRANGEMENT:
> (XEN)  Loaded kernel: ffffffff80100000->ffffffff805ab086
> (XEN)  Init. ramdisk: ffffffff805ac000->ffffffff80857400
> (XEN)  Phys-Mach map: ffffffff80858000->ffffffff80a4dd3c
> (XEN)  Page tables:   ffffffff80a4e000->ffffffff80a57000
> (XEN)  Start info:    ffffffff80a57000->ffffffff80a58000
> (XEN)  Boot stack:    ffffffff80a58000->ffffffff80a59000
> (XEN)  TOTAL:         ffffffff80000000->ffffffff80c00000
> (XEN)  ENTRY ADDRESS: ffffffff80100000
> (XEN) Initrd len 0x2ab400, start at 0xffffffff805ac000
> (XEN) Scrubbing Free RAM: .....................done.
> (XEN) Xen trace buffers: initialised
> (XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to 
> switch input to Xen).
> Linux version 2.6.12.2-xen0 (chrisw@xxxxxxxxxxxxxxxxxx) (gcc 
> version 3.4.3 20041125 (Red Hat 3.4.3-7)) #6 Thu Jul 28 
> 10:53:46 PDT 2005 kernel direct mapping tables upto 
> ffff88007d74f000 @ c00000-fef000 Registering memory for 
> bootmem: from  1000000, size = 7c74f000
> ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
> ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled)
> ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1])
> ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
> ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
> IOAPIC[0]: apic_id 2, version 17, address 0xfec00000, GSI 0-23
> ACPI: IOAPIC (id[0x03] address[0xc0000000] gsi_base[24])
> IOAPIC[1]: apic_id 3, version 17, address 0xc0000000, GSI 24-27
> ACPI: IOAPIC (id[0x04] address[0xc0001000] gsi_base[28])
> IOAPIC[2]: apic_id 4, version 17, address 0xc0001000, GSI 28-31
> ACPI: IOAPIC (id[0x05] address[0xc0600000] gsi_base[32])
> IOAPIC[3]: apic_id 5, version 17, address 0xc0600000, GSI 32-35
> ACPI: IOAPIC (id[0x06] address[0xc0601000] gsi_base[36])
> IOAPIC[4]: apic_id 6, version 17, address 0xc0601000, GSI 36-39
> ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 high edge) 
> Setting APIC routing to flat Using ACPI (MADT) for SMP 
> configuration information Allocating PCI resources starting 
> at 7d800000 (gap: 7d74f000:828b1000) Built 1 zonelists Kernel 
> command line: root=/dev/VolGroup00/LogVol00 ro 
> console=tty0,ttyS0 Initializing CPU#0 PID hash table entries: 
> 4096 (order: 12, 131072 bytes) Xen reported: 1994.377 MHz processor.
> Console: colour VGA+ 80x25
> Dentry cache hash table entries: 262144 (order: 9, 2097152 
> bytes) Inode-cache hash table entries: 131072 (order: 8, 
> 1048576 bytes)
> Memory: 2007552k/2055484k available (2820k kernel code, 
> 31364k reserved, 1119k data, 272k init) Mount-cache hash 
> table entries: 256
> CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
> CPU: L2 Cache: 1024K (64 bytes/line)
> CPU: AMD Opteron(tm) Processor 246 stepping 08 checking if 
> image is initramfs... it is
> NET: Registered protocol family 16
> PCI: Using configuration type 1
> ACPI: Subsystem revision 20050309
> ACPI: Interpreter enabled
> ACPI: Using IOAPIC for interrupt routing
> ACPI: PCI Root Bridge [PCI0] (0000:00)
> PCI: Probing PCI hardware (bus 00)
> ACPI: PCI Interrupt Link [LNKA] (IRQs 3 *5 10 11)
> ACPI: PCI Interrupt Link [LNKB] (IRQs 3 5 10 *11)
> ACPI: PCI Interrupt Link [LNKC] (IRQs 3 5 *10 11)
> ACPI: PCI Interrupt Link [LNKD] (IRQs 3 5 10 *11)
> ACPI: PCI Root Bridge [PCI1] (0000:08)
> PCI: Probing PCI hardware (bus 08)
> xen_mem: Initialising balloon driver.
> SCSI subsystem initialized
> PCI: Using ACPI for IRQ routing
> PCI: If a device doesn't work, try "pci=routeirq".  If it 
> helps, post a report Grant table initialized
> IA-32 Microcode Update Driver: v1.14 <tigran@xxxxxxxxxxx>
> IA32 emulation $Id: sys_ia32.c,v 1.32 2002/03/24 13:02:28 ak 
> Exp $ Initializing Cryptographic API
> PCI: MSI quirk detected. pci_msi_quirk set.
> PCI: MSI quirk detected. pci_msi_quirk set.
> PCI: MSI quirk detected. pci_msi_quirk set.
> PCI: MSI quirk detected. pci_msi_quirk set.
> Real Time Clock Driver v1.12
> serio: i8042 AUX port at 0x60,0x64 irq 12
> serio: i8042 KBD port at 0x60,0x64 irq 1 io scheduler noop 
> registered io scheduler anticipatory registered io scheduler 
> deadline registered io scheduler cfq registered
> floppy0: no floppy controllers found
> RAMDISK driver initialized: 16 RAM disks of 16384K size 1024 blocksize
> loop: loaded (max 8 devices)
> HP CISS Driver (v 2.6.6)
> Intel(R) PRO/1000 Network Driver - version 6.0.54-k2 
> Copyright (c) 1999-2004 Intel Corporation.
> pcnet32.c:v1.30j 29.04.2005 tsbogend@xxxxxxxxxxxxxxxx
> e100: Intel(R) PRO/100 Network Driver, 3.4.8-k2-NAPI
> e100: Copyright(c) 1999-2005 Intel Corporation
> tg3.c:v3.31 (June 8, 2005)
> ACPI: PCI Interrupt 0000:03:02.0[A] -> GSI 28 (level, low) -> IRQ 28
> (XEN) (file=dom_mem_ops.c, line=61) Could not allocate a frame
> ----------- [cut here ] --------- [please bite here ] 
> --------- Kernel BUG at "arch/xen/x86_64/mm/hypervisor.c":299
> invalid operand: 0000 [1]
> CPU 0
> Modules linked in:
> Pid: 1, comm: swapper Not tainted 2.6.12.2-xen0
> RIP: e030:[<ffffffff8011a32a>] 
> <ffffffff8011a32a>{xen_contig_memory+586}
> RSP: e02b:ffff880002ee1bf8  EFLAGS: 00010296
> RAX: 0000000000000000 RBX: 000000000000000c RCX: ffffffff8011a324
> RDX: 0000000000000001 RSI: ffff880002ee1c28 RDI: 00000000ffffffff
> RBP: 0000000000000001 R08: 0000000000007ff0 R09: 0000000000000001
> R10: 0000000000000001 R11: 0000000000000246 R12: 0000000000000000
> R13: 0000000000000002 R14: 0000000000000001 R15: ffff800000000000
> FS:  0000000000000000(0000) GS:ffffffff80533a80(0000) 
> knlGS:0000000000000000
> CS:  e033 DS: 0000 ES: 0000
> Process swapper (pid: 1, threadinfo ffff880002ee0000, task 
> ffff880002edf410)
> Stack: 000000001002029a ffff880078cc6000 0000000202edf410 
> 0000000052aa4d54 
>        0000000000000000 ffff880078cc4000 0000000000044a89 
> 0000000000000001 
>        ffff880078cc4000 0000000000000000 Call 
> Trace:<ffffffff80114920>{dma_alloc_coherent+224} 
> <ffffffff802a8096>{tg3_init_one+7302}
>        <ffffffff803bdafa>{wait_for_completion+170} 
> <ffffffff80126dd0>{default_wake_function+0}
>        <ffffffff802212d6>{pci_device_probe+134} 
> <ffffffff8025ac0d>{driver_probe_device+77}
>        <ffffffff8025ad2b>{driver_attach+75} 
> <ffffffff8025b118>{bus_add_driver+152}
>        <ffffffff80220fe5>{pci_register_driver+117} 
> <ffffffff801090f2>{init+178}
>        <ffffffff8010e567>{child_rip+8} <ffffffff80109040>{init+0}
>        <ffffffff8010e55f>{child_rip+0} 
> 
> Code: 0f 0b 08 91 3d 80 ff ff ff ff 2b 01 b8 01 00 00 00 44 
> 89 f1 RIP <ffffffff8011a32a>{xen_contig_memory+586} RSP 
> <ffff880002ee1bf8>  <0>Kernel panic - not syncing: Attempted 
> to kill init!
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-devel
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel