Re: [Xen-users] Xen 3.4.2 DomU Network Issues

[Mark Pace <pace@xxxxxxxxxxxxxxxxxxx>]

This is all based on Centos 5.4.

This is all run atop CentOS 5.4

Kernel: 2.6.18-164.6.1.el5xen
Driver: module -- igb (came with CentOS)

I just peeked at my other Xen 3.4.1 installs and they are using the e1000 driver module.  There are different motherboards in those boxes, however, and CentOS 5.3.


pace

On Sun, 2009-11-22 at 18:31 +0100, Peter Braun wrote:
> Dom0 kernel?
>
> xen stock 2.6.18.8
> any other?
>
> What kind of Intel netwokr driver do you use?
>
> kernel - built in
> kernel - module
> igb - intel linux driver from sourceforge compiled as module?
>
>
>
> 2009/11/22 Mark Pace <pace@xxxxxxxxxxxxxxxxxxx>:
> > I've got a new system with Dual Quad Core Intel E5520 processors and Intel
> > 82576 NICs built on the motherboard (this is a Supermicro Nehalem based
> > blade).  I'm running Xen 3.4.2, but I've also tried 3.4.1 and had the same
> > problem.
> >
> > Dom0's network works flawlessly, but the DomU systems all have network
> > issues.  Whenever they transmit large packets, it is as if the DomU doesn't
> > recognize them arriving back.  You can watch them get to the DomU via
> > tcpdump, but the DomU programs doesn't bother to actually do anything about
> > them.  Sometimes after many retries it will be successful.  One of the
> > easiest ways to watch it fail is to tell curl to download a webpage.  If
> > that webpage has a relatively large response, curl will not get anything.
> > If you curl a small webpage, it will get it.
> >
> > I've tried disabling TCP offloading on the network cards and I've tried
> > cpuidle=0 cpufreq=none to no avail.  I've disabled everything fancy in the
> > BIOS except for the regular VT features (VT-d is disabled).
> >
> > Any help is greatly appreciated.
> >
> >
> > Thanks,
> > pace
> >
> > Here's the dmesg:
> >
> > __  __            _____ _  _    ____
> > \ \/ /___ _ __   |___ /| || |  |___ \
> >   \  // _ \ '_ \    |_ \| || |_   __) |
> >   /  \  __/ | | |  ___) |__   _| / __/
> > /_/\_\___|_| |_| |____(_) |_|(_)_____|
> >
> > (XEN) Xen version 3.4.2 (root@xxxxxxxxx) (gcc version 4.1.2 20080704 (Red
> > Hat 4.1.2-44)) Wed Nov 11 21:16:28 CET 2009
> > (XEN) Latest ChangeSet: unavailable
> > (XEN) Command line: dom0_mem=768M lowmem_emergency_pool=16M
> > (XEN) Video information:
> > (XEN)  VGA is text mode 80x25, font 8x16
> > (XEN)  VBE/DDC methods: none; EDID transfer time: 0 seconds
> > (XEN)  EDID info not retrieved because no DDC retrieval method detected
> > (XEN) Disc information:
> > (XEN)  Found 6 MBR signatures
> > (XEN)  Found 6 EDD information structures
> > (XEN) Xen-e820 RAM map:
> > (XEN)  0000000000000000 - 000000000009b400 (usable)
> > (XEN)  000000000009b400 - 00000000000a0000 (reserved)
> > (XEN)  00000000000e6000 - 0000000000100000 (reserved)
> > (XEN)  0000000000100000 - 00000000bf7b0000 (usable)
> > (XEN)  00000000bf7b0000 - 00000000bf7be000 (ACPI data)
> > (XEN)  00000000bf7be000 - 00000000bf7d0000 (ACPI NVS)
> > (XEN)  00000000bf7d0000 - 00000000bf7e0000 (reserved)
> > (XEN)  00000000bf7ec000 - 00000000c0000000 (reserved)
> > (XEN)  00000000fee00000 - 00000000fee01000 (reserved)
> > (XEN)  00000000ffc00000 - 0000000100000000 (reserved)
> > (XEN)  0000000100000000 - 0000000640000000 (usable)
> > (XEN) System RAM: 24567MB (25156908kB)
> > (XEN) ACPI: RSDP 000FADD0, 0024 (r2 ACPIAM)
> > (XEN) ACPI: XSDT BF7B0100, 004C (r1 073009 XSDT1518 20090730 MSFT       97)
> > (XEN) ACPI: FACP BF7B0290, 00F4 (r4 073009 FACP1518 20090730 MSFT       97)
> > (XEN) ACPI: DSDT BF7B04B0, 5776 (r2  1B8DT 1B8DT002        2 INTL 20051117)
> > (XEN) ACPI: FACS BF7BE000, 0040
> > (XEN) ACPI: APIC BF7B0390, 00D2 (r2 073009 APIC1518 20090730 MSFT       97)
> > (XEN) ACPI: MCFG BF7B0470, 003C (r1 073009 OEMMCFG  20090730 MSFT       97)
> > (XEN) ACPI: OEMB BF7BE040, 0072 (r1 073009 OEMB1518 20090730 MSFT       97)
> > (XEN) ACPI: SRAT BF7BA4B0, 01D0 (r1 073009 OEMSRAT         1 INTL        1)
> > (XEN) Domain heap initialised
> > (XEN) Processor #0 7:10 APIC version 21
> > (XEN) Processor #2 7:10 APIC version 21
> > (XEN) Processor #4 7:10 APIC version 21
> > (XEN) Processor #6 7:10 APIC version 21
> > (XEN) Processor #16 7:10 APIC version 21
> > (XEN) Processor #18 7:10 APIC version 21
> > (XEN) Processor #20 7:10 APIC version 21
> > (XEN) Processor #22 7:10 APIC version 21
> > (XEN) Processor #1 7:10 APIC version 21
> > (XEN) Processor #3 7:10 APIC version 21
> > (XEN) Processor #5 7:10 APIC version 21
> > (XEN) Processor #7 7:10 APIC version 21
> > (XEN) Processor #17 7:10 APIC version 21
> > (XEN) Processor #19 7:10 APIC version 21
> > (XEN) Processor #21 7:10 APIC version 21
> > (XEN) Processor #23 7:10 APIC version 21
> > (XEN) IOAPIC[0]: apic_id 8, version 32, address 0xfec00000, GSI 0-23
> > (XEN) Enabling APIC mode:  Phys.  Using 1 I/O APICs
> > (XEN) Using scheduler: SMP Credit Scheduler (credit)
> > (XEN) Detected 2261.051 MHz processor.
> > (XEN) VMX: Supported advanced features:
> > (XEN)  - APIC MMIO access virtualisation
> > (XEN)  - APIC TPR shadow
> > (XEN)  - Extended Page Tables (EPT)
> > (XEN)  - Virtual-Processor Identifiers (VPID)
> > (XEN)  - Virtual NMI
> > (XEN)  - MSR direct-access bitmap
> > (XEN) VMX: EPT is available.
> > (XEN) VMX: VPID is available.
> > (XEN) HVM: VMX enabled
> > (XEN) HVM: Hardware Assisted Paging detected.
> > (XEN) CPU0: Intel(R) Xeon(R) CPU           E5520  @ 2.27GHz stepping 05
> > (XEN) Booting processor 1/2 eip 8c000
> > (XEN) CPU1: Intel(R) Xeon(R) CPU           E5520  @ 2.27GHz stepping 05
> > (XEN) Booting processor 2/4 eip 8c000
> > (XEN) CPU2: Intel(R) Xeon(R) CPU           E5520  @ 2.27GHz stepping 05
> > (XEN) Booting processor 3/6 eip 8c000
> > (XEN) CPU3: Intel(R) Xeon(R) CPU           E5520  @ 2.27GHz stepping 05
> > (XEN) Booting processor 4/16 eip 8c000
> > (XEN) MCE: The hardware reports a non fatal, correctable incident occured on
> > CPU 0.
> > (XEN) Bank 8: ea1c6fc0008000b0[               0]
> > (XEN) CPU4: Intel(R) Xeon(R) CPU           E5520  @ 2.27GHz stepping 05
> > (XEN) Booting processor 5/18 eip 8c000
> > (XEN) CPU5: Intel(R) Xeon(R) CPU           E5520  @ 2.27GHz stepping 05
> > (XEN) Booting processor 6/20 eip 8c000
> > (XEN) CPU6: Intel(R) Xeon(R) CPU           E5520  @ 2.27GHz stepping 05
> > (XEN) Booting processor 7/22 eip 8c000
> > (XEN) CPU7: Intel(R) Xeon(R) CPU           E5520  @ 2.27GHz stepping 05
> > (XEN) Booting processor 8/1 eip 8c000
> > (XEN) CPU8: Intel(R) Xeon(R) CPU           E5520  @ 2.27GHz stepping 05
> > (XEN) Booting processor 9/3 eip 8c000
> > (XEN) CPU9: Intel(R) Xeon(R) CPU           E5520  @ 2.27GHz stepping 05
> > (XEN) Booting processor 10/5 eip 8c000
> > (XEN) CPU10: Intel(R) Xeon(R) CPU           E5520  @ 2.27GHz stepping 05
> > (XEN) Booting processor 11/7 eip 8c000
> > (XEN) CPU11: Intel(R) Xeon(R) CPU           E5520  @ 2.27GHz stepping 05
> > (XEN) Booting processor 12/17 eip 8c000
> > (XEN) CPU12: Intel(R) Xeon(R) CPU           E5520  @ 2.27GHz stepping 05
> > (XEN) Booting processor 13/19 eip 8c000
> > (XEN) CPU13: Intel(R) Xeon(R) CPU           E5520  @ 2.27GHz stepping 05
> > (XEN) Booting processor 14/21 eip 8c000
> > (XEN) CPU14: Intel(R) Xeon(R) CPU           E5520  @ 2.27GHz stepping 05
> > (XEN) Booting processor 15/23 eip 8c000
> > (XEN) CPU15: Intel(R) Xeon(R) CPU           E5520  @ 2.27GHz stepping 05
> > (XEN) Total of 16 processors activated.
> > (XEN) ENABLING IO-APIC IRQs
> > (XEN)  -> Using new ACK method
> > (XEN) checking TSC synchronization across 16 CPUs: passed.
> > (XEN) Platform timer is 3.579MHz ACPI PM Timer
> > (XEN) Brought up 16 CPUs
> > (XEN) I/O virtualisation disabled
> > (XEN) CPUIDLE: disabled due to no HPET. Force enable with 'cpuidle'.
> > (XEN) *** LOADING DOMAIN 0 ***
> > (XEN)  Xen  kernel: 64-bit, lsb, compat32
> > (XEN)  Dom0 kernel: 64-bit, lsb, paddr 0xffffffff80200000 ->
> > 0xffffffff8075550c
> > (XEN) PHYSICAL MEMORY ARRANGEMENT:
> > (XEN)  Dom0 alloc.:   000000062c000000->000000062e000000 (188416 pages to be
> > allocated)
> > (XEN) VIRTUAL MEMORY ARRANGEMENT:
> > (XEN)  Loaded kernel: ffffffff80200000->ffffffff8075550c
> > (XEN)  Init. ramdisk: ffffffff80756000->ffffffff8101f800
> > (XEN)  Phys-Mach map: ffffffff81020000->ffffffff811a0000
> > (XEN)  Start info:    ffffffff811a0000->ffffffff811a04b4
> > (XEN)  Page tables:   ffffffff811a1000->ffffffff811ae000
> > (XEN)  Boot stack:    ffffffff811ae000->ffffffff811af000
> > (XEN)  TOTAL:         ffffffff80000000->ffffffff81400000
> > (XEN)  ENTRY ADDRESS: ffffffff80200000
> > (XEN) Dom0 has maximum 16 VCPUs
> > (XEN) Scrubbing Free RAM:
> > ...........................................................................................................................................................................................................................................done.
> > (XEN) Xen trace buffers: disabled
> > (XEN) Std. Loglevel: Errors and warnings
> > (XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings)
> > (XEN) Xen is relinquishing VGA console.
> > (XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to
> > Xen)
> > (XEN) Freed 128kB init memory.
> >
> >
> > _______________________________________________
> > Xen-users mailing list
> > Xen-users@xxxxxxxxxxxxxxxxxxx
> > http://lists.xensource.com/xen-users
> >
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-users


_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users