Xen project Mailing List

Re: [Xen-devel] Nested virtualization instabilities

To: Etzion Bar-Noy <etzion@xxxxxxxxxxxx>, "xen-devel@xxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxx>

From: "Zhang, Yang Z" <yang.z.zhang@xxxxxxxxx>

Date: Wed, 18 Dec 2013 01:33:09 +0000

Accept-language: en-US

Cc: "Dong, Eddie" <eddie.dong@xxxxxxxxx>, "Nakajima, Jun" <jun.nakajima@xxxxxxxxx>

Delivery-date: Wed, 18 Dec 2013 01:33:45 +0000

List-id: Xen developer discussion <xen-devel.lists.xen.org>

Thread-index: AQHO+k7dCbcDb2xoj0C75+o+sAjpt5pZJ8gg

Thread-topic: [Xen-devel] Nested virtualization instabilities

Etzion Bar-Noy wrote on 2013-12-16: > Hi. > Hi Etzion Bar-Noy It's glad to know someone is using Nested Xen and report bugs to us. Also, if there are more nested bugs you found, please CC me too. > > I am using the unstable build-from-source of Xen community. It reports > as 4.4, and was pulled from Git few days ago. The problem I will > describe below appears on version 4.3.1, compiled from source as well. > In both cases - I used spec file to create my own RPMs, because it's the > right method of doing it. > > > Problem: When VM running under HVM which is a hypervisor (XenServer > 6.2 or VMware ESXi5.5) attempts to start a (HVM) virtual machine, the > following happens, 100% reproducible on this server. > > XenServer: domain crash. XenServer restarts itself. It doesn't happen > for PV on XenServer VM > > VMware: An "unknown failure", which leads me to think that the nested > virtualization properties are not forwarded correctly to DomU To using nested, you need to add the set bit 31 of cupid.1.ecx to 0, add the following to your config file: cpuid = ['0x1:ecx=0xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'] For VMware ESX server, the attached patch is missing in current Xen which will caused the guest boot up fail. Also, I know some windows guests will boot fail on ESX. I never tried XenServer. But latest Xen on Xen is working well. So can you try the upsteam Xen to see whether it fails too? BTW, which L2 VM boot fail? > > > Just to clarify: > > Physical hardware: host > > Nested hypervisor: level 1 virtualization: XenServer or VMware ESXi > > Nested VM: level 2 virtualization: some generic VM inside XenServer or > ESXi (level 1) > > > This is the configuration used for XS: > name = "XenServer" > uuid = "a0b8c668-afaf-887a-1f4b-856dac4c8401" > maxmem = 4096 > memory = 4096 > vcpus = 2 > builder = "hvm" > boot = "c" > pae = 1 > acpi = 1 > apic = 1 > hap = 0 > viridian = 0 > rtc_timeoffset = 0 > localtime = 0 > on_poweroff = "destroy" > on_reboot = "restart" > on_crash = "restart" > sdl = 0 > vnc = 1 > vncunused = 1 > keymap = "en-us" > disk = [ "tap2:aio:/dev/share/VMS/xenserver01.lun,hda,w", > ",hdc:cdrom,r" ] vif = [ > "mac=00:16:3e:14:0d:72,bridge=lan,script=vif-bridge,model=e1000" ] parallel = > "none" > serial = "pty" > nestedhvm = 1 > > nx=1 > > > # A note - this problem appears with and without the 'nx' flag. > > A snippet from strace to qemu process: > > poll([{fd=30, events=POLLIN|POLLERR|POLLHUP}, {fd=18, > events=POLLIN|POLLERR|POLLHUP}, {fd=29, > events=POLLIN|POLLERR|POLLHUP}, {fd=21, > events=POLLIN|POLLERR|POLLHUP}, {fd=15, > events=POLLIN|POLLERR|POLLHUP}, {fd=5, events= > POLLIN|POLLERR|POLLHUP}, {fd=7, events=POLLIN}, {fd=11, > POLLIN|POLLERR|events=POLLIN}, {fd=28, events=POLLIN}, {fd=6, > POLLIN|POLLERR|events=POLLIN}], 10, 307) = ? > ERESTART_RESTARTBLOCK (To POLLIN|POLLERR|be restarted) --- SIGHUP > (Hangup) @ 0 (0) --- write(6, "\1\0\0\0\0\0\0\0", 8) = 8 > rt_sigreturn(0x2) = -1 EINTR (Interrupted system > call) gettimeofday({1386761621, 705266}, NULL) = 0 > clock_gettime(CLOCK_MONOTONIC, {1840, 693236049}) = 0 > clock_gettime(CLOCK_MONOTONIC, {1840, 693333096}) = 0 > gettimeofday({1386761621, 705546}, NULL) = 0 write(2, "qemu: terminating > on signal 1", 29) = 29 write(2, " from pid 6933\n", 15) = 15 > > > From the xl log for this domain: > Waiting for domain XenServer (domid 5) to die [pid 6933] Domain 5 has > shut down, reason code 3 0x3 Action for shutdown reason code 3 is > restart Domain > 5 needs to be cleaned up: destroying the domain Done. Rebooting now > Waiting for domain XenServer (domid 6) to die [pid 6933] > > > From 'xl dmesg' (XEN) <vm_launch_fail> error code 7 (XEN) > domain_crash_sync called from vmcs.c:1293 (XEN) Domain 8 (vcpu#0) > crashed on cpu#4: (XEN) ----[ Xen-4.4-unstable x86_64 debug=y Not > tainted ]---- (XEN) CPU: 4 (XEN) RIP: 0000:[<0000000000000000>] > (XEN) RFLAGS: 0000000000000002 CONTEXT: hvm guest (XEN) rax: > 0000000000000000 rbx: ffff830824f98000 rcx: ffff830824f9ff80 (XEN) > rdx: ffff82d0801d0cf9 rsi: 0000000000000000 rdi: ffff82d0801de7fc > (XEN) rbp: ffff82d080105ab1 rsp: 0000000000000000 r8: > 0000000000000004 (XEN) r9: ffff82d080105b20 r10: ffff830824f9ff70 > r11: 0000000000000000 (XEN) r12: ffff830824f9ff50 r13: > ffff830654f9e000 r14: ffff830824f9ff30 (XEN) r15: ffff82d080187e43 > cr0: 0000000000000039 cr4: 0000000000002050 (XEN) cr3: > 00000000feffa000 cr2: 0000000000000000 (XEN) ds: 0000 es: 0000 fs: > 0000 gs: 0000 ss: 0000 cs: 0000 (XEN) event_channel.c:270:d0 > EVTCHNOP failure: domain 9, error -22 (XEN) event_channel.c:270:d0 > EVTCHNOP failure: domain 9, error -22 (XEN) event_channel.c:270:d0 > EVTCHNOP failure: domain 9, error -22 (XEN) event_channel.c:270:d0 > EVTCHNOP failure: domain 9, error -22 (XEN) event_channel.c:270:d0 > EVTCHNOP failure: domain 9, error -22 (XEN) event_channel.c:270:d0 > EVTCHNOP failure: domain 9, error -22 (XEN) event_channel.c:270:d0 > EVTCHNOP failure: domain 9, error -22 (XEN) event_channel.c:270:d0 > EVTCHNOP failure: domain 9, error -22 (XEN) event_channel.c:270:d0 > EVTCHNOP failure: domain 9, error -22 (XEN) event_channel.c:270:d0 > EVTCHNOP failure: domain 9, error -22 (XEN) event_channel.c:270:d0 > EVTCHNOP failure: domain 9, error -22 (XEN) event_channel.c:270:d0 > EVTCHNOP failure: domain 9, error -22 (d9) HVM Loader (d9) Detected > Xen v4.4-unstable (d9) Xenbus rings @0xfeffc000, event channel 4 (d9) > System requested SeaBIOS (d9) CPU speed is 3400 MHz (d9) Relocating > guest memory for lowmem MMIO space disabled (XEN) irq.c:270: Dom9 PCI > link 0 changed 0 -> 5 (d9) PCI-ISA link 0 routed to IRQ5 (XEN) > irq.c:270: Dom9 PCI link 1 changed 0 -> 10 (d9) PCI-ISA link 1 routed > to IRQ10 (XEN) > irq.c:270: Dom9 PCI link 2 changed 0 -> 11 (d9) PCI-ISA link 2 routed > to > IRQ11 (XEN) irq.c:270: Dom9 PCI link 3 changed 0 -> 5 (d9) PCI-ISA > link > 3 routed to IRQ5 (d9) pci dev 01:3 INTA->IRQ10 (d9) pci dev 02:0 > INTA->IRQ11 (d9) pci dev 04:0 INTA->IRQ5 (d9) RAM in high memory; > setting high_mem resource base to 10f800000 (d9) pci dev 03:0 bar 10 > size 002000000: 0f0000008 (d9) pci dev 02:0 bar 14 size 001000000: > 0f2000008 (d9) pci dev 04:0 bar 30 size 000040000: 0f3000000 (d9) pci > dev 04:0 bar 10 size 000020000: 0f3040000 (d9) pci dev 03:0 bar 30 > size > 000010000: 0f3060000 (d9) pci dev 03:0 bar 14 size 000001000: > 0f3070000 > (d9) pci dev 02:0 bar 10 size 000000100: 00000c001 (d9) pci dev 04:0 > bar > 14 size 000000040: 00000c101 (d9) pci dev 01:1 bar 20 size 000000010: > 00000c141 (d9) Multiprocessor initialisation: (d9) - CPU0 ... 46-bit > phys ... fixed MTRRs ... var MTRRs [2/8] ... done. (d9) - CPU1 ... > 46-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done. (d9) Testing > HVM environment: (d9) - REP INSB across page boundaries ... passed > (d9) > - GS base MSRs and SWAPGS ... passed (d9) Passed 2 of 2 tests (d9) > Writing SMBIOS tables ... (d9) Loading SeaBIOS ... (d9) Creating MP > tables ... (d9) Loading ACPI ... (d9) vm86 TSS at fc00a080 (d9) BIOS > map: (d9) 10000-100d3: Scratch space (d9) e0000-fffff: Main BIOS > (d9) > E820 table: (d9) [00]: 00000000:00000000 - 00000000:000a0000: RAM > (d9) > HOLE: 00000000:000a0000 - 00000000:000e0000 (d9) [01]: > 00000000:000e0000 - 00000000:00100000: RESERVED (d9) [02]: > 00000000:00100000 - 00000000:f0000000: RAM (d9) HOLE: > 00000000:f0000000 > - 00000000:fc000000 (d9) [03]: 00000000:fc000000 - 00000001:00000000: > RESERVED (d9) [04]: 00000001:00000000 - 00000001:0f800000: RAM (d9) > Invoking SeaBIOS ... (d9) SeaBIOS (version > rel-1.7.3.1-0-g7d9cbe6-20131211_100730-compilevm) (d9) (d9) Found Xen > hypervisor signature at 40000000 (d9) xen: copy e820... (d9) > Relocating init from 0x000e2e88 to 0xeffe1d80 (size 57784) (d9) CPU > Mhz=3400 (d9) Found 7 PCI devices (max PCI bus is 00) (d9) Allocated > Xen hypercall page at effff000 (d9) Detected Xen v4.4-unstable (d9) > xen: copy BIOS tables... (d9) Copying SMBIOS entry point from > 0x00010010 to 0x000f1020 > (d9) Copying MPTABLE from 0xfc001170/fc001180 to 0x000f0f20 (d9) > Copying PIR from 0x00010030 to 0x000f0ea0 (d9) Copying ACPI RSDP from > 0x000100b0 to 0x000f0e70 (d9) Using pmtimer, ioport 0xb008, freq 3579 > kHz (d9) Scan for VGA option rom (d9) WARNING! Found unaligned PCI rom > (vd=1013:00b8) > (d9) Running option rom at c000:0003 (XEN) stdvga.c:147:d9 entering > stdvga and caching modes (d9) Turning on vga text mode console (d9) > SeaBIOS (version rel-1.7.3.1-0-g7d9cbe6-20131211_100730-compilevm) > (d9) Machine UUID a0b8c668-afaf-887a-1f4b-856dac4c8401 (d9) Found 0 > lpt ports > (d9) Found 1 serial ports (d9) ATA controller 1 at 1f0/3f4/c140 (irq > 14 dev 9) (d9) ATA controller 2 at 170/374/c148 (irq 15 dev 9) (d9) ata0-0: > QEMU HARDDISK ATA-7 Hard-Disk (14336 MiBytes) (d9) Searching bootorder > for: /pci@i0cf8/*@1,1/drive@0/disk@0 (d9) DVD/CD [ata1-0: QEMU DVD-ROM > ATAPI-4 DVD/CD] (d9) Searching bootorder for: > /pci@i0cf8/*@1,1/drive@1/disk@0 (d9) PS2 keyboard initialized (d9) All > threads complete. (d9) Scan for option roms (d9) Running option rom at > c900:0003 (d9) pmm call arg1=1 (d9) pmm call arg1=0 (d9) pmm call > arg1=1 > (d9) pmm call arg1=0 (d9) Searching bootorder for: /pci@i0cf8/*@4 (d9) > (d9) Press F12 for boot menu. (d9) (d9) Searching bootorder for: HALT > (d9) drive 0x000f0e20: PCHS=16383/16/63 translation=lba > LCHS=1024/255/63 > s=29360128 (d9) Space available for UMB: ca000-ee800, f0000-f0dc0 (d9) > Returned 61440 bytes of ZoneHigh (d9) e820 map has 7 items: (d9) 0: > 0000000000000000 - 000000000009fc00 = 1 RAM (d9) 1: 000000000009fc00 - > 00000000000a0000 = 2 RESERVED (d9) 2: 00000000000f0000 - > 0000000000100000 = 2 RESERVED (d9) 3: 0000000000100000 - > 00000000effff000 = 1 RAM (d9) 4: 00000000effff000 - 00000000f0000000 = > 2 RESERVED (d9) 5: 00000000fc000000 - 0000000100000000 = 2 RESERVED > (d9) 6: 0000000100000000 - 000000010f800000 = 1 RAM (d9) enter > handle_19: (d9) NULL (d9) Booting from Hard Disk... (d9) Booting from > 0000:7c00 [root@monster ~]# xl dmesg (XEN) <vm_launch_fail> error code > 7 > (XEN) domain_crash_sync called from vmcs.c:1293 (XEN) Domain 8 > (vcpu#0) crashed on cpu#4: (XEN) ----[ Xen-4.4-unstable x86_64 debug=y Not > tainted ]---- (XEN) CPU: 4 (XEN) RIP: 0000:[<0000000000000000>] > (XEN) RFLAGS: 0000000000000002 CONTEXT: hvm guest (XEN) rax: > 0000000000000000 rbx: ffff830824f98000 rcx: ffff830824f9ff80 (XEN) > rdx: ffff82d0801d0cf9 rsi: 0000000000000000 rdi: ffff82d0801de7fc > (XEN) rbp: ffff82d080105ab1 rsp: 0000000000000000 r8: > 0000000000000004 (XEN) r9: ffff82d080105b20 r10: ffff830824f9ff70 > r11: 0000000000000000 (XEN) r12: ffff830824f9ff50 r13: > ffff830654f9e000 r14: ffff830824f9ff30 (XEN) r15: ffff82d080187e43 > cr0: 0000000000000039 cr4: 0000000000002050 (XEN) cr3: > 00000000feffa000 cr2: 0000000000000000 (XEN) ds: 0000 es: 0000 fs: > 0000 gs: 0000 ss: 0000 cs: 0000 (XEN) event_channel.c:270:d0 > EVTCHNOP failure: domain 9, error -22 (XEN) event_channel.c:270:d0 > EVTCHNOP failure: domain 9, error -22 (XEN) event_channel.c:270:d0 > EVTCHNOP failure: domain 9, error -22 (XEN) event_channel.c:270:d0 > EVTCHNOP failure: domain 9, error -22 (XEN) event_channel.c:270:d0 > EVTCHNOP failure: domain 9, error -22 (XEN) event_channel.c:270:d0 > EVTCHNOP failure: domain 9, error -22 (XEN) event_channel.c:270:d0 > EVTCHNOP failure: domain 9, error -22 (XEN) event_channel.c:270:d0 > EVTCHNOP failure: domain 9, error -22 (XEN) event_channel.c:270:d0 > EVTCHNOP failure: domain 9, error -22 (XEN) event_channel.c:270:d0 > EVTCHNOP failure: domain 9, error -22 (XEN) event_channel.c:270:d0 > EVTCHNOP failure: domain 9, error -22 (XEN) event_channel.c:270:d0 > EVTCHNOP failure: domain 9, error -22 (d9) HVM Loader (d9) Detected > Xen v4.4-unstable (d9) Xenbus rings @0xfeffc000, event channel 4 (d9) > System requested SeaBIOS (d9) CPU speed is 3400 MHz (d9) Relocating > guest memory for lowmem MMIO space disabled (XEN) irq.c:270: Dom9 PCI > link 0 changed 0 -> 5 (d9) PCI-ISA link 0 routed to IRQ5 (XEN) > irq.c:270: Dom9 PCI link 1 changed 0 -> 10 (d9) PCI-ISA link 1 routed > to IRQ10 (XEN) > irq.c:270: Dom9 PCI link 2 changed 0 -> 11 (d9) PCI-ISA link 2 routed > to > IRQ11 (XEN) irq.c:270: Dom9 PCI link 3 changed 0 -> 5 (d9) PCI-ISA > link > 3 routed to IRQ5 (d9) pci dev 01:3 INTA->IRQ10 (d9) pci dev 02:0 > INTA->IRQ11 (d9) pci dev 04:0 INTA->IRQ5 (d9) RAM in high memory; > setting high_mem resource base to 10f800000 (d9) pci dev 03:0 bar 10 > size 002000000: 0f0000008 (d9) pci dev 02:0 bar 14 size 001000000: > 0f2000008 (d9) pci dev 04:0 bar 30 size 000040000: 0f3000000 (d9) pci > dev 04:0 bar 10 size 000020000: 0f3040000 (d9) pci dev 03:0 bar 30 > size > 000010000: 0f3060000 (d9) pci dev 03:0 bar 14 size 000001000: > 0f3070000 > (d9) pci dev 02:0 bar 10 size 000000100: 00000c001 (d9) pci dev 04:0 > bar > 14 size 000000040: 00000c101 (d9) pci dev 01:1 bar 20 size 000000010: > 00000c141 (d9) Multiprocessor initialisation: (d9) - CPU0 ... 46-bit > phys ... fixed MTRRs ... var MTRRs [2/8] ... done. (d9) - CPU1 ... > 46-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done. (d9) Testing > HVM environment: (d9) - REP INSB across page boundaries ... passed > (d9) > - GS base MSRs and SWAPGS ... passed (d9) Passed 2 of 2 tests (d9) > Writing SMBIOS tables ... (d9) Loading SeaBIOS ... (d9) Creating MP > tables ... (d9) Loading ACPI ... (d9) vm86 TSS at fc00a080 (d9) BIOS > map: (d9) 10000-100d3: Scratch space (d9) e0000-fffff: Main BIOS > (d9) > E820 table: (d9) [00]: 00000000:00000000 - 00000000:000a0000: RAM > (d9) > HOLE: 00000000:000a0000 - 00000000:000e0000 (d9) [01]: > 00000000:000e0000 - 00000000:00100000: RESERVED (d9) [02]: > 00000000:00100000 - 00000000:f0000000: RAM (d9) HOLE: > 00000000:f0000000 > - 00000000:fc000000 (d9) [03]: 00000000:fc000000 - 00000001:00000000: > RESERVED (d9) [04]: 00000001:00000000 - 00000001:0f800000: RAM (d9) > Invoking SeaBIOS ... (d9) SeaBIOS (version > rel-1.7.3.1-0-g7d9cbe6-20131211_100730-compilevm) (d9) (d9) Found Xen > hypervisor signature at 40000000 (d9) xen: copy e820... (d9) > Relocating init from 0x000e2e88 to 0xeffe1d80 (size 57784) (d9) CPU > Mhz=3400 (d9) Found 7 PCI devices (max PCI bus is 00) (d9) Allocated > Xen hypercall page at effff000 (d9) Detected Xen v4.4-unstable (d9) > xen: copy BIOS tables... (d9) Copying SMBIOS entry point from > 0x00010010 to 0x000f1020 > (d9) Copying MPTABLE from 0xfc001170/fc001180 to 0x000f0f20 (d9) > Copying PIR from 0x00010030 to 0x000f0ea0 (d9) Copying ACPI RSDP from > 0x000100b0 to 0x000f0e70 (d9) Using pmtimer, ioport 0xb008, freq 3579 > kHz (d9) Scan for VGA option rom (d9) WARNING! Found unaligned PCI rom > (vd=1013:00b8) > (d9) Running option rom at c000:0003 (XEN) stdvga.c:147:d9 entering > stdvga and caching modes (d9) Turning on vga text mode console (d9) > SeaBIOS (version rel-1.7.3.1-0-g7d9cbe6-20131211_100730-compilevm) > (d9) Machine UUID a0b8c668-afaf-887a-1f4b-856dac4c8401 (d9) Found 0 > lpt ports > (d9) Found 1 serial ports (d9) ATA controller 1 at 1f0/3f4/c140 (irq > 14 dev 9) (d9) ATA controller 2 at 170/374/c148 (irq 15 dev 9) (d9) ata0-0: > QEMU HARDDISK ATA-7 Hard-Disk (14336 MiBytes) (d9) Searching bootorder > for: /pci@i0cf8/*@1,1/drive@0/disk@0 (d9) DVD/CD [ata1-0: QEMU DVD-ROM > ATAPI-4 DVD/CD] (d9) Searching bootorder for: > /pci@i0cf8/*@1,1/drive@1/disk@0 (d9) PS2 keyboard initialized (d9) All > threads complete. (d9) Scan for option roms (d9) Running option rom at > c900:0003 (d9) pmm call arg1=1 (d9) pmm call arg1=0 (d9) pmm call > arg1=1 > (d9) pmm call arg1=0 (d9) Searching bootorder for: /pci@i0cf8/*@4 (d9) > (d9) Press F12 for boot menu. (d9) (d9) Searching bootorder for: HALT > (d9) drive 0x000f0e20: PCHS=16383/16/63 translation=lba > LCHS=1024/255/63 > s=29360128 (d9) Space available for UMB: ca000-ee800, f0000-f0dc0 (d9) > Returned 61440 bytes of ZoneHigh (d9) e820 map has 7 items: (d9) 0: > 0000000000000000 - 000000000009fc00 = 1 RAM (d9) 1: 000000000009fc00 - > 00000000000a0000 = 2 RESERVED (d9) 2: 00000000000f0000 - > 0000000000100000 = 2 RESERVED (d9) 3: 0000000000100000 - > 00000000effff000 = 1 RAM (d9) 4: 00000000effff000 - 00000000f0000000 = > 2 RESERVED (d9) 5: 00000000fc000000 - 0000000100000000 = 2 RESERVED > (d9) 6: 0000000100000000 - 000000010f800000 = 1 RAM (d9) enter > handle_19: (d9) NULL (d9) Booting from Hard Disk... (d9) Booting from > 0000:7c00 > > > System description: > > 1. Centos 6.4 x86_64, latest updates > > 2. Xen4centos packages - current kernel is > 3.10.23-11.el6.centos.alt.x86_64 > > 3. System with 32GB RAM > > 4. Virtual disks are ZFS volumes (aka - block devices). Same problem > appeared when on local ext3 disks > > 5. Using OpenVSwitch version 2.0, compiled to RPM, again. > > 6. Intel CPU: > processor : 0 > vendor_id : GenuineIntel > cpu family : 6 > model : 62 > model name : Intel(R) Core(TM) i7-4930K CPU @ 3.40GHz > stepping : 4 > microcode : 0x416 > cpu MHz : 3400.158 > cache size : 12288 KB > physical id : 0 > siblings : 12 > core id : 0 > cpu cores : 6 > apicid : 0 > initial apicid : 0 > fpu : yes > fpu_exception : yes > cpuid level : 13 > wp : yes > flags : fpu de tsc msr pae mce cx8 apic sep mca cmov pat clflush acpi > mmx fxsr sse sse2 ss ht syscall nx lm constant_tsc rep_good nopl > nonstop_tsc eagerfpu pni pclmulqdq monitor est ssse3 cx16 sse4_1 > sse4_2 popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor > lahf_lm ida arat epb xsaveopt pln pts dtherm fsgsbase erms > bogomips : 6800.31 > clflush size : 64 > cache_alignment : 64 > address sizes : 46 bits physical, 48 bits virtual > power management: > > > It was suggested by Mr. Ian Campbell to CC both Mr. Jun Nakajima and Mr. > Eddie Dong, so, my apologies if this action is improper. > > > Any tip or advice would be great. Any request for further info would > be also great! > > Thanks! > > Etzion Bar-Noy Best regards, Yang

_______________________________________________ Xen-devel mailing list Xen-devel@xxxxxxxxxxxxx http://lists.xen.org/xen-devel

©2013 Xen Project, A Linux Foundation Collaborative Project. All Rights Reserved.
Linux Foundation is a registered trademark of The Linux Foundation.
Xen Project is a trademark of The Linux Foundation.