Re: [Xen-devel] [PATCH 2/2 v2] graphics passthrough with VT-d

["Mr. Teo En Ming (Zhang Enming)" <enming.teo@xxxxxxxxxxxxxxx>]

Dear Weidong,

I checked out xen 3.5-unstable changeset 20145.

I applied xen-gfx-passthrough.patch to changeset 20145.

There were some errors with

Reversed (or previously applied) patch detected! Assume -R

So I think the maintainers of xen-unstable had applied some of the patches from xen-gfx-passthrough.patch you posted to the tree.

Then I applied the rest of the patches:

qemu-gfx-passthrough.patch
xen-load-vbios-file.patch
xen-vBAR-pBAR.patch
qemu-change-for-vBAR-pBAR.patch
qemu-claim-vga-cycle-for-secondary-gfx-passthrough.patch

to changeset 20145 without problems.

Here are the test results. Note that all of the test results are based on starting Windows XP Home HVM domU while in X server on dom0.

If I try to start domU in text mode on dom0 (without X server), then the screen output will be garbaged and I won't be able to type anything at all in dom0 text mode.

Test results with pvops dom 0 kernel 2.6.30-rc3
=============================

A. 1 VCPU
======

Tried starting win xp home domu with vcpu=1. DomU crashed and exited, leaving only dom0.

(XEN) HVM1: HVM Loader
(XEN) HVM1: Detected Xen v3.5-unstable
(XEN) HVM1: CPU speed is 2800 MHz
(XEN) irq.c:243: Dom1 PCI link 0 changed 0 -> 5
(XEN) HVM1: PCI-ISA link 0 routed to IRQ5
(XEN) irq.c:243: Dom1 PCI link 1 changed 0 -> 10
(XEN) HVM1: PCI-ISA link 1 routed to IRQ10
(XEN) irq.c:243: Dom1 PCI link 2 changed 0 -> 11
(XEN) HVM1: PCI-ISA link 2 routed to IRQ11
(XEN) irq.c:243: Dom1 PCI link 3 changed 0 -> 5
(XEN) HVM1: PCI-ISA link 3 routed to IRQ5
(XEN) HVM1: pci dev 01:2 INTD->IRQ5
(XEN) HVM1: pci dev 01:3 INTA->IRQ10
(XEN) HVM1: pci dev 02:0 INTA->IRQ11
(XEN) HVM1: pci dev 03:0 INTA->IRQ5
(XEN) HVM1: Make vBAR = pBAR of assigned gfx
(XEN) domctl.c:836:d0 memory_map:add: gfn=d2000 mfn=d2000 nr_mfns=1000
(XEN) domctl.c:836:d0 memory_map:add: gfn=c0000 mfn=c0000 nr_mfns=10000
(XEN) domctl.c:836:d0 memory_map:add: gfn=d0000 mfn=d0000 nr_mfns=2000
(XEN) domctl.c:887:d0 ioport_map:add f_gport=d000 f_mport=d000 np=80
(XEN) HVM1: pci dev 02:0 bar 14 size 01000000: e0000008
(XEN) HVM1: pci dev 02:0 bar 10 size 00000100: 0000c001
(XEN) HVM1: pci dev 03:0 bar 10 size 00000100: 0000c101
(XEN) HVM1: pci dev 03:0 bar 14 size 00000100: e1000000
(XEN) HVM1: pci dev 01:2 bar 20 size 00000020: 0000c201
(XEN) HVM1: pci dev 01:1 bar 20 size 00000010: 0000c221
(XEN) HVM1: Multiprocessor initialisation:
(XEN) HVM1:  - CPU0 ... 36-bit phys ... fixed MTRRs ... var MTRRs [3/8] ... done.
(XEN) HVM1: Testing HVM environment:
(XEN) HVM1:  - REP INSB across page boundaries ... passed
(XEN) HVM1:  - GS base MSRs and SWAPGS ... passed
(XEN) HVM1: Passed 2 of 2 tests
(XEN) HVM1: Writing SMBIOS tables ...
(XEN) HVM1: Loading ROMBIOS ...
(XEN) HVM1: 11900 bytes of ROMBIOS high-memory extensions:
(XEN) HVM1:   Relocating to 0xfc000000-0xfc002e7c ... done
(XEN) HVM1: Creating MP tables ...
(XEN) HVM1: Loading VGABIOS of passthroughed gfx ...
(XEN) HVM1: Loading PCI Option ROM ...
(XEN) HVM1:  - Manufacturer: http://etherboot.org
(XEN) HVM1:  - Product name: gPXE
(XEN) HVM1: Loading ACPI ...
(XEN) HVM1:  - Lo data: 000ea020-000ea04f
(XEN) HVM1:  - Hi data: fc003000-fc01261f
(XEN) HVM1: vm86 TSS at fc012800
(XEN) HVM1: BIOS map:
(XEN) HVM1:  c0000-cf7ff: VGA BIOS
(XEN) HVM1:  cf800-dbfff: Etherboot ROM
(XEN) HVM1:  eb000-eb14e: SMBIOS tables
(XEN) HVM1:  f0000-fffff: Main BIOS
(XEN) HVM1: Invoking ROMBIOS ...
(XEN) HVM1: $Revision: 1.221 $ $Date: 2008/12/07 17:32:29 $
(XEN) HVM1: *** int 15h function AX=5f14, BX=078f not yet supported!
(XEN) realmode.c:115:d1 Failed to emulate insn.
(XEN) realmode.c:165:d1 Real-mode emulation failed @ b4b4:0000b4b4: 00 00 00 00 00 00
(XEN) domain_crash called from realmode.c:166
(XEN) Domain 1 (vcpu#0) crashed on cpu#1:
(XEN) ----[ Xen-3.5-unstable  x86_64  debug=y  Not tainted ]----
(XEN) CPU:    1
(XEN) RIP:    c000:[<000000000000b4b4>]
(XEN) RFLAGS: 0000000000000082   CONTEXT: hvm guest
(XEN) rax: 000000000000b422   rbx: 000000000000b4b4   rcx: 0000000000010593
(XEN) rdx: 00000000000003d4   rsi: 0000000000009e48   rdi: 000000000000966a
(XEN) rbp: 000000000000a1a4   rsp: 0000000000009662   r8:  0000000000000000
(XEN) r9:  0000000000000000   r10: 0000000000000000   r11: 0000000000000000
(XEN) r12: 0000000000000000   r13: 0000000000000000   r14: 0000000000000000
(XEN) r15: 0000000000000000   cr0: 0000000000000010   cr4: 0000000000000000
(XEN) cr3: 0000000000800000   cr2: 0000000000000000
(XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: 0000   cs: c000
(XEN) [VT-D]iommu.c:1292:d0 domain_context_unmap:PCIe: bdf = 1:0.0
(XEN) [VT-D]iommu.c:1178:d0 domain_context_mapping:PCIe: bdf = 1:0.0

B. 2 VCPU
======

I switched to vcpu=2 for win xp home hvm domU. Now domU does not crash. xm list shows domU running. But vncviewer pops up a QEMU monitor. Looks like win xp home hvm domU did not start. All I get is a QEMU monitor.

(XEN) HVM2: HVM Loader
(XEN) HVM2: Detected Xen v3.5-unstable
(XEN) HVM2: CPU speed is 2800 MHz
(XEN) irq.c:243: Dom2 PCI link 0 changed 0 -> 5
(XEN) HVM2: PCI-ISA link 0 routed to IRQ5
(XEN) irq.c:243: Dom2 PCI link 1 changed 0 -> 10
(XEN) HVM2: PCI-ISA link 1 routed to IRQ10
(XEN) irq.c:243: Dom2 PCI link 2 changed 0 -> 11
(XEN) HVM2: PCI-ISA link 2 routed to IRQ11
(XEN) irq.c:243: Dom2 PCI link 3 changed 0 -> 5
(XEN) HVM2: PCI-ISA link 3 routed to IRQ5
(XEN) HVM2: pci dev 01:2 INTD->IRQ5
(XEN) HVM2: pci dev 01:3 INTA->IRQ10
(XEN) HVM2: pci dev 02:0 INTA->IRQ11
(XEN) HVM2: pci dev 03:0 INTA->IRQ5
(XEN) HVM2: Make vBAR = pBAR of assigned gfx
(XEN) domctl.c:836:d0 memory_map:add: gfn=d2000 mfn=d2000 nr_mfns=1000
(XEN) domctl.c:836:d0 memory_map:add: gfn=c0000 mfn=c0000 nr_mfns=10000
(XEN) domctl.c:836:d0 memory_map:add: gfn=d0000 mfn=d0000 nr_mfns=2000
(XEN) domctl.c:887:d0 ioport_map:add f_gport=d000 f_mport=d000 np=80
(XEN) HVM2: pci dev 02:0 bar 14 size 01000000: e0000008
(XEN) HVM2: pci dev 02:0 bar 10 size 00000100: 0000c001
(XEN) HVM2: pci dev 03:0 bar 10 size 00000100: 0000c101
(XEN) HVM2: pci dev 03:0 bar 14 size 00000100: e1000000
(XEN) HVM2: pci dev 01:2 bar 20 size 00000020: 0000c201
(XEN) HVM2: pci dev 01:1 bar 20 size 00000010: 0000c221
(XEN) HVM2: Multiprocessor initialisation:
(XEN) HVM2:  - CPU0 ... 36-bit phys ... fixed MTRRs ... var MTRRs [3/8] ... done.
(XEN) HVM2:  - CPU1 ... 36-bit phys ... fixed MTRRs ... var MTRRs [3/8] ... done.
(XEN) HVM2: Testing HVM environment:
(XEN) HVM2:  - REP INSB across page boundaries ... passed
(XEN) HVM2:  - GS base MSRs and SWAPGS ... passed
(XEN) HVM2: Passed 2 of 2 tests
(XEN) HVM2: Writing SMBIOS tables ...
(XEN) HVM2: Loading ROMBIOS ...
(XEN) HVM2: 11900 bytes of ROMBIOS high-memory extensions:
(XEN) HVM2:   Relocating to 0xfc000000-0xfc002e7c ... done
(XEN) HVM2: Creating MP tables ...
(XEN) HVM2: Loading VGABIOS of passthroughed gfx ...
(XEN) HVM2: Loading PCI Option ROM ...
(XEN) HVM2:  - Manufacturer: http://etherboot.org
(XEN) HVM2:  - Product name: gPXE
(XEN) HVM2: Loading ACPI ...
(XEN) HVM2:  - Lo data: 000ea020-000ea04f
(XEN) HVM2:  - Hi data: fc003000-fc01261f
(XEN) HVM2: vm86 TSS at fc012800
(XEN) HVM2: BIOS map:
(XEN) HVM2:  c0000-cf7ff: VGA BIOS
(XEN) HVM2:  cf800-dbfff: Etherboot ROM
(XEN) HVM2:  eb000-eb175: SMBIOS tables
(XEN) HVM2:  f0000-fffff: Main BIOS
(XEN) HVM2: Invoking ROMBIOS ...
(XEN) HVM2: $Revision: 1.221 $ $Date: 2008/12/07 17:32:29 $

The same thing happens when testing under pvops dom 0 kernel 2.6.31-rc6.

Test results with pvops dom 0 kernel 2.6.31-rc6
=============================

A. 1 VCPU
======

(XEN) HVM1: HVM Loader
(XEN) HVM1: Detected Xen v3.5-unstable
(XEN) HVM1: CPU speed is 2800 MHz
(XEN) irq.c:243: Dom1 PCI link 0 changed 0 -> 5
(XEN) HVM1: PCI-ISA link 0 routed to IRQ5
(XEN) irq.c:243: Dom1 PCI link 1 changed 0 -> 10
(XEN) HVM1: PCI-ISA link 1 routed to IRQ10
(XEN) irq.c:243: Dom1 PCI link 2 changed 0 -> 11
(XEN) HVM1: PCI-ISA link 2 routed to IRQ11
(XEN) irq.c:243: Dom1 PCI link 3 changed 0 -> 5
(XEN) HVM1: PCI-ISA link 3 routed to IRQ5
(XEN) HVM1: pci dev 01:2 INTD->IRQ5
(XEN) HVM1: pci dev 01:3 INTA->IRQ10
(XEN) HVM1: pci dev 02:0 INTA->IRQ11
(XEN) HVM1: pci dev 03:0 INTA->IRQ5
(XEN) HVM1: Make vBAR = pBAR of assigned gfx
(XEN) domctl.c:836:d0 memory_map:add: gfn=d2000 mfn=d2000 nr_mfns=1000
(XEN) domctl.c:836:d0 memory_map:add: gfn=c0000 mfn=c0000 nr_mfns=10000
(XEN) domctl.c:836:d0 memory_map:add: gfn=d0000 mfn=d0000 nr_mfns=2000
(XEN) domctl.c:887:d0 ioport_map:add f_gport=d000 f_mport=d000 np=80
(XEN) HVM1: pci dev 02:0 bar 14 size 01000000: e0000008
(XEN) HVM1: pci dev 02:0 bar 10 size 00000100: 0000c001
(XEN) HVM1: pci dev 03:0 bar 10 size 00000100: 0000c101
(XEN) HVM1: pci dev 03:0 bar 14 size 00000100: e1000000
(XEN) HVM1: pci dev 01:2 bar 20 size 00000020: 0000c201
(XEN) HVM1: pci dev 01:1 bar 20 size 00000010: 0000c221
(XEN) HVM1: Multiprocessor initialisation:
(XEN) HVM1:  - CPU0 ... 36-bit phys ... fixed MTRRs ... var MTRRs [3/8] ... done.
(XEN) HVM1: Testing HVM environment:
(XEN) HVM1:  - REP INSB across page boundaries ... passed
(XEN) HVM1:  - GS base MSRs and SWAPGS ... passed
(XEN) HVM1: Passed 2 of 2 tests
(XEN) HVM1: Writing SMBIOS tables ...
(XEN) HVM1: Loading ROMBIOS ...
(XEN) HVM1: 11900 bytes of ROMBIOS high-memory extensions:
(XEN) HVM1:   Relocating to 0xfc000000-0xfc002e7c ... done
(XEN) HVM1: Creating MP tables ...
(XEN) HVM1: Loading VGABIOS of passthroughed gfx ...
(XEN) HVM1: Loading PCI Option ROM ...
(XEN) HVM1:  - Manufacturer: http://etherboot.org
(XEN) HVM1:  - Product name: gPXE
(XEN) HVM1: Loading ACPI ...
(XEN) HVM1:  - Lo data: 000ea020-000ea04f
(XEN) HVM1:  - Hi data: fc003000-fc01261f
(XEN) HVM1: vm86 TSS at fc012800
(XEN) HVM1: BIOS map:
(XEN) HVM1:  c0000-cf7ff: VGA BIOS
(XEN) HVM1:  cf800-dbfff: Etherboot ROM
(XEN) HVM1:  eb000-eb14e: SMBIOS tables
(XEN) HVM1:  f0000-fffff: Main BIOS
(XEN) HVM1: Invoking ROMBIOS ...
(XEN) HVM1: $Revision: 1.221 $ $Date: 2008/12/07 17:32:29 $
(XEN) HVM1: *** int 15h function AX=5f14, BX=078f not yet supported!
(XEN) realmode.c:115:d1 Failed to emulate insn.
(XEN) realmode.c:165:d1 Real-mode emulation failed @ b4b4:0000b4b4: 00 00 00 00 00 00
(XEN) domain_crash called from realmode.c:166
(XEN) Domain 1 (vcpu#0) crashed on cpu#1:
(XEN) ----[ Xen-3.5-unstable  x86_64  debug=y  Not tainted ]----
(XEN) CPU:    1
(XEN) RIP:    c000:[<000000000000b4b4>]
(XEN) RFLAGS: 0000000000000086   CONTEXT: hvm guest
(XEN) rax: 000000000000b422   rbx: 000000000000b4b4   rcx: 0000000000021f0f
(XEN) rdx: 00000000000003d4   rsi: 0000000000009e48   rdi: 000000000000966a
(XEN) rbp: 000000000000d49c   rsp: 0000000000009662   r8:  0000000000000000
(XEN) r9:  0000000000000000   r10: 0000000000000000   r11: 0000000000000000
(XEN) r12: 0000000000000000   r13: 0000000000000000   r14: 0000000000000000
(XEN) r15: 0000000000000000   cr0: 0000000000000010   cr4: 0000000000000000
(XEN) cr3: 0000000000800000   cr2: 0000000000000000
(XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: 0000   cs: c000
(XEN) [VT-D]iommu.c:1292:d0 domain_context_unmap:PCIe: bdf = 1:0.0
(XEN) [VT-D]iommu.c:1178:d0 domain_context_mapping:PCIe: bdf = 1:0.0

B. 2 VCPU
======

(XEN) HVM2: HVM Loader
(XEN) HVM2: Detected Xen v3.5-unstable
(XEN) HVM2: CPU speed is 2800 MHz
(XEN) irq.c:243: Dom2 PCI link 0 changed 0 -> 5
(XEN) HVM2: PCI-ISA link 0 routed to IRQ5
(XEN) irq.c:243: Dom2 PCI link 1 changed 0 -> 10
(XEN) HVM2: PCI-ISA link 1 routed to IRQ10
(XEN) irq.c:243: Dom2 PCI link 2 changed 0 -> 11
(XEN) HVM2: PCI-ISA link 2 routed to IRQ11
(XEN) irq.c:243: Dom2 PCI link 3 changed 0 -> 5
(XEN) HVM2: PCI-ISA link 3 routed to IRQ5
(XEN) HVM2: pci dev 01:2 INTD->IRQ5
(XEN) HVM2: pci dev 01:3 INTA->IRQ10
(XEN) HVM2: pci dev 02:0 INTA->IRQ11
(XEN) HVM2: pci dev 03:0 INTA->IRQ5
(XEN) HVM2: Make vBAR = pBAR of assigned gfx
(XEN) domctl.c:836:d0 memory_map:add: gfn=d2000 mfn=d2000 nr_mfns=1000
(XEN) domctl.c:836:d0 memory_map:add: gfn=c0000 mfn=c0000 nr_mfns=10000
(XEN) domctl.c:836:d0 memory_map:add: gfn=d0000 mfn=d0000 nr_mfns=2000
(XEN) domctl.c:887:d0 ioport_map:add f_gport=d000 f_mport=d000 np=80
(XEN) HVM2: pci dev 02:0 bar 14 size 01000000: e0000008
(XEN) HVM2: pci dev 02:0 bar 10 size 00000100: 0000c001
(XEN) HVM2: pci dev 03:0 bar 10 size 00000100: 0000c101
(XEN) HVM2: pci dev 03:0 bar 14 size 00000100: e1000000
(XEN) HVM2: pci dev 01:2 bar 20 size 00000020: 0000c201
(XEN) HVM2: pci dev 01:1 bar 20 size 00000010: 0000c221
(XEN) HVM2: Multiprocessor initialisation:
(XEN) HVM2:  - CPU0 ... 36-bit phys ... fixed MTRRs ... var MTRRs [3/8] ... done.
(XEN) HVM2:  - CPU1 ... 36-bit phys ... fixed MTRRs ... var MTRRs [3/8] ... done.
(XEN) HVM2: Testing HVM environment:
(XEN) HVM2:  - REP INSB across page boundaries ... passed
(XEN) HVM2:  - GS base MSRs and SWAPGS ... passed
(XEN) HVM2: Passed 2 of 2 tests
(XEN) HVM2: Writing SMBIOS tables ...
(XEN) HVM2: Loading ROMBIOS ...
(XEN) HVM2: 11900 bytes of ROMBIOS high-memory extensions:
(XEN) HVM2:   Relocating to 0xfc000000-0xfc002e7c ... done
(XEN) HVM2: Creating MP tables ...
(XEN) HVM2: Loading VGABIOS of passthroughed gfx ...
(XEN) HVM2: Loading PCI Option ROM ...
(XEN) HVM2:  - Manufacturer: http://etherboot.org
(XEN) HVM2:  - Product name: gPXE
(XEN) HVM2: Loading ACPI ...
(XEN) HVM2:  - Lo data: 000ea020-000ea04f
(XEN) HVM2:  - Hi data: fc003000-fc01261f
(XEN) HVM2: vm86 TSS at fc012800
(XEN) HVM2: BIOS map:
(XEN) HVM2:  c0000-cf7ff: VGA BIOS
(XEN) HVM2:  cf800-dbfff: Etherboot ROM
(XEN) HVM2:  eb000-eb175: SMBIOS tables
(XEN) HVM2:  f0000-fffff: Main BIOS
(XEN) HVM2: Invoking ROMBIOS ...
(XEN) HVM2: $Revision: 1.221 $ $Date: 2008/12/07 17:32:29 $
(XEN) mce.c:694:d0 MCE: rdmsr MCG_CAP 0x806
(XEN) mce.c:694:d0 MCE: rdmsr MCG_CAP 0x806

Hope the test results helped.

I can start win xp home hvm dom U successfully if i remove pci in domU config and set gfx-passthru=0, basically starting with emulated vga bios instead of passing through pci-e graphics.

-- 
Mr. Teo En Ming (Zhang Enming) Dip(Mechatronics Engineering) BEng(Hons)(Mechanical Engineering) 
Technical Support Engineer 
Information Technology Department
Asiasoft Online Pte Ltd
Tampines Central 1 #04-01 Tampines Plaza 
Singapore 529541
Republic of Singapore
Company Website: http://www.asiasoft.sg/
Mobile: +65-9648-9798
MSN: teoenming@xxxxxxxxxxx
Alma Maters: Singapore Polytechnic, National University of Singapore

Teo En Ming (Zhang Enming) wrote:
  

Hi,

How about the patch for loading vga bios from .bin file and the 1:1
bar mapping in version 2 release?
    

Let's make the basic patches better and acceptable first. I haven't rebased
the hacks patches on version 2 patches. Currently pls use the hack patches
with version 1 patches. 

Regards,
Weidong

  

Thanks for the version 2.

Regards,

Mr. Teo En Ming (Zhang Enming) Dip(Mechatronics Engineering)
BEng(Hons)(Mechanical Engineering)
Technical Support Engineer
Information Technology Department
Asiasoft Online Pte Ltd
Tampines Central 1 #04-01 Tampines Plaza
Singapore 529541
Republic of Singapore
Mobile: +65-9648-9798
MSN: teoenming@xxxxxxxxxxx
-----Original Message-----
From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
[mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of Han,
Weidong Sent: Tuesday, September 01, 2009 2:01 PM
To: 'xen-devel@xxxxxxxxxxxxxxxxxxx'
Cc: Lin, Ben Y; 'bengheng@xxxxxxxxxxxxxx'; Kay,Allen M;
'Keir.Fraser@xxxxxxxxxxxxx'; 'Jean Guyader'
Subject: [Xen-devel] [PATCH 2/2 v2] graphics passthrough with VT-d

This patch supports basic gfx passthrough on QEMU:
  - disable emulated VGA adpater if there is passthroughed gfx
  - retrieve VGA bios from host 0xC0000, then load it to guest 0xC0000
  - register/unregister legacy VGA I/O ports and MMIOs for
passthroughed gfx 

Signed-off-by: Ben Lin <ben.y.lin@xxxxxxxxx>
Signed-off-by: Weidong Han <weidong.han@xxxxxxxxx>
No virus found in this incoming message.
Checked by AVG - www.avg.com
Version: 8.5.409 / Virus Database: 270.13.73/2338 - Release Date:
08/31/09 17:52:00

No virus found in this outgoing message.
Checked by AVG - www.avg.com
Version: 8.5.409 / Virus Database: 270.13.73/2338 - Release Date:
08/31/09 17:52:00
    

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel

No virus found in this incoming message.
Checked by AVG - www.avg.com 
Version: 8.5.409 / Virus Database: 270.13.73/2338 - Release Date: 08/31/09
17:52:00

No virus found in this outgoing message.
Checked by AVG - www.avg.com 
Version: 8.5.409 / Virus Database: 270.13.73/2338 - Release Date: 08/31/09
17:52:00

  

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel