Hi list!
I would like to access my InfiniBand HCA from a domU, so I first tried
the smart-io.hg Xen tree but it did not build right away. As a temporary
workaround I tried to let the domU directly access the HCA. The steps I
have taken have been:
1. In dom0:
# lspci -v
03:00.0 InfiniBand: Mellanox Technologies MT25204 [InfiniHost III Lx
HCA] (rev a0)
Subsystem: Mellanox Technologies MT25204 [InfiniHost III Lx HCA]
Flags: fast devsel, IRQ 16
Memory at f0800000 (64-bit, non-prefetchable) [size=1M]
Memory at f0000000 (64-bit, prefetchable) [size=8M]
Capabilities: [40] Power Management version 2
Capabilities: [48] Vital Product Data
Capabilities: [90] Message Signalled Interrupts: Mask- 64bit+
Queue=0/5 Enable-
Capabilities: [84] MSI-X: Enable- Mask- TabSize=32
Capabilities: [60] Express Endpoint IRQ 0
With that information I edited my grub config to exclude this particular
PCI device:
module /boot/vmlinuz-2.6.19-4-server root=/dev/sda1 ro console=tty0
console=ttyS0,9600n8 max_loop=64 pciback.hide=(0000:03:00.0)
2. After a reboot, I edited the domU config so that it includes the line:
pci = ['0000:03:00.0']
The output of lspci -v executed in the user domain is:
0000:00:00.0 InfiniBand: Mellanox Technology: Unknown device 6274 (rev a0)
Subsystem: Mellanox Technology: Unknown device 6274
Flags: fast devsel, IRQ 16
Memory at f0800000 (64-bit, non-prefetchable) [size=1M]
Memory at f0000000 (64-bit, prefetchable) [size=8M]
Capabilities: [40] Power Management version 2
Capabilities: [48] Vital Product Data
Capabilities: [90] Message Signalled Interrupts: 64bit+
Queue=0/5 Enable-
Capabilities: [84] #11 [001f]
Capabilities: [60] #10 [0001]
The module cannot be loaded and prints the following error:
xen-domU0:~# modprobe ib_mthca
[90558.241638] ib_mthca: Mellanox InfiniBand HCA driver v0.08 (February
14, 2006)
[90558.241651] ib_mthca: Initializing 0000:00:00.0
[90559.245730] ib_mthca 0000:00:00.0: QUERY_FW command failed, aborting.
[90559.245792] ib_mthca: probe of 0000:00:00.0 failed with error -11
The output of 'xm info' is:
host : xen-o-matic
release : 2.6.19-4-server
version : #2 SMP Thu Apr 5 06:07:34 UTC 2007
machine : i686
nr_cpus : 2
nr_nodes : 1
sockets_per_node : 1
cores_per_socket : 2
threads_per_core : 1
cpu_mhz : 2009
hw_caps :
178bfbff:ebd3fbff:00000000:00000010:00002001:00000000:0000001f
total_memory : 2030
free_memory : 1367
xen_major : 3
xen_minor : 0
xen_extra : .3-0
xen_caps : xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p
xen_pagesize : 4096
platform_params : virt_start=0xf5800000
xen_changeset : unavailable
cc_compiler : gcc version 4.1.2 (Ubuntu 4.1.2-0ubuntu4)
cc_compile_by : buildd
cc_compile_domain : buildd
cc_compile_date : Sat Mar 24 16:01:08 UTC 2007
xend_config_format : 2
Does anybody have an idea?
Best regards,
Alexander Petry
--
===========================================
| Dipl. Inf. Alexander Petry |
| Head Competence Center for HPC |
| Institut fuer Techno- und |
| Wirtschaftsmathematik (ITWM) |
| Fraunhofer-Platz 1 |
| |
| D-67663 Kaiserslautern |
===========================================
| E-Mail: petry@xxxxxxxxxxx |
| Internet: http://www.itwm.fhg.de |
===========================================
_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users
| 
Home