[Xen-devel] Issues with the PCI Passthrough and swiotlb on xen 3.3.1

["Vasudevan, Swaminathan (PNB Roseville)" <swaminathan.vasudevan@xxxxxx>]

Hi Folks,

I have been trying to do a PCI Passthrough of one of my PCI device.

The Pass through works fine initially.

But I found that after couple of reboots the kernal crashes with the message below:

 

I am running xen 3.3.1 64-bit on CentOS 5.3.

My DomU is pv-kernel CentOS 5.3 (32-bit)

 

Fatal DMA error! Please use 'swiotlb=force'
------------[ cut here ]------------
kernel BUG at arch/i386/kernel/pci-dma-xen.c:395!
invalid opcode: 0000 [#1]
SMP
last sysfs file: /class/dahdi/dahdi!24/dev
Modules linked in: wanec(U) af_wanpipe(U) wanpipe(U) wanrouter(U) dahdi(U) crc_ccitt sdladrv(U) dm_mirror dm_multipath scsi_dh scsi_mod parport_pc lp parport xennet pcspkr dm_raid45 dm_message dm_region_hash dm_log dm_mod dm_mem_cache xenblk ext3 jbd uhci_hcd ohci_hcd ehci_hcd
CPU:    0
EIP:    0061:[<c040a37c>]    Tainted: G      VLI
EFLAGS: 00010086   (2.6.18-128.7.1.el5.centos.plusxen #1)
EIP is at dma_map_single+0x12d/0x147
eax: 0000002f   ebx: 00000000   ecx: 00000001   edx: f5416000
esi: 348d1010   edi: 00000001   ebp: 00000400   esp: df984db0
ds: 007b   es: 007b   ss: 0069
Process wanconfig (pid: 890, ti=df984000 task=c0322aa0 task.ti=df984000)
Stack: c06292b6 c0c55848 01548010 00000000 00000002 00000000 c0298f48 00000400
       e1109d99 00000002 00000000 00000400 00000400 c1548010 fffffff0 c0298000
       ddc10000 c0298f48 e1783d63 00000400 00000400 00000002 ddc10000 00002034
Call Trace:
 [<e1109d99>] sdla_busdma_map+0xdc/0x140 [sdladrv]
 [<e1783d63>] aft_dma_rx+0x131/0x551 [wanpipe]
 [<e1785243>] enable_data_error_intr+0x3c7/0x787 [wanpipe]
 [<e17858a1>] handle_front_end_state+0x124/0x1df [wanpipe]
 [<e178c683>] new_if+0x615/0x837 [wanpipe]
 [<e113a000>] wan_netif_fake_init+0x0/0x29 [wanrouter]
 [<c04589bb>] __kzalloc+0xd/0x34
 [<e113a710>] wan_device_new_if+0x101/0x1e3 [wanrouter]
 [<e113ab70>] wanrouter_ioctl+0x37e/0x887 [wanrouter]
 [<e109ae91>] ext3_file_write+0x19/0x83 [ext3]
 [<c046ed76>] do_sync_write+0xb6/0xf1
 [<c042fe57>] autoremove_wake_function+0x0/0x2d
 [<c047ef03>] do_ioctl+0x47/0x5d
 [<c047f163>] vfs_ioctl+0x24a/0x25c
 [<c047f1bd>] sys_ioctl+0x48/0x5f
 [<c0405413>] syscall_call+0x7/0xb
 =======================
Code: c0 74 07 8b 08 8b 58 04 eb 05 83 c9 ff 31 db 89 da 89 c8 f7 d2 f7 d0 21 fa 21 f0 89 d1 09 c1 74 13 68 b6 92 62 c0 e8 0e 5e 01 00 <0f> 0b 8b 01 97 92 62 c0 5d f0 83 04 24 00 83 c4 0c 89 f0 89 fa
EIP: [<c040a37c>] dma_map_single+0x12d/0x147 SS:ESP 0069:df984db0
 <0>Kernel panic - not syncing: Fatal exception

Then I read some of the postings and found that I need to include the "swiotlb=force" as a kernel boot option and I did.

I included "swiotlb=128" in the dom0 kernel boot and in the domU I included "swiotlb=force".

 

After setting this up, the clean domU came up fine, but again when the domU for some reason crashed and after that, when I try to boot to the domU I get this kernel panic as shown below:

 

------------[ cut here ]------------
kernel BUG at lib/../arch/i386/kernel/swiotlb.c:160!
invalid opcode: 0000 [#1]
SMP
last sysfs file:
Modules linked in:
CPU:    0
EIP:    e019:[<c04ebc14>]    Not tainted VLI
EFLAGS: 00010282   (2.6.18-128.7.1.el5.centos.plusxen #1)
EIP is at swiotlb_init_with_default_size+0x88/0x14e
eax: fffffff4   ebx: 00000c80   ecx: 00000000   edx: f5416000
esi: c071fc64   edi: 04000000   ebp: c06effc0   esp: c06effb4
ds: e021   es: e021   ss: e021
Process swapper (pid: 0, ti=c06ef000 task=c06762c0 task.ti=c06ef000)
Stack: c1417b64 c071fc64 c1415b64 00000020 c06ff84d 0000000e 00000002 c1417b64
       c071fc64 c1415b64 00000020 c06f486f 00004b64 c0765800 01020800 c0d6a000
       00000000 00000000 c040006f
Call Trace:
 [<c06ff84d>] mem_init+0xc/0x343
 [<c06f486f>] start_kernel+0x1f4/0x381
 =======================
Code: ea 3a f3 ff 83 c8 ff 40 83 f8 06 75 fa 89 d8 b9 1e 00 00 00 c1 e0 0b ba 06 00 00 00 03 05 34 dc 7a c0 e8 c6 c2 f2 ff 85 c0 74 08 <0f> 0b a0 00 87 af 63 c0 83 eb 80 a1 38 dc 7a c0 39 c3 72 c5 b9
EIP: [<c04ebc14>] swiotlb_init_with_default_size+0x88/0x14e SS:ESP e021:c06effb4
 <0>Kernel panic - not syncing: Fatal exception
 [root@cvanzante xen]#

 

Is there any permanent solution for this errors. Or what might have caused this problem.

 

If this problem has been addressed earlier by this community please someone provide me the link to this solution.

 

 

Thanks.

 

Swaminathan Vasudevan
Systems Software Engineer (TC)

 

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel