WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-users

Re: [Xen-users] 3ware pbs : tw_cli strace -> Out of memory: swiotlb?

To: CHERAMY Guillaume <guillaume.cheramy@xxxxxxxxx>, XEN Users <xen-users@xxxxxxxxxxxxxxxxxxx>
Subject: Re: [Xen-users] 3ware pbs : tw_cli strace -> Out of memory: swiotlb?
From: Chris Joelly <chris-m-lists@xxxxxxxxxx>
Date: Mon, 4 Aug 2008 13:59:41 +0200
Cc:
Delivery-date: Mon, 04 Aug 2008 05:00:49 -0700
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <48932B6B.6070603@xxxxxxxxx>
List-help: <mailto:xen-users-request@lists.xensource.com?subject=help>
List-id: Xen user discussion <xen-users.lists.xensource.com>
List-post: <mailto:xen-users@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=unsubscribe>
References: <48932B6B.6070603@xxxxxxxxx>
Sender: xen-users-bounces@xxxxxxxxxxxxxxxxxxx
User-agent: Mutt/1.5.18 (2008-05-17)
Hello,

last night i experienced a very strange behavior on a newly installed box 
with a 9650 controller. sometimes the box hangs, and sometimes the tw_cli 
isn't able to find the controller. in syslog there are lots of out of memory
errors freezing the whole machine. amazing is that we have two identical 
boxes and it only happens on one box frequently.

we tried to set swiotlb=96 to the kernel module parameter in grub config as 
i found this setting with DMA and io memory related problems one user 
experienced, and the problem doesn't occur that often now. but the machine 
is still unusable in an production environment.

i read your elder post and maybe we have the same problem? 

The box also throws "Out of SW-IOMMU space ...on PIC devices" messages to 
the console, thats why i googled and found this swiotlb param. The PCI 
devices mentioned in the errors are:

03:00.0 RAID bus controller: 3ware Inc 9650SE SATA-II RAID (rev 01)
08:00.0 SCSI storage controller: LSI Logic / Symbios Logic SAS1064ET 
PCI-Express Fusion-MPT SAS (rev 04)

hope to find out why the 3ware controller isn't working with xen or what
the setting swiotlb is calculated end it it does the trick ...

Chris

On Fre, Aug 01, 2008, CHERAMY Guillaume wrote:
> Hello,
> 
>     when I done a strace on tw_cli on my dom0 I have got some ENOMEM
> (Cannot allocate memory) ...
> 
> Why ?
> 
> execve("/usr/sbin/tw_cli", ["tw_cli", "info"], [/* 17 vars */]) = 0
> uname({sys="Linux", node="geppo", ...}) = 0
> brk(0)                                  = 0x827d000
> brk(0x829e000)                          = 0x829e000
> uname({sys="Linux", node="geppo", ...}) = 0
> open("/proc/devices", O_RDONLY)         = 3
> fstat64(3, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
> old_mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS,
> -1, 0) = 0xb7f95000
> read(3, "Character devices:\n  1 mem\n  2 p"..., 1024) = 411
> read(3, "", 1024)                       = 0
> close(3)                                = 0
> munmap(0xb7f95000, 4096)                = 0
> open("/proc/devices", O_RDONLY)         = 3
> fstat64(3, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
> old_mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS,
> -1, 0) = 0xb7f95000
> read(3, "Character devices:\n  1 mem\n  2 p"..., 1024) = 411
> close(3)                                = 0
> munmap(0xb7f95000, 4096)                = 0
> stat64("/dev/twa0", {st_mode=S_IFCHR|0600, st_rdev=makedev(252, 0),
> ...}) = 0
> stat64("/dev/twa1", {st_mode=S_IFCHR|0600, st_rdev=makedev(252, 1),
> ...}) = 0
> stat64("/dev/twa2", {st_mode=S_IFCHR|0600, st_rdev=makedev(252, 2),
> ...}) = 0
> stat64("/dev/twa3", {st_mode=S_IFCHR|0600, st_rdev=makedev(252, 3),
> ...}) = 0
> stat64("/dev/twa4", {st_mode=S_IFCHR|0600, st_rdev=makedev(252, 4),
> ...}) = 0
> stat64("/dev/twa5", {st_mode=S_IFCHR|0600, st_rdev=makedev(252, 5),
> ...}) = 0
> stat64("/dev/twa6", {st_mode=S_IFCHR|0600, st_rdev=makedev(252, 6),
> ...}) = 0
> stat64("/dev/twa7", {st_mode=S_IFCHR|0600, st_rdev=makedev(252, 7),
> ...}) = 0
> stat64("/dev/twa8", {st_mode=S_IFCHR|0600, st_rdev=makedev(252, 8),
> ...}) = 0
> stat64("/dev/twa9", {st_mode=S_IFCHR|0600, st_rdev=makedev(252, 9),
> ...}) = 0
> stat64("/dev/twa10", {st_mode=S_IFCHR|0600, st_rdev=makedev(252, 10),
> ...}) = 0
> stat64("/dev/twa11", {st_mode=S_IFCHR|0600, st_rdev=makedev(252, 11),
> ...}) = 0
> stat64("/dev/twa12", {st_mode=S_IFCHR|0600, st_rdev=makedev(252, 12),
> ...}) = 0
> stat64("/dev/twa13", {st_mode=S_IFCHR|0600, st_rdev=makedev(252, 13),
> ...}) = 0
> stat64("/dev/twa14", {st_mode=S_IFCHR|0600, st_rdev=makedev(252, 14),
> ...}) = 0
> stat64("/dev/twa15", {st_mode=S_IFCHR|0600, st_rdev=makedev(252, 15),
> ...}) = 0
> open("/dev/null", O_RDONLY|O_NONBLOCK|O_DIRECTORY) = -1 ENOTDIR (Not a
> directory)
> open("/proc/scsi/3w-9xxx", O_RDONLY|O_NONBLOCK|O_LARGEFILE|O_DIRECTORY)
> = -1 ENOENT (No such file or directory)
> open("/sys/class/scsi_host",
> O_RDONLY|O_NONBLOCK|O_LARGEFILE|O_DIRECTORY) = 3
> fstat64(3, {st_mode=S_IFDIR|0755, st_size=0, ...}) = 0
> fcntl64(3, F_SETFD, FD_CLOEXEC)         = 0
> getdents64(3, /* 3 entries */, 4096)    = 80
> stat64("/sys/class/scsi_host/host0/stats", {st_mode=S_IFREG|0444,
> st_size=4096, ...}) = 0
> open("/sys/class/scsi_host/host0/stats", O_RDONLY) = 4
> read(4, "3w-9xxx Driver v", 16)         = 16
> close(4)                                = 0
> open("/dev/twa15", O_RDWR)              = -1 ENODEV (No such device)
> open("/dev/twa14", O_RDWR)              = -1 ENODEV (No such device)
> open("/dev/twa13", O_RDWR)              = -1 ENODEV (No such device)
> open("/dev/twa12", O_RDWR)              = -1 ENODEV (No such device)
> open("/dev/twa11", O_RDWR)              = -1 ENODEV (No such device)
> open("/dev/twa10", O_RDWR)              = -1 ENODEV (No such device)
> open("/dev/twa9", O_RDWR)               = -1 ENODEV (No such device)
> open("/dev/twa8", O_RDWR)               = -1 ENODEV (No such device)
> open("/dev/twa7", O_RDWR)               = -1 ENODEV (No such device)
> open("/dev/twa6", O_RDWR)               = -1 ENODEV (No such device)
> open("/dev/twa5", O_RDWR)               = -1 ENODEV (No such device)
> open("/dev/twa4", O_RDWR)               = -1 ENODEV (No such device)
> open("/dev/twa3", O_RDWR)               = -1 ENODEV (No such device)
> open("/dev/twa2", O_RDWR)               = -1 ENODEV (No such device)
> open("/dev/twa1", O_RDWR)               = -1 ENODEV (No such device)
> open("/dev/twa0", O_RDWR)               = 4
> close(4)                                = 0
> getdents64(3, /* 0 entries */, 4096)    = 0
> close(3)                                = 0
> uname({sys="Linux", node="geppo", ...}) = 0
> uname({sys="Linux", node="geppo", ...}) = 0
> getuid32()                              = 0
> open("/dev/twa0", O_RDWR)               = 3
> uname({sys="Linux", node="geppo", ...}) = 0
> ioctl(3, 0x108, 0x827dc28)              = -1 ENOMEM (Cannot allocate memory)
> nanosleep({0, 1000000}, NULL)           = 0
> ioctl(3, 0x108, 0x827dc28)              = -1 ENOMEM (Cannot allocate memory)
> nanosleep({0, 1000000}, NULL)           = 0
> ioctl(3, 0x108, 0x827dc28)              = -1 ENOMEM (Cannot allocate memory)
> nanosleep({0, 1000000}, NULL)           = 0
> ioctl(3, 0x108, 0x827dc28)              = -1 ENOMEM (Cannot allocate memory)
> nanosleep({0, 1000000}, NULL)           = 0
> ioctl(3, 0x108, 0x827dc28)              = -1 ENOMEM (Cannot allocate memory)
> nanosleep({0, 1000000}, NULL)           = 0
> ioctl(3, 0x108, 0x827dc28)              = -1 ENOMEM (Cannot allocate memory)
> nanosleep({0, 1000000}, NULL)           = 0
> ioctl(3, 0x108, 0x827dc28)              = -1 ENOMEM (Cannot allocate memory)
> nanosleep({0, 1000000}, NULL)           = 0
> ioctl(3, 0x108, 0x827dc28)              = -1 ENOMEM (Cannot allocate memory)
> nanosleep({0, 1000000}, NULL)           = 0
> ioctl(3, 0x108, 0x827dc28)              = -1 ENOMEM (Cannot allocate memory)
> nanosleep({0, 1000000}, NULL)           = 0
> ioctl(3, 0x108, 0x827dc28)              = -1 ENOMEM (Cannot allocate memory)
> nanosleep({0, 1000000}, NULL)           = 0
> ioctl(3, 0x108, 0x827dc28)              = -1 ENOMEM (Cannot allocate memory)
> nanosleep({0, 1000000}, NULL)           = 0
> ioctl(3, 0x108, 0x827dc28)              = -1 ENOMEM (Cannot allocate memory)
> nanosleep({0, 1000000}, NULL)           = 0
> ioctl(3, 0x108, 0x827dc28)              = -1 ENOMEM (Cannot allocate memory)
> nanosleep({0, 1000000}, NULL)           = 0
> ioctl(3, 0x108, 0x827dc28)              = -1 ENOMEM (Cannot allocate memory)
> nanosleep({0, 1000000}, NULL)           = 0
> ioctl(3, 0x108, 0x827dc28)              = -1 ENOMEM (Cannot allocate memory)
> nanosleep({0, 1000000}, NULL)           = 0
> ioctl(3, 0x108, 0x827dc28)              = -1 ENOMEM (Cannot allocate memory)
> nanosleep({0, 1000000}, NULL)           = 0
> ioctl(3, 0x108, 0x827dc28)              = -1 ENOMEM (Cannot allocate memory)
> nanosleep({0, 1000000}, NULL)           = 0
> ioctl(3, 0x108, 0x827dc28)              = -1 ENOMEM (Cannot allocate memory)
> nanosleep({0, 1000000}, NULL)           = 0
> ioctl(3, 0x108, 0x827dc28)              = -1 ENOMEM (Cannot allocate memory)
> nanosleep({0, 1000000}, NULL)           = 0
> ioctl(3, 0x108, 0x827dc28)              = -1 ENOMEM (Cannot allocate memory)
> nanosleep({0, 1000000}, NULL)           = 0
> ioctl(3, 0x108, 0x827dc28)              = -1 ENOMEM (Cannot allocate memory)
> nanosleep({0, 1000000}, NULL)           = 0
> ioctl(3, 0x108, 0x827dc28)              = -1 ENOMEM (Cannot allocate memory)
> nanosleep({0, 1000000}, NULL)           = 0
> ioctl(3, 0x108, 0x827dc28)              = -1 ENOMEM (Cannot allocate memory)
> nanosleep({0, 1000000}, NULL)           = 0
> ioctl(3, 0x108, 0x827dc28)              = -1 ENOMEM (Cannot allocate memory)
> nanosleep({0, 1000000}, NULL)           = 0
> ioctl(3, 0x108, 0x827dc28)              = -1 ENOMEM (Cannot allocate memory)
> nanosleep({0, 1000000}, NULL)           = 0
> ioctl(3, 0x108, 0x827dc28)              = -1 ENOMEM (Cannot allocate memory)
> nanosleep({0, 1000000}, NULL)           = 0
> ioctl(3, 0x108, 0x827dc28)              = -1 ENOMEM (Cannot allocate memory)
> nanosleep({0, 1000000}, NULL)           = 0
> ioctl(3, 0x108, 0x827dc28)              = -1 ENOMEM (Cannot allocate memory)
> nanosleep({0, 1000000}, NULL)           = 0
> ioctl(3, 0x108, 0x827dc28)              = -1 ENOMEM (Cannot allocate memory)
> nanosleep({0, 1000000}, NULL)           = 0
> ioctl(3, 0x108, 0x827dc28)              = -1 ENOMEM (Cannot allocate memory)
> nanosleep({0, 1000000}, NULL)           = 0
> ioctl(3, 0x108, 0x827dc28)              = -1 ENOMEM (Cannot allocate memory)
> nanosleep({0, 1000000}, NULL)           = 0
> ioctl(3, 0x108, 0x827dc28)              = -1 ENOMEM (Cannot allocate memory)
> nanosleep({0, 1000000}, NULL)           = 0
> ioctl(3, 0x108, 0x827dc28)              = -1 ENOMEM (Cannot allocate memory)
> nanosleep({0, 1000000}, NULL)           = 0
> ioctl(3, 0x108, 0x827dc28)              = -1 ENOMEM (Cannot allocate memory)
> nanosleep({0, 1000000}, NULL)           = 0
> ioctl(3, 0x108, 0x827dc28)              = -1 ENOMEM (Cannot allocate memory)
> nanosleep({0, 1000000}, NULL)           = 0
> ioctl(3, 0x108, 0x827dc28)              = -1 ENOMEM (Cannot allocate memory)
> nanosleep({0, 1000000}, NULL)           = 0
> ioctl(3, 0x108, 0x827dc28)              = -1 ENOMEM (Cannot allocate memory)
> nanosleep({0, 1000000}, NULL)           = 0
> ioctl(3, 0x108, 0x827dc28)              = -1 ENOMEM (Cannot allocate memory)
> nanosleep({0, 1000000}, NULL)           = 0
> ioctl(3, 0x108, 0x827dc28)              = -1 ENOMEM (Cannot allocate memory)
> nanosleep({0, 1000000}, NULL)           = 0
> ioctl(3, 0x108, 0x827dc28)              = -1 ENOMEM (Cannot allocate memory)
> nanosleep({0, 1000000}, NULL)           = 0
> ioctl(3, 0x108, 0x827dc28)              = -1 ENOMEM (Cannot allocate memory)
> nanosleep({0, 1000000}, NULL)           = 0
> ioctl(3, 0x108, 0x827dc28)              = -1 ENOMEM (Cannot allocate memory)
> nanosleep({0, 1000000}, NULL)           = 0
> ioctl(3, 0x108, 0x827dc28)              = -1 ENOMEM (Cannot allocate memory)
> nanosleep({0, 1000000}, NULL)           = 0
> ioctl(3, 0x108, 0x827dc28)              = -1 ENOMEM (Cannot allocate memory)
> nanosleep({0, 1000000}, NULL)           = 0
> ioctl(3, 0x108, 0x827dc28)              = -1 ENOMEM (Cannot allocate memory)
> nanosleep({0, 1000000}, NULL)           = 0
> ioctl(3, 0x108, 0x827dc28)              = -1 ENOMEM (Cannot allocate memory)
> nanosleep({0, 1000000}, NULL)           = 0
> ioctl(3, 0x108, 0x827dc28)              = -1 ENOMEM (Cannot allocate memory)
> nanosleep({0, 1000000}, NULL)           = 0
> ioctl(3, 0x108, 0x827dc28)              = -1 ENOMEM (Cannot allocate memory)
> nanosleep({0, 1000000}, NULL)           = 0
> ioctl(3, 0x108, 0x827dc28)              = -1 ENOMEM (Cannot allocate memory)
> nanosleep({0, 1000000}, NULL)           = 0
> ioctl(3, 0x108, 0x827dc28)              = -1 ENOMEM (Cannot allocate memory)
> nanosleep({0, 1000000}, NULL)           = 0
> ioctl(3, 0x108, 0x827dc28)              = -1 ENOMEM (Cannot allocate memory)
> nanosleep({0, 1000000}, NULL)           = 0
> ioctl(3, 0x108, 0x827dc28)              = -1 ENOMEM (Cannot allocate memory)
> nanosleep({0, 1000000}, NULL)           = 0
> ioctl(3, 0x108, 0x827dc28)              = -1 ENOMEM (Cannot allocate memory)
> nanosleep({0, 1000000}, NULL)           = 0
> ioctl(3, 0x108, 0x827dc28)              = -1 ENOMEM (Cannot allocate memory)
> 
> ......
> 
> ioctl(3, 0x108, 0x82822a8)              = -1 ENOMEM (Cannot allocate memory)
> nanosleep({0, 1000000}, NULL)           = 0
> close(3)                                = 0
> fstat64(1, {st_mode=S_IFCHR|0600, st_rdev=makedev(136, 2), ...}) = 0
> old_mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS,
> -1, 0) = 0xb7f95000
> write(1, "\n", 1)                       = 1
> write(1, "No controller found.\n", 21)  = 21
> write(1, "Make sure appropriate AMCC/3ware"..., 62) = 62
> write(1, "\n", 1)                       = 1
> munmap(0xb7f95000, 4096)                = 0
> exit_group(0)                           = ?
> Process 5170 detached
> 
> :-(
> 
> PS : why when i send a mail on this list i don't receive them ?
> 
> Thanks all
> 
> -- 
>                          ''~``
>                         ( o o )
> +------------------.oooO--(_)--Oooo.---------------------+
> | Guillaume Chéramy - Guidtz                           |
> | E-Mail : guillaume.cheramy@xxxxxxxxx                   |
> |                  (   )                               |
> +---------------------\ (----(   )-----------------------+
>                        \_)    ) /
>                              (_/
> 
> 
> _______________________________________________
> Xen-users mailing list
> Xen-users@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-users

-- 
"The greatest proof that intelligent life other that humans exists in
 the universe is that none of it has tried to contact us!"


_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users

<Prev in Thread] Current Thread [Next in Thread>