WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-devel

[Xen-devel] Re: Bug#637234: linux-image-3.0.0-1-686-pae: I/O errors usin

To: Gedalya <gedalya@xxxxxxxxxxx>, 637234@xxxxxxxxxxxxxxx, xen-devel <xen-devel@xxxxxxxxxxxxxxxxxxx>
Subject: [Xen-devel] Re: Bug#637234: linux-image-3.0.0-1-686-pae: I/O errors using ext4 under xen
From: Ian Campbell <ijc@xxxxxxxxxxxxxx>
Date: Thu, 25 Aug 2011 07:47:08 +0100
Cc: Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx>
Delivery-date: Wed, 24 Aug 2011 23:48:09 -0700
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <20110809180728.2279.11548.reportbug@xxxxxxxxxxxxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
References: <20110809180728.2279.11548.reportbug@xxxxxxxxxxxxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
Hi Konrad,

Does this look at all familiar? There is some more info in the full bug
log at http://bugs.debian.org/637234 . In particular, contrary to the
message below, the user subsequently confirmed that the issue appears to
be Xen specific (doesn't happen on native or vmware) and that it arose
between 2.6.39-2-686-pae and 3.0.0-1-686-pae.

Could it be related to edf6ef59ec7e "xen-blkfront: Introduce
BLKIF_OP_FLUSH_DISKCACHE support"? That looks like the only pertinent
change between 2.6.39 and 3.0.

Gedalya, 2.6.39-2-686-pae could be anything from v2.6.39..v2.6.39.2
please could you confirm which package version you have installed in
case it makes a difference.

Cheers,
Ian.

On Tue, 2011-08-09 at 14:07 -0400, Gedalya wrote:
> Package: linux-2.6
> Version: 3.0.0-1
> Severity: important
> 
> 
> Hello,
> 
> I have a xen host running debian squeeze, amd64, some of the DomU's are
> running wheezy. My mail server is a DomU called "mail", using ext4 for the
> root (and other) FS. A dist-upgrade on "mail" has upgraded the kernel to
> linux-image-3.0.0-1-686-pae, and at this point I started getting I/O errors
> during the boot process, as follows:
> 
> -----------
> Starting MySQL database server: mysqld[    6.453894] end_request: I/O error, 
> dev xvda, sector 4456704
> [    6.453919] end_request: I/O error, dev xvda, sector 4456704
> [    6.453964] Aborting journal on device xvda-8.
> [    6.462873] EXT4-fs error (device xvda): ext4_journal_start_sb:296: 
> Detected aborted journal
> [    6.462903] EXT4-fs (xvda): Remounting filesystem read-only
> [    6.463276] journal commit I/O error
>  . . . . . . . . . . . . . . failed!
> Starting MTA: exim4.
> Starting IMAP/POP3 mail server: dovecot.
> startpar: service(s) returned failure: mysql ... failed!
> -----------
> 
> So I went ahead and installed wheezy on a brand new DomU, and this
> was repeated immediately when booting the machine after the installation
> completed.
> 
> -----------
> Starting NFS common utilities: statd[    3.977392] end_request: I/O error, 
> dev xvda, sector 4456808
> [    3.977415] end_request: I/O error, dev xvda, sector 4456808
> [    3.977470] Aborting journal on device xvda-8.
> [    3.990442] journal commit I/O error
> [    3.991041] EXT4-fs error (device xvda): ext4_journal_start_sb:296: 
> Detected aborted journal
> [    3.991126] EXT4-fs (xvda): Remounting filesystem read-only
>  failed!
> Cleaning up temporary files....
> Setting up console font and keymap...done.
> startpar: service(s) returned failure: nfs-common ... failed!
> INIT: Entering runlevel: 2
> Using makefile-style concurrent boot in runlevel 2.
> Starting rpcbind daemon...Already running..
> Starting NFS common utilities: statd failed!
> touch: cannot touch `/var/log/dmesg.new': Read-only file system
> chown: cannot access `/var/log/dmesg.new': No such file or directory
> chmod: cannot access `/var/log/dmesg.new': No such file or directory
> ln: creating hard link `/var/log//dmesg.0': Read-only file system
> ... etc. ...
> -----------
> 
> Now, it happenes this way exactly every _other_ time the machines boot.
> When I reboot after these I/O errors, fsck is run and then the machine
> seems to be actually fine until the next reboot when it all happens
> again.
> 
> For me, this is happening on xen DomU's, only when running linux
> 3.0.0-1-686-pae, only when using ext4 for the root FS.
> No problems when booting back to 2.6.39-2-686-pae.
> 
> Please let me know what more specific testing needs to be done, if
> necessary I can test more platforms / flavors.
> 
> I have observed nothing to suggest this is related to xen, it's just my
> platform here.
> 
> -- Package-specific info:
> ** Version:
> Linux version 3.0.0-1-686-pae (Debian 3.0.0-1) (ben@xxxxxxxxxxxxxxx) (gcc 
> version 4.5.3 (Debian 4.5.3-3) ) #1 SMP Sun Jul 24 14:27:32 UTC 2011
> 
> ** Command line:
> root=UUID=8a1a7bca-b0e2-4714-baf1-b852eab25843 ro  quiet 
> 
> ** Not tainted
> 
> ** Kernel log:
> [    0.016117] PCI: System does not support PCI
> [    0.016120] PCI: System does not support PCI
> [    0.016231] Switching to clocksource xen
> [    0.017739] pnp: PnP ACPI: disabled
> [    0.017742] PnPBIOS: Disabled
> [    0.018820] Switched to NOHz mode on CPU #1
> [    0.018902] Switched to NOHz mode on CPU #0
> [    0.020460] PCI: max bus depth: 0 pci_try_num: 1
> [    0.020696] NET: Registered protocol family 2
> [    0.020967] IP route cache hash table entries: 8192 (order: 3, 32768 bytes)
> [    0.021437] TCP established hash table entries: 32768 (order: 6, 262144 
> bytes)
> [    0.021752] TCP bind hash table entries: 32768 (order: 6, 262144 bytes)
> [    0.022063] TCP: Hash tables configured (established 32768 bind 32768)
> [    0.022069] TCP reno registered
> [    0.022077] UDP hash table entries: 512 (order: 2, 16384 bytes)
> [    0.022100] UDP-Lite hash table entries: 512 (order: 2, 16384 bytes)
> [    0.022469] NET: Registered protocol family 1
> [    0.022486] PCI: CLS 0 bytes, default 64
> [    0.022574] Unpacking initramfs...
> [    0.042069] Freeing initrd memory: 22480k freed
> [    0.046257] platform rtc_cmos: registered platform RTC device (no PNP 
> device found)
> [    0.046605] audit: initializing netlink socket (disabled)
> [    0.046616] type=2000 audit(1312911347.921:1): initialized
> [    0.056740] HugeTLB registered 2 MB page size, pre-allocated 0 pages
> [    0.057039] VFS: Disk quotas dquot_6.5.2
> [    0.057099] Dquot-cache hash table entries: 1024 (order 0, 4096 bytes)
> [    0.057194] msgmni has been set to 999
> [    0.057354] alg: No test for stdrng (krng)
> [    0.057382] Block layer SCSI generic (bsg) driver version 0.4 loaded 
> (major 253)
> [    0.057386] io scheduler noop registered
> [    0.057388] io scheduler deadline registered
> [    0.057402] io scheduler cfq registered (default)
> [    0.057598] isapnp: Scanning for PnP cards...
> [    0.409558] isapnp: No Plug & Play device found
> [    0.409873] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
> [    0.412773] Linux agpgart interface v0.103
> [    0.413203] i8042: PNP: No PS/2 controller found. Probing ports directly.
> [    0.414033] i8042: No controller found
> [    0.414227] mousedev: PS/2 mouse device common for all mice
> [    0.454109] rtc_cmos rtc_cmos: rtc core: registered rtc_cmos as rtc0
> [    0.454143] rtc_cmos: probe of rtc_cmos failed with error -38
> [    0.454162] cpuidle: using governor ladder
> [    0.454164] cpuidle: using governor menu
> [    0.454336] TCP cubic registered
> [    0.454455] NET: Registered protocol family 10
> [    0.454980] Mobile IPv6
> [    0.454983] NET: Registered protocol family 17
> [    0.454987] Registering the dns_resolver key type
> [    0.455001] Using IPI No-Shortcut mode
> [    0.455069] PM: Hibernation image not present or could not be loaded.
> [    0.455080] registered taskstats version 1
> [    0.455093] XENBUS: Device with no driver: device/vbd/51712
> [    0.455095] XENBUS: Device with no driver: device/vbd/51744
> [    0.455097] XENBUS: Device with no driver: device/vif/0
> [    0.455099] XENBUS: Device with no driver: device/vif/1
> [    0.455102] XENBUS: Device with no driver: device/console/0
> [    0.455114] 
> /build/buildd-linux-2.6_3.0.0-1-i386-ML66CU/linux-2.6-3.0.0/debian/build/source_i386_none/drivers/rtc/hctosys.c:
>  unable to open rtc device (rtc0)
> [    0.455175] Initializing network drop monitor service
> [    0.455438] Freeing unused kernel memory: 404k freed
> [    0.456030] Write protecting the kernel text: 2768k
> [    0.456248] Write protecting the kernel read-only data: 1068k
> [    0.456250] NX-protecting the kernel data: 3376k
> [    0.490525] udevd[50]: starting version 172
> [    0.510452] Initialising Xen virtual ethernet driver.
> [    0.526964] blkfront: xvda: barrier: enabled
> [    0.528495]  xvda:
> [    0.528633] Setting capacity to 10485760
> [    0.528637] xvda: detected capacity change from 0 to 5368709120
> [    0.529412] blkfront: xvdc: barrier: enabled
> [    0.558774]  xvdc: unknown partition table
> [    0.559489] Setting capacity to 1048576
> [    0.559502] xvdc: detected capacity change from 0 to 536870912
> [    0.973128] PM: Starting manual resume from disk
> [    0.973131] PM: Hibernation image partition 202:32 present
> [    0.973133] PM: Looking for hibernation image.
> [    0.973405] PM: Image not found (code -22)
> [    0.973408] PM: Hibernation image not present or could not be loaded.
> [    0.983577] EXT4-fs (xvda): INFO: recovery required on readonly filesystem
> [    0.983581] EXT4-fs (xvda): write access will be enabled during recovery
> [    1.024513] EXT4-fs warning (device xvda): ext4_clear_journal_err:4155: 
> Filesystem error recorded from previous mount: IO failure
> [    1.024524] EXT4-fs warning (device xvda): ext4_clear_journal_err:4156: 
> Marking fs in need of filesystem check.
> [    1.025790] EXT4-fs (xvda): recovery complete
> [    1.026596] EXT4-fs (xvda): mounted filesystem with ordered data mode. 
> Opts: (null)
> [    1.928491] udevd[160]: starting version 172
> [    2.124852] input: PC Speaker as /devices/platform/pcspkr/input/input0
> [    2.204922] Error: Driver 'pcspkr' is already registered, aborting...
> [    2.550476] Adding 524284k swap on /dev/xvdc.  Priority:-1 extents:1 
> across:524284k SS
> [    2.564932] EXT4-fs (xvda): re-mounted. Opts: (null)
> [    3.156251] blkfront: barrier: empty write xvda op failed
> [    3.156255] blkfront: xvda: barrier or flush: disabled
> [    3.185628] EXT4-fs (xvda): re-mounted. Opts: errors=remount-ro
> [    3.251006] loop: module loaded
> [    4.326336] RPC: Registered named UNIX socket transport module.
> [    4.326344] RPC: Registered udp transport module.
> [    4.326350] RPC: Registered tcp transport module.
> [    4.326356] RPC: Registered tcp NFSv4.1 backchannel transport module.
> [    4.361714] FS-Cache: Loaded
> [    4.382614] FS-Cache: Netfs 'nfs' registered for caching
> [    4.402479] Installing knfsd (copyright (C) 1996 okir@xxxxxxxxxxxx).
> [   14.460105] eth0: no IPv6 routers present
> 
> ** Model information
> not available
> 
> ** Loaded modules:
> Module                  Size  Used by
> nfsd                  197933  2 
> nfs                   218404  0 
> lockd                  61314  2 nfsd,nfs
> fscache                31952  1 nfs
> auth_rpcgss            32183  2 nfsd,nfs
> nfs_acl                12463  2 nfsd,nfs
> sunrpc                139050  6 nfsd,nfs,lockd,auth_rpcgss,nfs_acl
> loop                   17866  0 
> evdev                  12995  0 
> snd_pcm                53315  0 
> snd_timer              22027  1 snd_pcm
> snd                    38562  2 snd_pcm,snd_timer
> soundcore              12992  1 snd
> snd_page_alloc         12899  1 snd_pcm
> pcspkr                 12515  0 
> ext4                  274801  1 
> mbcache                12898  1 ext4
> jbd2                   56798  1 ext4
> crc16                  12327  1 ext4
> xen_netfront           21670  0 
> xen_blkfront           17215  2 
> 
> ** PCI devices:
> 
> ** USB devices:
> not available
> 
> 
> -- System Information:
> Debian Release: wheezy/sid
>   APT prefers testing
>   APT policy: (500, 'testing')
> Architecture: i386 (i686)
> 
> Kernel: Linux 3.0.0-1-686-pae (SMP w/2 CPU cores)
> Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8)
> Shell: /bin/sh linked to /bin/dash
> 
> Versions of packages linux-image-3.0.0-1-686-pae depends on:
> ii  debconf [debconf-2.0]         1.5.40     Debian configuration management 
> sy
> ii  initramfs-tools [linux-initra 0.99       tools for generating an initramfs
> ii  linux-base                    3.3        Linux image base package
> ii  module-init-tools             3.16-1     tools for managing Linux kernel 
> mo
> 
> Versions of packages linux-image-3.0.0-1-686-pae recommends:
> pn  firmware-linux-free           <none>     (no description available)
> ii  libc6-i686                    2.13-10    Embedded GNU C Library: Shared 
> lib
> 
> Versions of packages linux-image-3.0.0-1-686-pae suggests:
> ii  grub-pc                       1.99-9     GRand Unified Bootloader, 
> version 
> pn  linux-doc-3.0.0               <none>     (no description available)
> 
> Versions of packages linux-image-3.0.0-1-686-pae is related to:
> pn  firmware-bnx2                 <none>     (no description available)
> pn  firmware-bnx2x                <none>     (no description available)
> pn  firmware-ipw2x00              <none>     (no description available)
> pn  firmware-ivtv                 <none>     (no description available)
> pn  firmware-iwlwifi              <none>     (no description available)
> pn  firmware-linux                <none>     (no description available)
> pn  firmware-linux-nonfree        <none>     (no description available)
> pn  firmware-qlogic               <none>     (no description available)
> pn  firmware-ralink               <none>     (no description available)
> pn  xen-hypervisor                <none>     (no description available)
> 
> -- debconf information:
>   linux-image-3.0.0-1-686-pae/prerm/removing-running-kernel-3.0.0-1-686-pae: 
> true
>   linux-image-3.0.0-1-686-pae/postinst/ignoring-ramdisk:
>   linux-image-3.0.0-1-686-pae/postinst/missing-firmware-3.0.0-1-686-pae:
>   linux-image-3.0.0-1-686-pae/postinst/depmod-error-initrd-3.0.0-1-686-pae: 
> false
> 
> 
> 
> -- 
> To UNSUBSCRIBE, email to debian-kernel-REQUEST@xxxxxxxxxxxxxxxx
> with a subject of "unsubscribe". Trouble? Contact listmaster@xxxxxxxxxxxxxxxx
> Archive: 
> http://lists.debian.org/20110809180728.2279.11548.reportbug@xxxxxxxxxxxxxxxxx
> 
> 

-- 
Ian Campbell


In those days he was wiser than he is now -- he used to frequently take
my advice.
                -- Winston Churchill

Attachment: signature.asc
Description: This is a digitally signed message part

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel