WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-users

[Xen-users] I/O errors in domU with LVM on DRBD

To: xen-users@xxxxxxxxxxxxxxxxxxx
Subject: [Xen-users] I/O errors in domU with LVM on DRBD
From: jai lamerton <jai@xxxxxxxxxxxxxx>
Date: Mon, 25 Jun 2007 11:16:07 +1000
Delivery-date: Sun, 24 Jun 2007 18:14:00 -0700
Envelope-to: www-data@xxxxxxxxxxxxxxxxxx
List-help: <mailto:xen-users-request@lists.xensource.com?subject=help>
List-id: Xen user discussion <xen-users.lists.xensource.com>
List-post: <mailto:xen-users@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=unsubscribe>
Sender: xen-users-bounces@xxxxxxxxxxxxxxxxxxx
Hi,

Sorry for the need of the long winded email. Looking for some answers to the following.

I am setting up a xen PV domU on top of a LVM partitioned DRBD device. Everything was going just fine until I tried to test the filesystems in the domU.

Here is my setup;
Dom0 OS: CentOS release 5 (Final)
Kernel: 2.6.18-8.1.4.el5.centos.plusxen
Xen: xen-3.0.3-25.0.3.el5
DRBD: drbd-8.0.3-1.el5.centos

PV DomU OS: CentOS release 4.5 (Final)
Kernel: 2.6.9-55.ELxenU

my drbd.conf
global {
        usage-count yes;
}
common {
        startup {
                wfc-timeout 0;
                degr-wfc-timeout 60;
        }
        disk {
                on-io-error detach;
        }
        syncer {
                rate 100M;
                al-extents 257;

        }
        net {
                max-buffers   2048;
                unplug-watermark   2048;
                max-epoch-size  2048;
                ko-count 4;
                cram-hmac-alg "sha1";
                shared-secret "asecretstring";
                after-sb-0pri disconnect;
                after-sb-1pri consensus;
                after-sb-2pri disconnect;
                rr-conflict disconnect;
        }
}
resource r0 {
        protocol C;
        handlers {
                pri-on-incon-degr "echo 0 > /proc/sysrq-trigger ; halt -f";
                pri-lost-after-sb "echo o > /proc/sysrq-trigger ; halt -f";
                local-io-error "echo 0 > /proc/sysrq-trigger ; halt -f";
                outdate-peer "/usr/lib64/heartbeat/drbd-peer-outdater";
        }
        on xenhostA {
                device /dev/drbd0;
                disk /dev/sda3;
                address 172.16.0.30:7788;
                meta-disk internal;
        }

        on xenhostB {
                device /dev/drbd0;
                disk /dev/sda3;
                address 172.16.0.31:7788;
                meta-disk internal;
        }
}

LVM
# pvscan
  PV /dev/drbd0   VG VGvm1      lvm2 [15.54 GB / 3.05 GB free]
# lvscan
  ACTIVE            '/dev/VGvm1/LVroot' [10.00 GB] inherit
  ACTIVE            '/dev/VGvm1/LVtmp' [500.00 MB] inherit
  ACTIVE            '/dev/VGvm1/LVswap' [2.00 GB] inherit

lvm.conf has;
filter = [ "r|/dev/sda3|", "a|/dev/drbd0|" ]

My xen domU conf
import os, re
arch = os.uname()[4]
if re.search('64', arch):
    arch_libdir = 'lib64'
else:
    arch_libdir = 'lib'
name = "centos45domU"
memory = "512"
disk = [ 'phy:/dev/mapper/VGvm1-LVroot,xvda1,w',
         'phy:/dev/mapper/VGvm1-LVtmp,xvda2,w',
         'phy:/dev/mapper/VGvm1-LVswap,xvda3,w', ]
vif = [ 'mac=00:16:3e:d2:14:70, bridge=xenbr0', ]
vfb = ["type=vnc,vncunused=1"]
bootloader="/usr/bin/pygrub"
vcpus=2
on_reboot   = 'restart'
on_crash    = 'restart'

DomU /etc/fstab
/dev/xvda1              /                       ext3    defaults,usrquota        1 1
none                    /dev/pts                devpts  gid=5,mode=620  0 0
none                    /dev/shm                tmpfs   defaults        0 0
none                    /proc                   proc    defaults        0 0
none                    /sys                    sysfs   defaults        0 0
/dev/xvda2              /tmp                    ext3    defaults        0 0
/dev/xvda3              swap                    swap    defaults        0 0


** Problem **
While doing some tests writing files in domU I got;

# dd if=/dev/zero of=/deleteme bs=1M count=100
100+0 records in
100+0 records out

# dd if=/dev/zero of=/tmp/deleteme bs=1M count=100
end_request: I/O error, dev xvda2, sector 366778
end_request: I/O error, dev xvda2, sector 366844
end_request: I/O error, dev xvda2, sector 381116
printk: 8071 messages suppressed.
Buffer I/O error on device xvda2, logical block 190558
lost page write due to I/O error on xvda2
end_request: I/O error, dev xvda2, sector 381182
end_request: I/O error, dev xvda2, sector 415678
end_request: I/O error, dev xvda2, sector 450164
end_request: I/O error, dev xvda2, sector 1204
end_request: I/O error, dev xvda2, sector 1340
end_request: I/O error, dev xvda2, sector 1454
end_request: I/O error, dev xvda2, sector 1522
end_request: I/O error, dev xvda2, sector 1652
end_request: I/O error, dev xvda2, sector 1788
end_request: I/O error, dev xvda2, sector 1902
Aborting journal on device xvda2.
ext3_abort called.
EXT3-fs error (device xvda2): ext3_journal_start_sb: Detected aborted journal
Remounting filesystem read-only

DomU /var/log/messages contains
Jun 25 10:34:29 xenpv1 kernel: SELinux: initialized (dev xvda2, type ext3),uses xattr
Jun 25 10:34:34 xenpv1 kernel: end_request: I/O error, dev xvda2, sector 264182
Jun 25 10:34:34 xenpv1 kernel: printk: 32 messages suppressed.
Jun 25 10:34:34 xenpv1 kernel: Buffer I/O error on device xvda2, logical block 132091
Jun 25 10:34:34 xenpv1 kernel: lost page write due to I/O error on xvda2
Jun 25 10:34:34 xenpv1 kernel: Buffer I/O error on device xvda2, logical block 132092

Dom0 /var/log/messages contains
Jun 25 10:34:34 xenhosta kernel: drbd0: bio would need to, but cannot, be split: (vcnt=4,idx=0,size=4096,sector=10750334)
Jun 25 10:34:34 xenhosta kernel: drbd0: bio would need to, but cannot, be split: (vcnt=4,idx=0,size=4096,sector=10750458)
Jun 25 10:34:34 xenhosta kernel: drbd0: bio would need to, but cannot, be split: (vcnt=4,idx=0,size=4096,sector=10750524)
Jun 25 10:34:34 xenhosta kernel: drbd0: bio would need to, but cannot, be split: (vcnt=4,idx=0,size=4096,sector=10750590)

However running the same test on the device that is assigned to xvda2 mounted under dom0, there is no errors.

I will also ask the DRBD lists what "bio would need to, but cannot, be split:" means.

If anyone can shed some light on this, would be great. 

Thank you kindly.
Jai.



_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users
<Prev in Thread] Current Thread [Next in Thread>