Re: [Xen-users] Re: domU corrupt after server crash, help needed

Have you checked /dev/mapper/?
Since you ran "kpartx -a /dev/loop4", you should see:
  /dev/mapper/loop4p1
  /dev/mapper/loop4p2
  /dev/mapper/loop4p3
and so on...

You can treat each of them like a device. So, you can fsck/dev/mapper/loop4p1, for example.And mount it from dom0 to get the data. Well, if fsck works out ok,maybe the domU will work.

Since you went as far as doing losetup, I figure you tried doing fsckand mount, but since I don't see it in your email, just in case... Ihope this helps.


-Paul Choi

Rudi Ahlers wrote:

Does anyone know how to fix this?

On 5/8/09, Rudi Ahlers <rudiahlers@xxxxxxxxx> wrote:

On Fri, May 8, 2009 at 2:29 PM, Rudi Ahlers <rudiahlers@xxxxxxxxx> wrote:

Hi all,

One of our Dell servers has failed badly, and one of the domU's has been
corrupted in the process. It boots up to a point and then gives me a
kernel
panic:

Loading dm-zero.ko module
Loading dm-snapshot.ko module
Scanning and configuring dmraid supported devices
Scanning logical volumes
  Reading all physical volumes.  This may take a while...
  No volume groups found
Activating logical volumes
  Volume group "VolGroup00" not found
Creating root device.
Mounting root filesystem.
mount: could not find filesystem '/dev/root'
Setting up other filesystems.
Setting up new root fs
setuproot: moving /dev failed: No such file or directory
no fstab.sys, mounting internal defaults
setuproot: error mounting /proc: No such file or directory
setuproot: error mounting /sys: No such file or directory
Switching to new root and running init.
unmounting old /dev
unmounting old /proc
unmounting old /sys
switchroot: mount failed: No such file or directory
Kernel panic - not syncing: Attempted to kill init!


It shows up as a Zombie:

[root@xen ~]# xm list
Name                                      ID Mem(MiB) VCPUs State
Time(s)
Domain-0                                   0     1439     1 r-----
329.0
Zombie-hfserver2                          15     1024     1 ----c-
0.5
hfdns02                                   10      519     2 r-----
1552.8


I can't mount either:

root@xen ~]# mount /dev/data/hf
hfdns02    hfserver2
[root@xen ~]# mount /dev/data/hfserver2 /mnt/cpanel/
mount: you must specify the filesystem type
[root@xen ~]# mount -o loop /dev/data/hfserver2 /mnt/cpanel/
mount: you must specify the filesystem type

Here's the output of the LVM partitions:

[root@xen ~]# lvscan
  ACTIVE            '/dev/data/cpanel002' [100.00 GB] inherit
  ACTIVE            '/dev/data/windows2003_web' [30.00 GB] inherit
  ACTIVE            '/dev/data/storage' [50.00 GB] inherit
  ACTIVE   Original '/dev/data/hfserver2' [30.00 GB] inherit
  ACTIVE            '/dev/data/hfdns02' [30.00 GB] inherit
  ACTIVE            '/dev/data/pluto' [30.00 GB] inherit
  ACTIVE   Snapshot '/dev/data/pluto_s' [30.00 GB] inherit
  ACTIVE            '/dev/system/root' [39.06 GB] inherit
  ACTIVE            '/dev/system/swap' [9.75 GB] inherit
[root@xen ~]# vgscan
  Reading all physical volumes.  This may take a while...
  Found volume group "data" using metadata type lvm2
  Found volume group "system" using metadata type lvm2
[root@xen ~]#


Does anyone know how to fix a LVM like this?


--


 Here's what I've done so far:

[root@xen ~]# losetup /dev/loop4  /dev/data/hfserver2

# This mounts the LVM partition as imaginary physical to /dev/loop4
# data is the virtual group (VG) name

[root@xen ~]#  kpartx -va /dev/loop4
add map loop4p1 : 0 208782 linear /dev/loop4 63
add map loop4p2 : 0 62701695 linear /dev/loop4 208845

# This creates a device map with partitions in /dev/data/hfserver in
/dev/mapper

[root@xen ~]# vgscan
  Reading all physical volumes.  This may take a while...
  Found volume group "VolGroup00" using metadata type lvm2
  Found volume group "data" using metadata type lvm2
  Found volume group "system" using metadata type lvm2


[root@xen ~]# lvscan
  inactive          '/dev/VolGroup00/LogVol00' [27.94 GB] inherit
  inactive          '/dev/VolGroup00/LogVol01' [1.94 GB] inherit
  ACTIVE            '/dev/data/cpanel002' [100.00 GB] inherit
  ACTIVE            '/dev/data/windows2003_web' [30.00 GB] inherit
  ACTIVE            '/dev/data/storage' [50.00 GB] inherit
  ACTIVE   Original '/dev/data/hfserver2' [30.00 GB] inherit
  ACTIVE            '/dev/data/hfdns02' [30.00 GB] inherit
  ACTIVE            '/dev/data/pluto' [30.00 GB] inherit
  ACTIVE   Snapshot '/dev/data/pluto_s' [30.00 GB] inherit
  ACTIVE            '/dev/system/root' [39.06 GB] inherit
  ACTIVE            '/dev/system/swap' [9.75 GB] inherit

[root@xen ~]# lvchange -ay VolGroup00
[root@xen ~]# lvscan
  ACTIVE            '/dev/VolGroup00/LogVol00' [27.94 GB] inherit
  ACTIVE            '/dev/VolGroup00/LogVol01' [1.94 GB] inherit
  ACTIVE            '/dev/data/cpanel002' [100.00 GB] inherit
  ACTIVE            '/dev/data/windows2003_web' [30.00 GB] inherit
  ACTIVE            '/dev/data/storage' [50.00 GB] inherit
  ACTIVE   Original '/dev/data/hfserver2' [30.00 GB] inherit
  ACTIVE            '/dev/data/hfdns02' [30.00 GB] inherit
  ACTIVE            '/dev/data/pluto' [30.00 GB] inherit
  ACTIVE   Snapshot '/dev/data/pluto_s' [30.00 GB] inherit
  ACTIVE            '/dev/system/root' [39.06 GB] inherit
  ACTIVE            '/dev/system/swap' [9.75 GB] inherit

[root@xen ~]# e2fsck /dev/VolGroup00/LogVol00
e2fsck 1.39 (29-May-2006)
/dev/VolGroup00/LogVol00: clean, 631982/7325696 files, 4512772/7323648
blocks


At first it found a whole lot of damages inodes which I repaired.

Then, I reversed the process:

[root@xen ~]# lvchange -an VolGroup00
[root@xen ~]# lvscan
  inactive          '/dev/VolGroup00/LogVol00' [27.94 GB] inherit
  inactive          '/dev/VolGroup00/LogVol01' [1.94 GB] inherit
  ACTIVE            '/dev/data/cpanel002' [100.00 GB] inherit
  ACTIVE            '/dev/data/windows2003_web' [30.00 GB] inherit
  ACTIVE            '/dev/data/storage' [50.00 GB] inherit
  ACTIVE   Original '/dev/data/hfserver2' [30.00 GB] inherit
  ACTIVE            '/dev/data/hfdns02' [30.00 GB] inherit
  ACTIVE            '/dev/data/pluto' [30.00 GB] inherit
  ACTIVE   Snapshot '/dev/data/pluto_s' [30.00 GB] inherit
  ACTIVE            '/dev/system/root' [39.06 GB] inherit
  ACTIVE            '/dev/system/swap' [9.75 GB] inherit



[root@xen ~]# vgchange -an VolGroup00
  0 logical volume(s) in volume group "VolGroup00" now active
[root@xen ~]# kpartx -d /dev/loop4
[root@xen ~]# losetup -d /dev/loop4
[root@xen ~]#

[root@xen ~]# xm create -c /etc/xen/hfserver2


And then it dies:


 Reading all physical volumes.  This may take a while...
  Found volume group "VolGroup00" using metadata type lvm2
Activating logical volumes
  2 logical volume(s) in volume group "VolGroup00" now active
Creating root device.
Mounting root filesystem.
kjournald starting.  Commit interval 5 seconds
EXT3-fs: mounted filesystem with ordered data mode.
Setting up other filesystems.
Setting up new root fs
no fstab.sys, mounting internal defaults
Switching to new root and running init.
unmounting old /dev
unmounting old /proc
unmounting old /sys
exec of init (/sbin/init) failed!!!: No such file or directory
Kernel panic - not syncing: Attempted to kill init!
 [root@xen ~]#


--
Kind Regards
Rudi Ahlers
CEO, SoftDux Hosting
Web: http://www.SoftDux.com
Office: 087 805 9573
Cell: 082 554 7532




_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users

WARNING - OLD ARCHIVES

xen-users

Re: [Xen-users] Re: domU corrupt after server crash, help needed trying