This was a bit interesting... I was moving a domain off an old xen 2.0 box
to a newer (and more stable) 3.0.0 system. I accidentally copied the
/etc/fstab off the old box over the version I had updated for the new xen3
box... which caused a couple reboots from single user mode. Somewhere
along the way xen (xenstored I think) got confused and wouldn't let me
have one of my partitions. The error message being:
Device /dev/vg1/domain-dns-data is mounted in a guest domain,
and so cannot be mounted now.
but given that I hadn't launched any new domains other than the one I
trying to get to start, this seemed clearly bogus.
To cut a long story short, I learned about the xenstore* commands and
though I'd mention them on the list.
WARNING. WARNING. WARNING. WARNING.
--------------------------------------
I'm sure you can use these to blow your feet off. I'm no expert
--------------------------------------
xenstore-list seems to be a crude equivalent of /bin/ls
e.g.
[root@xen5 scripts]# xenstore-list /
local
[root@xen5 scripts]# xenstore-list /local
domain
[root@xen5 scripts]# xenstore-list /local/domain
0
1
2
You can see we're walking down a psuedo filesystem. At this point it
forks to show the 3 domains running on this box. This list is
actually important, because there is no domain # 6, but we will
see later that there is a device "connected" to domain 6, which is
what was causing my problem.
If you go down the dom0 tree, you get things like this:
[root@xen5 scripts]# xenstore-list /local/domain/0/backend/vbd/1
2049
2065
2081
which is a list of the block devices for dom1
my problem is to some degree shown here:
[root@xen5 log]# xenstore-list /local/domain/0/backend/vbd
1
2
6
Here is where we see that dom0 has a backend device for domains
1, 2, and the phantom domain 6
As a quick aside, the proper/more traditional view into this realm
comes from xm block-list DOMID
[root@xen5 log]# xm block-list palladium
(2049 ((virtual-device 2049) (backend-id 0) (state 4) (backend
/local/domain/0/backend/vbd/1/2049) (ring-ref 8) (event-channel 9)))
(2065 ((virtual-device 2065) (backend-id 0) (state 4) (backend
/local/domain/0/backend/vbd/1/2065) (ring-ref 9) (event-channel 10)))
(2081 ((virtual-device 2081) (backend-id 0) (state 4) (backend
/local/domain/0/backend/vbd/1/2081) (ring-ref 10) (event-channel 11)))
which matches the view I showed above of the block devices for
dom1
when you get far enough down into this pseudo filesystem you hit
the equivalent of files, and you need to use xenstore-read to see
what is in those nodes. Here is an example that dumps all the
parameters for /local/domain/0/backend/vbd/1/2081
root@xen5 xenstore]# D=/local/domain/0/backend/vbd/1/2081; for d
in `xenstore-list $D`; do echo -n "$d: "; xenstore-read $D/$d; done
domain: palladium
frontend: /local/domain/1/device/vbd/2081
dev: sdc1
state: 4
params: /dev/vg1/pd-raid2
mode: w
frontend-id: 1
type: phy
physical-device: fd:4
hotplug-status: connected
sectors: 125829120
info: 0
sector-size: 512
Anyhow, returning to my issue... With the phantom domain 6...
[root@xen5 xenstore]# xenstore-list /local/domain/0/backend/vbd/6
2065
[root@xen5 xenstore]# xenstore-list /local/domain/0/backend/vbd/6/2065
physical-device
hotplug-status
[root@xen5 xenstore]# xenstore-read
/local/domain/0/backend/vbd/6/2065/physical-device
fd:6
[root@xen5 xenstore]# xenstore-read
/local/domain/0/backend/vbd/6/2065/hotplug-status
connected
fd:6 happens to match my target device (e.g 253,6)
[root@xen5 xen]# ll /dev/mapper/vg1-domain--dns--data
brw-rw---- 1 root disk 253, 6 Jul 3 22:21 /dev/mapper/vg1-domain--dns--data
So that seems to pretty clearly be my problem. WARNING, so far we have
just been poking at thinks, and looking around...
let's delete the node for that bogus backend device...
[root@xen5 xenstore]# xenstore-rm /local/domain/0/backend/vbd/6/2065
and what does xenstore-list say?
[root@xen5 xenstore]# xenstore-list /local/domain/0/backend/vbd/6/2065
xenstore-list: could not list path /local/domain/0/backend/vbd/6/2065
good, it seems to be gone, and voila, my "xm create" command is
no longer bitching about "Device /dev/vg1/domain-dns-data is mounted
in a guest domain..."
Pity that it took me more than an hour to work through it all, and
I had the service down for a more than to couple of minutes it
should have taken to sync the filesystems :(
Anyhow, I hope someone else finds this usefull. This was with xen
3.0.0, hopefully 3.0.2 is more robust about tracking device
assignments. AFAIK, it should actually have been simple enough to find
this issue, as the device was "attached" to a non-existant
domain, and inconsistent assignments like that should be easy
enough to find, but I haven't taken the time to build scripts to
try to handle that or make the process more friendly. (hhmm, 3.0.2
has a xenstore-ls which seems to be a tool which walks through the
xenstore-list output in a much more friendly manner... :)
My apologies if this is all documented and I'm just clued out
because I don't do texinfo type docs
-Tom
_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users
|
Home