The following is a mini-howto, skip to the end of this mail for some
EVMS specific questions/problems.
So far I've not seen anybody yet on the Xen list talking about a
succesfull setup of EVMS with heartbeat 2. Setting up EVMS-HA is not
that terribly difficul.
If you're not experienced with heartbeat 2, perhaps read some basic
material on it at www.linux-ha.org especially on the administration
tools. Now you've 2 direct options you'd want to use EVMS-HA for:
2-node, DRDB-sync + EVMS-HA resource failover (possibly with HA-Xen
scripts): I haven't tested this yet, but it should be possible as far
as I've read.
n-node, iSCSI/FC-SAN + EVMS-HA; my current testing environment.
Notice there's a big difference, afaik when using DRDB you must
actually failover your resources to the other node. In a SAN based env
this is not the case as all nodes constantly have full I/O access,
which is why EVMS-HA should be usefull (at least I thought, read my
remarks on that at the end of this mail).
I installed plain SLES-10 copies (NOT using EVMS at installation
time), booted into Xen kernel in which all is configured. My intention
is/was only to use EVMS for Xen DomU volume management, not for local
disk management. Just for the ease of administration to keep those
strictly separated.
So to begin, make sure you've got all I/O access on all nodes to the
same resources (preferably with a unique name on all nodes for the
ease of administration, but this isn't necessary as EVMS can fix
this).
As local disk configuration is not EVMS aware, exclude the local disks
from EVMS management (in this case cciss mirrored disks, but this
could well be your hda/sda disks).
/etc/evms.conf:
sysfs_devices {
include = [ * ]
exclude = [ cciss* ]
...
}
Make sure admin_mode is off on all nodes, admin_mode has little to do
with admininistration but more with recovery/maintenance if things
have gone bad. More on this in the evms user guide:
http://evms.sourceforge.net/user_guide/
Setup heartbeat on both clusters to enable EVMS cluster awareness,
next to the usual ip and node configuration (which can be done using
yast in SLES10), just adding 2 lines to /etc/ha.d/ha.cf will do:
respawn root /sbin/evmsd
apiauth evms uid=hacluster,root
Start the cluster node by node:
/etc/init.d/hearbeat start
Make sure both sync and come up (keep an eye on /var/log/messages).
You can use the crmadmin tool to query the state of master and nodes.
Also usefull is cl_status for checking link & daemon status.
Note: If you're using bonding, you can run into some trouble here. Use
unicast for node sync, not multicast as somehow the Xen software
bridge doesn't fully cope with that yet (at least I didn't get it to
work, perhaps someone who did?).
When all nodes are up and running, start evmsgui (or evmsn, whichever
you prefer) on one of the nodes. If you click the settings menu and
find the option "Node administered" enabled, congratulations you've
got an cluster aware EVMS). Be sure to know some essentials of EVMS
(it's a little different from plain LVM2).
Create a Containter with the Cluster Segm Manager, select your
attached SAN storage objects (could be named sdb, sdc ...), choose
whichever node name, type shared storage and name the container
"c_sanstorage" for example.
You can patch through the SAN disks to EVMS volumes (see the disk list
in available objects). Don't do that, as these volumes will be fixed
in size (as they were originally presented from the SAN), instead use
EVMS for storage management. For this, create another container, this
time a LVM2 Region Manager, in which you store all the objects from
the CCM c_sanstorage (objects will have a name like c_sanstorage/sdb,
c_sanstorage/sdc, ...). Choose the extent size at will and name it
(vgsan for example).
Go into the Region tab, and create a region from the LVM2 freespace
named and sized at your will, for example domu01stor, domu02stor, ...
Save the configuration, all settings will now be applied and you will
find your correctly sized+named volumes in /dev/evms/c_sanstorage/
Now partition (if wanted) the evms disks like you used to
(fdisk),format,place domU's on it and launch.
As for the problems & remarks I've seen with this setup.
For the EVMS configuration to be updated at all nodes, you have to
select each node in "node administered" and select save for each node
(as only then the correct devices will be created on the node).
This could be a structural question, but... being cluster aware,
shouldn't the EVMS-HA combination (with CCM) provide locking on
volumes created beneath CCM? It is perfectly possible for me to
corrupt data on an EVMS volume on node 2 which volume is also mounted
on node 1. I expected some kind of locking to step up:
dom0_2# mount /dev/evms/c_sanstorage/domu01stor /mnt
failure: volume domu01stor is already mounted on node dom0_1
Or something amongst those lines. My initial thoughts: it had to do
with my CCM being "shared". But when creating a CCM as "private", the
same issues were possible! And even more remarkable, if I create a CCM
as private on node dom0_1 and I launch evmsgui om dom0_2 it recognizes
the CCM as private owned by dom0_2 ?!? This strikes me as very odd.
Are these problems due to faults in my procedure, if so let me know
please, or are they of a more structural nature (or perhaps SLES10
specific)?
They are kind of essential with Xen domains, you wouldn't want to boot
the same domain twice (one copy on dom0_1 and another running on
dom0_2) as data corruption is garanteed.
That is why this mail is crossposted at all 3 lists:
information for xen-users
technical questions for evms and linux-ha.
greetings
Tijl Van den Broeck
_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users
|
Home