Solaris Cluster on the cheap - part 2

Posted by Ceri Davies Fri, 15 Jun 2007 08:44:00 GMT

Part 2 of a series on setting up Solaris Cluster for no money

Contents

• Prologue
• Part 1
• Part 2

At the end of the last article, I left you with an cluster that is still in “install mode”; since it’s a two node cluster the nodes can’t yet guarantee that they’ll know the status of the other cluster nodes should the heartbeat interconnects fail, so they refuse to do anything cluster related at this point. In order to get out of this situation, we need to add a quorum device.

Quorum Devices

The classical quorum device is a shared disk. However, I don’t have any shared disks spare and the point of this exercise is to avoid spending any money. Solaris Cluster provides another method, which is that of a quorum server.

Quorum Server

This is basically a service on a node outside the cluster¹ that the cluster nodes communicate their status to. It’s provided with all of the other Solaris Cluster bits and is installed by choosing “Quorum Server” from the installation menu.

At the end of the installation, you’re told to follow the post-installation instructions from the documentation – they’re as simple as “init 6” – after which the quorum server will come up on port 9000. Now, we can use it in our cluster:

# clq add -t quorum_server -p qshost=peasant.example.ac.uk -p port 9000 qserver

Now we can take the cluster out of install mode:

# scconf -c -q installmodeoff
# clq reset

Quorum Disks

As mentioned above, I don’t have any shared storage available for a test setup like this, so I had to make do with OpenSolaris backed iSCSI targets. These are not supported as quorum devices – which is why I mentioned the quorum server first – but they work (and if you want support, we’re not in the “cheap” realm any more, not by a long chalk). Setting up iSCSI targets is adequately discussed elsewhere² so I won’t repeat it.

To set up the initiator to see the iSCSI devices, run the following on each node, where 172.25.5.8 is the IP address of the system providing the targets.

# iscsiadm add discovery-address 172.25.5.8
# iscsiadm modify discovery -t enable

That’s it. Now you may need to refresh the cluster’s view of the devices. To do this, run the following on each node.

# cldevice refresh

Then on any one node:

# cldevice populate

You should now be able to see the iSCSI targets with both node paths in the cluster’s idea of what’s where:

# scdidadm -L

We can now finally use a quorum disk. Just add it in using the did.

# clq add d13

If you were previously using a quorum server, you can remove it now if you like.

# clq remove qserver
# clq reset

What’s next?

Now I have a cluster that can keep itself up if one of the nodes disappears, and some shared storage. This means that we can finally do something interesting, which will be in the next post on this subject.

[1] A common question is whether the quorum server can reside on one of the cluster nodes. While it can, this doesn’t provide high availability (rebooting the node hosting the quorum server will panic the other node) and misses the point.

[2] I believe that those happen to be the exact instructions used on the targets I used.