Docker Galera Cluster

When I was considering switching jobs a few years ago, I had some interest for a database admin position that was heavy on Galera. I had a lot of MySQL replication experience, but had never touched Galera. It still sounded like an interesting opportunity and I was confident I could do the job, but I needed to learn some Galera!

This is an area where Docker shines. I was able to get a three node cluster up and running in containers to experiment with pretty quick, but there were some gotchas that I had to deal with to make things run smoothly.

First, Galera nodes reference each other by IP address not by host. For this experimental environment to work, I needed to make sure our Galera nodes get the same IP address every time.

Second, when the cluster is brought down the last node to go offline will be the only node that’s safe to bootstrap the cluster. If the cluster is brought up with any other node first it will throw errors by design.

To deal with the IP and bootstrapping, I wrote a bash script to cleanly start and stop the cluster:

#!/bin/bash

action="$1"		#up/down

if [ -z "$action" ]; then
	echo "Usage: $0 [up/down]"
	exit 1
fi

if [ "$action" = "up" ]; then
	docker network create --subnet="192.168.128.0/17" galeranet
fi

case "$action" in
	up)
		action+=" -d"
		docker compose -p galera1 -f docker-compose-db1.yaml $action; sleep 10
		docker compose -p galera2 -f docker-compose-db2.yaml $action
		docker compose -p galera3 -f docker-compose-db3.yaml $action
		;;
	down)
		for i in $(seq 3 -1 1); do
			if [ $i = 1 ]; then
				sleep 10 
			fi
			docker compose -p galera${i} down
		done
		;;
esac

if [[ "$action" = "down" && $? -eq 0 ]]; then
	docker network rm galeranet
fi

exit 0

The network is created using the same subnet and the IP addresses for each node is defined in their compose file. The -p flag gives predictable names to the containers so I can reference them when bringing down the cluster. When bringing up the cluster it just starts them in numeric order. When issuing the down argument the cluster will be brought down in reverse.

This works pretty well. Of course, there are times where I forget to down the cluster before closing my laptop. Once the battery dies and crashes the cluster, no nodes will be set as safe to bootstrap and at that point I have to manually set a node as safe. That’s just Galera doing it’s job.

I actually still run most Docker stuff I’m playing around with that requires MySQL through this cluster. It’s a good way to keep learning Galera by load balancing things across the cluster or testing database level failover.

If you’re looking to do some learning or testing with Galera, clone the galera-docker-cluster repo and you should be off and running pretty quick.

Copyright 2024. All rights reserved.

Built with Hugo and Bulma.