MariaDB IPv6 only galera cluster

Storing data in a database at first looks like a trivial task. A database server is quickly configured and used. But when it comes to Availability of the data, it begins to get complex. To provide Availability, you also have to keep Integrity in mind. Both come bundled together for your data. At all costs you must avoid a situation in a cluster, where not all nodes are confident of the data and its integrity.

To fulfil this task, we use MariaDB with the Galera protocol, for a synchronous database cluster infrastructure. If this setup should run in an IPv6-only network, additional challenges pop up.

Configuration example

The following example shows an example wsrep configuration to use within an IPv6-only environment:

[mysqld]
bind-address                   = ::

wsrep_on                       = ON
wsrep_provider                 = /usr/lib/galera/libgalera_smm.so
wsrep_cluster_address          = "gcomm://node1.example.tld,node2.example.tld,node3.example.tld"
wsrep_sst_method               = mariabackup
wsrep_sst_auth                 = "mariabackup:securepassword"
wsrep_cluster_name             = "dbcluster"
wsrep_provider_options         = "gmcast.listen_addr=tcp://[::]:4567;ist.recv_addr=[2001:db8::1];gcache.size=512M;evs.suspect_timeout=PT5S"
wsrep_sst_receive_address      = "[2001:db8::1]:4444"
wsrep_node_address             = "[2001:db8::1]"
wsrep_node_name                = "node1"
binlog_format                  = ROW
default_storage_engine         = InnoDB
innodb_autoinc_lock_mode       = 2

[sst]
sockopt=",pf=ip6"

Precondition

Let's take a deeper look at the configuration options step-by-step. As a precondition, we have the three cluster nodes node1 - node3, with the IPv6 addresses 2001:db8::1, 2001:db8::2 and 2001:db8::3.

Bind Address

bind-address                   = ::

When you build a cluster, the database should be reachable from other systems, at least you want to access the database cluster at some point ;) So that MariaDB also listens on an IPv6 address, you should set the bind-address to :: or as alternative, to the IPv6 address of the interface you want the database to listen on, e.g. 2001:db8::1.

Enable WSREP

wsrep_on                       = ON
wsrep_provider                 = /usr/lib/galera/libgalera_smm.so

Enable Galera replication and add the path to the Galera library.

Cluster Replication

wsrep_cluster_address          = "gcomm://node1.example.tld,node2.example.tld,node3.example.tld"

This line should contain the list of cluster members. You can use FQDNs here. Depending on your DNS infrastructure, it may make sense to add these FQDNs to the /etc/hosts file.

SST Method

wsrep_sst_method               = mariabackup
wsrep_sst_auth                 = "mariabackup:securepassword"

For the SST (initial state transfer, or when the difference from the joining node is too large for an IST) we use the mariabackup tool. You should create a user for mariabackup and grant privileges:

CREATE USER 'mariabackup'@'localhost' IDENTIFIED BY 'securepassword';
GRANT RELOAD, PROCESS, LOCK TABLES, REPLICATION CLIENT ON *.* TO 'mariabackup'@'localhost';

Cluster Name

wsrep_cluster_name             = "dbcluster"

This states the cluster name. If you run multiple clusters you should use a custom name for each cluster.

Cluster Communication

wsrep_provider_options         = "gmcast.listen_addr=tcp://[::]:4567;ist.recv_addr=[2001:db8::1]"

This is also an important configuration line for IPv6-only networks. The config gmcast.listen_addr=tcp://[::]:4567 states that Galera should also listen on all IPv6 addresses. This should be equal to the bind-address. If the bind-address is configured to a specific interface IP, the same IP should be used in this config option, e.g. gmcast.listen_addr=tcp://[2001:db8::1]:4567. The option ist.recv_addr=[2001:db8::1] states on which IP address an IST update should be received. This should be the IP address of the node. E.g. on node3 this option would look like ist.recv_addr=[2001:db8::3].

SST Communication

wsrep_sst_receive_address      = "[2001:db8::1]:4444"

Like the ist.recv_addr option above, this line states at which IP a SST should be received. Again, this should be the IP address of the node.

Local Name and Address

wsrep_node_address             = "[2001:db8::1]"
wsrep_node_name                = "node1"

And in general, you should state the node IP address and hostname in the options above.

Cluster Log-Format and Storage-Engine

binlog_format                  = ROW
default_storage_engine         = InnoDB

Because of limitations on the Galera protocol, the binlog format must be set to ROW format, and only InnoDB storage engines are supported for replication.

SST socat IPv6 configuration

[sst]
sockopt=",pf=ip6"

Last but not least, socat, which is used for the state transfers, must know that it should use IPv6, which is done by the config above.