MariaDB IPv6 only galera cluster

Storing data in a database at first looks like a trivial task. A database server is quickly configured and used. But when it comes to Availability of the data, it begins to get complex. To provide Availability, you also have to keep Integrity in mind. Both comes bundled together for your data. At all costs you must avoid a situation in a cluster, where not all nodes are confident of the data and its integrity.

To fullfil this task, we use MariaDB with the galera protocol, for a synchron database cluster infrastructure. If this setup should run in a IPv6-only network, additional challenges pops up.

Configuration example

The following example shows a example wsrep configuration to use within a IPv6-only environment:

[mysqld]
bind-address                   = ::

wsrep_on                       = ON
wsrep_provider                 = /usr/lib/galera/libgalera_smm.so
wsrep_cluster_address          = "gcomm://node1.example.tld,node2.example.tld,node3.example.tld"
wsrep_sst_method               = mariabackup
wsrep_sst_auth                 = "mariabackup:securepassword
wsrep_cluster_name             = "dbcluster"
wsrep_provider_options         = "gmcast.listen_addr=tcp://[::]:4567;ist.recv_addr=[2001:db8::1];gcache.size=512M;evs.suspect_timeout=PT5S"
wsrep_sst_receive_address      = "[2001:db8::1]:4444"
wsrep_node_address             = "[2001:db8::1]"
wsrep_node_name                = "node1"
binlog_format                  = ROW
default_storage_engine         = InnoDB
innodb_autoinc_lock_mode       = 2

[sst]
sockopt=",pf=ip6"

Precondition

Let's take a deeper look on the configration options step-by-step. As precondition we have the three cluster nodes node1 - node3, with the IPv6 addresses 2001:db8::1, 2001:db8::2 and 2001:db8::3.

Bind Address

bind-address                   = ::

When you build a cluster, the database should be reacheable from other systems, at least you want access the database cluster at some point ;) So that MariaDB also listens on IPv6 address, you should set the bind-address to :: or as alternative, to the IPv6 address of the interface you want the database to listen on, e.g. 2001:db8::1.

Enable WSREP

wsrep_on                       = ON
wsrep_provider                 = /usr/lib/galera/libgalera_smm.so

Enable galera replication and add the path to the galera library.

Cluster Replication

wsrep_cluster_address          = "gcomm://node1.example.tld,node2.example.tld,node3.example.tld"

This line should contain the list of cluster members. You can use FQDNs here. Depending on your DNS infrastructer it may make sense to add this FQDNs to the /etc/hosts file.

SST Method

wsrep_sst_method               = mariabackup
wsrep_sst_auth                 = "mariabackup:securepassword

For the SST (initial state transfer, or when the difference from the joining node is to large for a IST) we use the mariadb backup tool. You should create a user for mariabackup and grant privileges:

CREATE USER 'mariabackup'@'localhost' IDENTIFIED BY 'securepassword';
GRANT RELOAD, PROCESS, LOCK TABLES, REPLICATION CLIENT ON *.* TO 'mariabackup'@'localhost';

Cluster Name

wsrep_cluster_name             = "dbcluster"

This states the cluster name. If you run multiple clusters you should use a custom name for each cluster.

Cluster Commincation

wsrep_provider_options         = "gmcast.listen_addr=tcp://[::]:4567;ist.recv_addr=[2001:db8::1]"

This is also a important configuration line for IPv6-only networks. The config gmcast.listen_addr=tcp://[::]:4567 states that the galera should listen also on all IPv6 addresses. This should be equal to the bind-address. If the bind-address is configured to a specific interface IP, the same IP should be use in this config option, e.g. gmcast.listen_addr=tcp://[2001:db8::1]:4567. The option ist.recv_addr=[2001:db8::1] states on which IP address a IST update should be received. This should be the IP address of the node. E.g. on node3 this option would look like ist.recv_addr=[2001:db8::3].

SST Communication

wsrep_sst_receive_address      = "[2001:db8::1]:4444"

Like the ist.recv_addr option above, this line states at which IP a SST should be received. Again, this should be the IP address of the node.

Local Name and Address

wsrep_node_address             = "[2001:db8::1]"
wsrep_node_name                = "node1"

And in general, you should state the node IP address and hostname in the options above.

Cluster Log-Format and Storage-Engine

binlog_format                  = ROW
default_storage_engine         = InnoDB

Because of limitiations on the galera protocol, the binlog format must be set to ROW format, and only InnoDB storages engines are supported for replication.

SST socat IPv6 configuration

[sst]
sockopt=",pf=ip6"

Last but not least, socat which is used for the state transfers must know, it should use IPv6, which is done by the config above.