<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"><title>Chapter 14. High Availability, Scalability, and DRBD</title><link rel="stylesheet" href="mysql-html.css" type="text/css"><meta name="generator" content="DocBook XSL Stylesheets V1.69.1"><link rel="start" href="index.html" title="MySQL 5.0 Reference Manual"><link rel="up" href="index.html" title="MySQL 5.0 Reference Manual"><link rel="prev" href="storage-engines.html" title="Chapter 13. Storage Engines"><link rel="next" href="replication.html" title="Chapter 15. Replication"></head><body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"><div class="navheader"><table width="100%" summary="Navigation header"><tr><th colspan="3" align="center">Chapter 14. High Availability, Scalability, and DRBD</th></tr><tr><td width="20%" align="left"><a accesskey="p" href="storage-engines.html">Prev</a> </td><th width="60%" align="center"> </th><td width="20%" align="right"> <a accesskey="n" href="replication.html">Next</a></td></tr></table><hr></div><div class="chapter" lang="en"><div class="titlepage"><div><div><h2 class="title"><a name="ha-overview"></a>Chapter 14. High Availability, Scalability, and DRBD</h2></div></div></div><div class="toc"><p><b>Table of Contents</b></p><dl><dt><span class="section"><a href="ha-overview.html#replication-drbd">14.1. Using MySQL with DRBD for High Availability</a></span></dt><dd><dl><dt><span class="section"><a href="ha-overview.html#replication-drbd-install">14.1.1. Configuring a MySQL and DRBD Environment</a></span></dt><dt><span class="section"><a href="ha-overview.html#replication-drbd-install-mysql">14.1.2. Configuring MySQL for DRBD</a></span></dt></dl></dd></dl></div><p>
When using MySQL you may need to ensure the availability or
scalability of your MySQL installation. Availability refers to the
ability to cope with, and if necessary recover from, failures on the
host, including failures of MySQL, the operating system, or the
hardware. Scalability refers to the ability to spread the load of
your application queries across multiple MySQL servers. As your
application and usage increases, you may need to spread the queries
for the application across multiple servers to improve response
times.
</p><p>
There are a number of solutions available for solving issues of
availability and scalability. The two primary solutions supported by
MySQL are MySQL Replication and MySQL Cluster. Further options are
available using third-party solutions such as DRBD (Distributed
Replicated Block Device) and Heartbeat, and more complex scenarios
can be solved through a combination of these technologies. These
tools work in different ways:
</p><div class="itemizedlist"><ul type="disc"><li><p>
<span class="emphasis"><em>MySQL Replication</em></span> enables statements and
data from one MySQL server instance to be replicated to another
MySQL server instance. Without using more complex setups, data
can only be replicated from a single master server to any number
of slaves. The replication is asynchronous, so the
synchronization does not take place in real time, and there is
no guarantee that data from the master will have been replicated
to the slaves.
</p><div class="itemizedlist"><ul type="circle"><li><p>
<span class="bold"><strong>Advantages</strong></span>
</p></li><li><p>
MySQL Replication is available on all platforms supported by
MySQL, and since it isn't operating system-specific it can
operate across different platforms.
</p></li><li><p>
Replication is asynchronous and can be stopped and restarted
at any time, making it suitable for replicating over slower
links, partial links and even across geographical
boundaries.
</p></li><li><p>
Data can be replicated from one master to any number of
slaves, making replication suitable in environments with
heavy reads, but light writes (for example, many web
applications), by spreading the load across multiple slaves.
</p></li><li><p>
<span class="bold"><strong>Disadvantages</strong></span>
</p></li><li><p>
Data can only be written to the master. In advanced
configurations, though, you can set up a multiple-master
configuration where the data is replicated around a ring
configuration.
</p></li><li><p>
There is no guarantee that data on master and slaves will be
consistent at a given point in time. Because replication is
asynchronous there may be a small delay between data being
written to the master and it being available on the slaves.
This can cause problems in applications where a write to the
master must be available for a read on the slaves (for
example a web application).
</p></li><li><p>
<span class="bold"><strong>Recommended uses</strong></span>
</p></li><li><p>
Scale-out solutions that require a large number of reads but
fewer writes (for example, web serving).
</p></li><li><p>
Logging/data analysis of live data. By replicating live data
to a slave you can perform queries on the slave without
affecting the operation of the master.
</p></li><li><p>
Online backup (availability), where you need an active copy
of the data available. You need to combine this with other
tools, such as custom scripts or Heartbeat. However, because
of the asynchronous architecture, the data may be
incomplete.
</p></li><li><p>
Offline backup. You can use replication to keep a copy of
the data. By replicating the data to a slave, you take the
slave down and get a reliable snapshot of the data (without
MySQL running), then restart MySQL and replication to catch
up. The master (and any other slaves) can be kept running
during this period.
</p></li></ul></div><p>
For information on setting up and configuring replication, see
<a href="replication.html" title="Chapter 15. Replication">Chapter 15, <i>Replication</i></a>.
</p></li><li><p>
<span class="emphasis"><em>MySQL Cluster</em></span> is a synchronous solution
that enables multiple MySQL instances to share database
information. Unlike replication, data in a cluster can be read
from or written to any node within the cluster, and information
will be distributed to the other nodes.
</p><div class="itemizedlist"><ul type="circle"><li><p>
<span class="bold"><strong>Advantages</strong></span>
</p></li><li><p>
Offers multiple read and write nodes for data storage.
</p></li><li><p>
Provides automatic failover between nodes. Only transaction
information for the active node being used is lost in the
event of a failure.
</p></li><li><p>
Data on nodes is instantaneously distributed to the other
data nodes.
</p></li><li><p>
<span class="bold"><strong>Disadvantages</strong></span>
</p></li><li><p>
Available on a limited range of platforms.
</p></li><li><p>
Nodes within a cluster should be connected via a LAN;
geographically separate nodes are not supported. However,
you can replicate from one cluster to another using MySQL
Replication, although the replication in this case is still
asynchronous.
</p></li><li><p>
<span class="bold"><strong>Recommended uses</strong></span>
</p></li><li><p>
Applications that need very high availability, such as
telecoms and banking.
</p></li><li><p>
Applications that require an equal or higher number of
writes compared to reads.
</p></li></ul></div><p>
For information on MySQL Cluster, see
<a href="mysql-cluster.html" title="Chapter 16. MySQL Cluster">Chapter 16, <i>MySQL Cluster</i></a>.
</p></li><li><p>
<span class="emphasis"><em>DRBD (Distributed Replicated Block Device)</em></span>
is a third-party solution from Linbit supported only on Linux.
DRBD creates a virtual block device (which is associated with an
underlying physical block device) that can be replicated from
the primary server to a secondary server. You create a
filesystem on the virtual block device, and this information is
then replicated, at the block level, to the secondary server.
</p><p>
Because the block device, not the data you are storing on it, is
being replicated the validity of the information is more
reliable than with data-only replication solutions. DRBD can
also ensure data integrity by only returning from a write
operation on the primary server when the data has been written
to the underlying physical block device on both the primary and
secondary servers.
</p><div class="itemizedlist"><ul type="circle"><li><p>
<span class="bold"><strong>Advantages</strong></span>
</p></li><li><p>
Provides high availability and data integrity across two
servers in the event of hardware or system failure.
</p></li><li><p>
Ensures data integrity by enforcing write consistency on the
primary and secondary nodes.
</p></li><li><p>
<span class="bold"><strong>Disadvantages</strong></span>
</p></li><li><p>
Only provides a method for duplicating data across the
nodes. Secondary nodes cannot use the DRBD device while data
is being replicated, and so the MySQL on the secondary node
cannot be simultaneously active.
</p></li><li><p>
Cannot provide scalability, since secondary nodes don't have
access to the secondary data.
</p></li><li><p>
<span class="bold"><strong>Recommended uses</strong></span>
</p></li><li><p>
High availability situations where concurrent access to the
data is not required, but instant access to the active data
in the event of a system or hardware failure is required.
</p></li></ul></div><p>
For information on configuring DRBD and configuring MySQL for
use with a DRBD device, see <a href="ha-overview.html#replication-drbd" title="14.1. Using MySQL with DRBD for High Availability">Section 14.1, “Using MySQL with DRBD for High Availability”</a>.
</p></li><li><p>
<span class="emphasis"><em>Heartbeat</em></span> is a third party software
solution for Linux. It is not a data replication or
synchronization solution, but a solution for monitoring servers
and switching active MySQL servers automatically in the event of
failure. Heartbeat needs to be combined with MySQL Replication
or DRBD to provide automatic failover.
</p></li></ul></div><p>
The information and suitability of the various technologies and
different scenarios is summarized in the table below.
</p><div class="informaltable"><a name="drbd-availability-comparison"></a><table border="1"><colgroup><col><col><col><col><col></colgroup><thead><tr><th>Requirements</th><th>MySQL Replication</th><th>MySQL Replication + Heartbeat</th><th>MySQL Heartbeat + DRBD</th><th>MySQL Cluster</th></tr></thead><tbody><tr><td><span class="bold"><strong>Availability</strong></span></td><td class="auto-generated"> </td><td class="auto-generated"> </td><td class="auto-generated"> </td><td class="auto-generated"> </td></tr><tr><td>Automated IP failover</td><td>No</td><td>Yes</td><td>Yes</td><td>No</td></tr><tr><td>Automated database failover</td><td>No</td><td>No</td><td>Yes</td><td>Yes</td></tr><tr><td>Typical failover time</td><td>User/script-dependent</td><td>Varies</td><td>< 30 seconds</td><td>< 3 seconds</td></tr><tr><td>Automatic resynchronization of data</td><td>No</td><td>No</td><td>Yes</td><td>Yes</td></tr><tr><td>Geographic redundancy support</td><td>Yes</td><td>Yes</td><td>Yes, when combined with MySQL Replication</td><td>Yes, when combined with MySQL Replication</td></tr><tr><td><span class="bold"><strong>Scalability</strong></span></td><td class="auto-generated"> </td><td class="auto-generated"> </td><td class="auto-generated"> </td><td class="auto-generated"> </td></tr><tr><td>Built-in load balancing</td><td>No</td><td>No</td><td>No</td><td>Yes</td></tr><tr><td>Supports Read-intensive applications</td><td>Yes</td><td>Yes</td><td>Yes, when combined with MySQL Replication</td><td>Yes</td></tr><tr><td>Supports Write-intensive applications</td><td>No</td><td>No</td><td>Yes</td><td>Yes</td></tr><tr><td>Maximum number of nodes per group</td><td>One master, multiple slaves</td><td>One master, multiple slaves</td><td>One active (primary), multiple passive (secondary) nodes</td><td>255</td></tr><tr><td>Maximum number of slaves</td><td>Unlimited (reads only)</td><td>Unlimited (reads only)</td><td>Unlimited (reads only)</td><td>Unlimited (reads only)</td></tr></tbody></table></div><div class="section" lang="en"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="replication-drbd"></a>14.1. Using MySQL with DRBD for High Availability</h2></div></div></div><div class="toc"><dl><dt><span class="section"><a href="ha-overview.html#replication-drbd-install">14.1.1. Configuring a MySQL and DRBD Environment</a></span></dt><dt><span class="section"><a href="ha-overview.html#replication-drbd-install-mysql">14.1.2. Configuring MySQL for DRBD</a></span></dt></dl></div><p>
The Distributed Replicated Block Device (DRBD) is a Linux Kernel
module that supports a distributed storage system. You can use DRBD
to share block devices between Linux servers and, in turn, share
filesystems and data.
</p><p>
DRBD implements a block device which can be used for storage and
which is replicated from a primary server to one or more secondary
servers. The distributed block device is handled by the DRBD
service. Writes to the DRBD block device are distributed among the
servers. Each DRBD service writes the information from the DRBD
block device to a local physical block device (hard disk).
</p><p>
On the primary, for example, the data writes are written both to the
underlying physical block device and distributed to the secondary
DRBD services. On the secondary, the writes received through DRBD
and written to the local physical block device. On both the primary
and the secondary, reads from the DRBD block device are handled by
the underlying physical block device. The information is shared
between the primary DRBD server and the secondary DRBD server
synchronously and at a block level, and this means that DRBD can be
used in high-availability solutions where you need failover support.
</p><div class="mediaobject"><img src="images/drbd-main.png" alt="DRBD Architecture"></div><p>
When used with MySQL, DRBD can be used to ensure availability in the
event of a failure. MySQL is configured to store information on the
DRBD block device, with one server acting as the primary and a
second machine available to operate as an immediate replacement in
the event of a failure.
</p><p>
For automatic failover support you can combine DRBD with the Linux
Heartbeat project, which will manage the interfaces on the two
servers and automatically configure the secondary (passive) server
to replace the primary (active) server in the event of a failure.
You can also combine DRBD with MySQL Replication to provide both
failover and scalability within your MySQL environment.
</p><p>
For information on how to configure DRBD and MySQL, including
Heartbeat support, see <a href="ha-overview.html#replication-drbd-install" title="14.1.1. Configuring a MySQL and DRBD Environment">Section 14.1.1, “Configuring a MySQL and DRBD Environment”</a>.
</p><p>
An FAQ for using DRBD and MySQL is available. See
<a href="faqs.html#faqs-mysql-drbd-heartbeat" title="A.14. MySQL 5.0 FAQ — MySQL, DRBD, and Heartbeat">Section A.14, “MySQL 5.0 FAQ — MySQL, DRBD, and Heartbeat”</a>.
</p><div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"><h3 class="title">Note</h3><p>
Because DRBD is a Linux Kernel module it is currently not
supported on platforms other than Linux.
</p></div><div class="section" lang="en"><div class="titlepage"><div><div><h3 class="title"><a name="replication-drbd-install"></a>14.1.1. Configuring a MySQL and DRBD Environment</h3></div></div></div><div class="toc"><dl><dt><span class="section"><a href="ha-overview.html#replication-drbd-install-os">14.1.1.1. Setting Up the OS for DRBD</a></span></dt><dt><span class="section"><a href="ha-overview.html#replication-drbd-install-drbd">14.1.1.2. Installing and Configuring DRBD</a></span></dt><dt><span class="section"><a href="ha-overview.html#replication-drbd-install-drbd-primary">14.1.1.3. Setting Up a DRBD Primary Node</a></span></dt><dt><span class="section"><a href="ha-overview.html#replication-drbd-install-drbd-secondary">14.1.1.4. Setting Up a DRBD Secondary Node</a></span></dt><dt><span class="section"><a href="ha-overview.html#replication-drbd-install-drbd-using">14.1.1.5. Monitoring and Managing Your DRBD Device</a></span></dt><dt><span class="section"><a href="ha-overview.html#replication-drbd-install-drbd-othercfg">14.1.1.6. Additional DRBD Configuration Options</a></span></dt></dl></div><p>
To set up DRBD, MySQL and Heartbeat you need to follow a number of
steps that affect the operating system, DRBD and your MySQL
installation.
</p><p>
Before starting the installation process, you should be aware of
the following information, terms and requirements on using DRBD:
</p><div class="itemizedlist"><ul type="disc"><li><p>
DRBD is a solution for enabling high-availability, and
therefore you need to ensure that the two machines within your
DRBD setup are as identically configured as possible so that
the secondary machine can act as a direct replacement for the
primary machine in the event of system failure.
</p></li><li><p>
DRBD works through two (or more) servers, each called a
<em class="firstterm">node</em>
</p></li><li><p>
The node that contains the primary data, has read/write access
to the data, and in an HA environment is the currently active
node is called the <em class="firstterm">primary</em>.
</p></li><li><p>
The server to which the data is replicated is designated as
<em class="firstterm">secondary</em>.
</p></li><li><p>
A collection of nodes that are sharing information are
referred to as a <em class="firstterm">DRBD cluster</em>.
</p></li><li><p>
For DRBD to operate you must have a block device on which the
information can be stored on <span class="emphasis"><em>each</em></span> DRBD
node. The <em class="firstterm">lower level</em> block device can
be a physical disk partition, a partition from a volume group
or RAID device or any other block device.
</p><p>
Typically you use a spare partition on which the physical data
will be stored . On the primary node, this disk will hold the
raw data that you want replicated. On the secondary nodes, the
disk will hold the data replicated to the secondary server by
the DRBD service. Ideally, the size of the partition on the
two DRBD servers should be identical, but this is not
necessary as long as there is enough space to hold the data
that you want distributed between the two servers.
</p></li><li><p>
For the distribution of data to work, DRBD is used to create a
logical block device that uses the lower level block device
for the actual storage of information. To store information on
the distributed device, a filesystem is created on the DRBD
logical block device.
</p></li><li><p>
When used with MySQL, once the filesystem has been created,
you move the MySQL data directory (including InnoDB data files
and binary logs) to the new filesystem.
</p></li><li><p>
When you set up the secondary DRBD server, you set up the
physical block device and the DRBD logical block device that
will store the data. The block device data is then copied from
the primary to the secondary server.
</p></li></ul></div><p>
The overview for the installation and configuration sequence is as
follows:
</p><div class="orderedlist"><ol type="1"><li><p>
First you need to set up your operating system and
environment. This includes setting the correct hostname,
updating the system and preparing the available packages and
software required by DRBD, and configuring a physical block
device to be used with the DRBD block device. See
<a href="ha-overview.html#replication-drbd-install-os" title="14.1.1.1. Setting Up the OS for DRBD">Section 14.1.1.1, “Setting Up the OS for DRBD”</a>.
</p></li><li><p>
Installing DRBD requires installing or compiling the DRBD
source code and then configuring the DRBD service to set up
the block devices that will be shared. See
<a href="ha-overview.html#replication-drbd-install-drbd" title="14.1.1.2. Installing and Configuring DRBD">Section 14.1.1.2, “Installing and Configuring DRBD”</a>.
</p></li><li><p>
Once DRBD has been configured, you must alter the
configuration and storage location of the MySQL data. See
<a href="ha-overview.html#replication-drbd-install-mysql" title="14.1.2. Configuring MySQL for DRBD">Section 14.1.2, “Configuring MySQL for DRBD”</a>.
</p></li></ol></div><div class="section" lang="en"><div class="titlepage"><div><div><h4 class="title"><a name="replication-drbd-install-os"></a>14.1.1.1. Setting Up the OS for DRBD</h4></div></div></div><p>
To set your Linux environment for using DRBD there are a number
of system configuration steps that you must follow.
</p><div class="itemizedlist"><ul type="disc"><li><p>
Make sure that the primary and secondary DRBD servers have
the correct hostname, and that the hostnames are unique. You
can verify this by using the <span><strong class="command">uname</strong></span>
command:
</p><pre class="programlisting">$ uname -n
drbd-master
</pre><p>
If the hostname is not set correctly then edit the
appropriate file (usually
<code class="filename">/etc/sysconfig/network</code> or
<code class="filename">/etc/conf.d/hostname</code>) and set the name
correctly.
</p></li><li><p>
Each DRBD node must have a unique IP address. Make sure that
the IP address information is set correctly within the
network configuration and that the hostname and IP address
has been set correctly within the
<code class="filename">/etc/hosts</code> file.
</p></li><li><p>
Although you can rely on the DNS or NIS system for host
resolving, in the event of a major network failure these
services may not be available. If possible, add the IP
address and hostname of each DRBD node into the /etc/hosts
file for each machine. This will ensure that the node
information can always be determined even if the DNS/NIS
servers are unavailable.
</p></li><li><p>
As a general rule, the faster your network connection the
better. Because the block device data is exchanged over the
network, everything that will be written to the local disk
on the DRBD primary will also be written to the network for
distribution to the DRBD secondary.
</p></li><li><p>
You must have a spare disk or disk partition that you can
use as the physical storage location for the DRBD data that
will be replicated. You do not have to have a complete disk
available, a partition on an existing disk is acceptable.
</p><p>
If the disk is unpartitioned, partition the disk using
<span><strong class="command">fdisk</strong></span>. Do not create a filesystem on the
new partition.
</p><p>
Remember that you must have a physical disk available for
the storage of the replicated information on each DRBD node.
Ideally the partitions that will be used on each node should
be of an identical size, although this is not strictly
necessary. Do, however, ensure that the physical partition
on the DRBD secondary is at least as big as the partitions
on the DRBD primary node.
</p></li><li><p>
If possible, upgrade your system to the latest available
Linux kernel for your distribution. Once the kernel has been
installed, you must reboot to make the kernel active. To use
DRBD you will also need to install the relevant kernel
development and header files that are required for building
kernel modules. Platform specification information for this
is available later in this section.
</p></li></ul></div><p>
Before you compile or install DRBD, you must make sure the
following tools and files are in place:
</p><div class="itemizedlist"><ul type="disc"><li><p>
Kernel header files
</p></li><li><p>
Kernel source files
</p></li><li><p>
GCC Compiler
</p></li><li><p>
<code class="literal">glib 2</code>
</p></li><li><p>
<span><strong class="command">flex</strong></span>
</p></li><li><p>
<span><strong class="command">bison</strong></span>
</p></li></ul></div><p>
Additionally, if you want to support secure communication
between your DRBD nodes then the following items will be
required:
</p><div class="itemizedlist"><ul type="disc"><li><p>
OpenSSL
</p></li><li><p>
GNU TLS
</p></li><li><p>
GNU Crypt
</p></li></ul></div><p>
Here are some operating system specific tips for setting up your
installation:
</p><div class="itemizedlist"><ul type="disc"><li><p>
<span class="bold"><strong>Tips for Red Hat (including CentOS and
Fedora)</strong></span>:
</p><p>
Use <span><strong class="command">up2date</strong></span> or <span><strong class="command">yum</strong></span> to
update and install the latest kernel and kernel header
files:
</p><pre class="programlisting"># up2date kernel-smp-devel kernel-smp</pre><p>
Reboot. If you are going to build DRBD from source, then
update your system with the required development packages
</p><pre class="programlisting"># up2date glib-devel openssl-devel libgcrypt-devel glib2-devel \
pkgconfig ncurses-devel rpm-build rpm-devel redhat-rpm-config gcc \
gcc-c++ bison flex gnutls-devel lm_sensors-devel net-snmp-devel \
python-devel bzip2-devel libselinux-devel perl-DBI</pre><p>
If you are going to use the pre-built DRBD RPMs:
</p><pre class="programlisting"># up2date gnutls lm_sensors net-snmp ncurses libgcrypt glib2 openssl glib</pre></li><li><p>
<span class="bold"><strong>Tips for Debian, Ubuntu,
Kubuntu</strong></span>:
</p><p>
Use <span><strong class="command">apt-get</strong></span> to install the kernel
packages
</p><pre class="programlisting"># apt-get install linux-headers linux-image-server</pre><p>
If you are going to use the pre-built Debian packages for
DRBD then you should not need any additional packages.
</p><p>
If you want to build DRBD from source, you will need to use
the following command to install the required components:
</p><pre class="programlisting"># apt-get install devscripts flex bison build-essential \
dpkg-dev kernel-package debconf-utils dpatch debhelper \
libnet1-dev e2fslibs-dev libglib2.0-dev automake1.9 \
libgnutls-dev libtool libltdl3 libltdl3-dev</pre></li><li><p>
<span class="bold"><strong>Tips for Gentoo</strong></span>:
</p><p>
Gentoo is a source based Linux distribution and therefore
many of the source files and components that you will need
are either already installed or will be installed
automatically by <span><strong class="command">emerge</strong></span>.
</p><p>
To install DRBD 0.8.x, you must unmask the
<code class="literal">sys-cluster/drbd</code> build by adding the
following line to
<code class="filename">/etc/portage/package.keywords</code>:
</p><pre class="programlisting">sys-cluster/drbd ~x86
sys-cluster/drbd-kernel ~x86</pre><p>
If your kernel does not already have the userspace to
kernelspace linker enabled, then you will need to rebuild
the kernel with this option. The best way to do this is to
use <span><strong class="command">genkernel</strong></span> with the
<code class="option">--menuconfig</code> option to select the option
and then rebuild the kernel. For example, at the command
line as <code class="literal">root</code>:
</p><pre class="programlisting"># genkernel --menuconfig all</pre><p>
Then through the menu options, select <span class="guimenu">Device
Drivers</span>, <span class="guimenu">Connector - unified userspace
$lt;-> kernelspace linker</span> and finally press 'y'
or 'space' to select the <span class="guimenu">Connector - unified
userspace $lt;-> kernelspace linker</span> option.
Then exit the menu configuration. The kernel will be rebuilt
and installed. If this is a new kernel, make sure you update
your bootloader accordingly. Now reboot to enable the new
kernel.
</p><p>
You can now <span><strong class="command">emerge</strong></span> DRBD 0.8.x into your
Gentoo installation:
</p><pre class="programlisting"># emerge drbd</pre><p>
Once <code class="literal">drbd</code> has been downloaded and
installed, you need to decompress and copy the default
configuration file from
<code class="filename">/usr/share/doc/drbd-8.0.7/drbd.conf.bz2</code>
into <code class="filename">/etc/drbd.conf</code>.
</p></li></ul></div></div><div class="section" lang="en"><div class="titlepage"><div><div><h4 class="title"><a name="replication-drbd-install-drbd"></a>14.1.1.2. Installing and Configuring DRBD</h4></div></div></div><p>
To install DRBD you can choose either the pre-built binary
installation packages or you can use the source packages and
build from source. If you want to build from source you must
have installed the source and development packages.
</p><p>
If you are installing using a binary distribution then you must
ensure that the kernel version number of the binary package
matches your currently active kernel. You can use
<span><strong class="command">uname</strong></span> to find out this information:
</p><pre class="programlisting">$ uname -r
2.6.20-gentoo-r6</pre><p>
To build from the sources, download the source
<code class="literal">tar.gz</code> package, extract the contents and then
follow the instructions within the <code class="filename">INSTALL</code>
file.
</p><p>
Once DRBD has been built and installed, you need to edit the
<code class="filename">/etc/drbd.conf</code> file and then run a number
of commands to build the block device and set up the
replication.
</p><p>
Although the steps below are split into those for the primary
node and the secondary node, it should be noted that the
configuration files for all nodes should be identical, and many
of the same steps have to be repeated on each node to enable the
DRBD block device.
</p></div><div class="section" lang="en"><div class="titlepage"><div><div><h4 class="title"><a name="replication-drbd-install-drbd-primary"></a>14.1.1.3. Setting Up a DRBD Primary Node</h4></div></div></div><p>
To set up a DRBD primary node you need to configure the DRBD
service, create the first DRBD block device and then create a
filesystem on the device so that you can store files and data.
</p><p>
The DRBD configuration file
(<code class="filename">/etc/drbd.conf</code>) defined a number of
parameters for your DRBD configuration, including the frequency
of updates and block sizes, security information and the
definition of the DRBD devices that you want to create.
</p><p>
The key elements to configure are the <code class="literal">on</code>
sections which specify the configuration of each node.
</p><p>
To follow the configuration, the sequence below shows only the
changes from the default <code class="filename">drbd.conf</code> file.
Configurations within the file can be both global or tied to
specific resource.
</p><div class="orderedlist"><ol type="1"><li><p>
Set the synchronization rate between the two nodes. You
should keep this in check compared to the speed of your
network connection. Gigabit Ethernet can support up to 125
MB/second, 100Mbps Ethernet slightly less than a tenth of
that (12MBps). If you are using a shared network connection,
rather than a dedicated, then you should gauge accordingly.
</p><p>
To set the synchronization rate, edit the
<code class="literal">rate</code> setting within the
<code class="literal">syncer</code> block:
</p><pre class="programlisting">syncer {
rate 10M;
}</pre></li><li><p>
Set up some basic authentication. DRBD supports a simple
password hash exchange mechanism. This helps to ensure that
only those hosts with the same shared secret are able to
join the DRBD node group.
</p><pre class="programlisting">cram-hmac-alg “sha1”;
shared-secret "<em class="replaceable"><code>shared-string</code></em>";</pre></li><li><p>
Now you must configure the host information. Remember that
you must have the node information for the primary and
secondary nodes in the <code class="filename">drbd.conf</code> file
on each host. You need to configure the following
information for each node:
</p><div class="itemizedlist"><ul type="disc"><li><p>
<code class="literal">device</code> — the path of the
logical block device that will be created by DRBD.
</p></li><li><p>
<code class="literal">disk</code> — the block device that
will be used to store the data.
</p></li><li><p>
<code class="literal">address</code> — the IP address and
port number of the host that will hold this DRBD device.
</p></li><li><p>
<code class="literal">meta-disk</code> — the location where
the metadata about the DRBD device will be stored. You
can set this to <code class="literal">internal</code> and DRBD
will use the physical block device to store the
information, by recording the metadata within the last
sections of the disk. The exact size will depend on the
size of the logical block device you have created, but
it may involve up to 128MB.
</p></li></ul></div><p>
A sample configuration for our primary server might look
like this:
</p><pre class="programlisting">on drbd-master {
device /dev/drbd0;
disk /dev/hdd1;
address 192.168.0.240:8888;
meta-disk internal;
}</pre><p>
The <code class="literal">on</code> configuration block should be
repeated for the secondary node (and any further) nodes:
</p><pre class="programlisting">on drbd-slave {
device /dev/drbd0;
disk /dev/hdd1;
address 192.168.0.241:8888;
meta-disk internal;
}</pre><p>
The IP address of eac <code class="literal">on</code> block must match
the IP address of the corresponding host. Do not set this
value to the IP address of the corresponding primary or
secondary in each case.
</p></li><li><p>
Before starting the primary node, you should create the
metadata for the devices:
</p><pre class="programlisting"># drbdadm create-md all</pre></li><li><p>
You are now ready to start DRBD:
</p><pre class="programlisting"># /etc/init.d/drbd start</pre><p>
DRBD should now start and initialize, creating the DRBD
devices that you have configured.
</p></li><li><p>
DRBD creates a standard block device - to make it usable,
you must create a filesystem on the block device just as you
would with any standard disk partition. Before you can
create the filesystem, you must mark the new device as the
primary device (i.e. where the data will be written and
stored), and and initialize the device. Because this is a
destructive operation, you must specify the command line
option to overwrite the raw data:
</p><pre class="programlisting"># drbdadm -- --overwrite-data-of-peer primary all</pre><p>
If you are using a version of DRBD earlier than 0.8.0, then
you need to use a different command-line option:
</p><pre class="programlisting"># drbdadm -- --do-what-I-say primary all</pre><p>
Now create a filesystem using your chosen filesystem type:
</p><pre class="programlisting"># mkfs.ext3 /dev/drbd0</pre></li><li><p>
You can now mount the filesystem and if necessary copy files
to the mount point:
</p><pre class="programlisting"># mkdir /mnt/drbd
# mount /dev/drbd0 /mnt/drbd
# echo "DRBD Device" >/mnt/drbd/samplefile</pre></li></ol></div><p>
Your primary node is now ready to use. You should now configure
your secondary node or nodes.
</p></div><div class="section" lang="en"><div class="titlepage"><div><div><h4 class="title"><a name="replication-drbd-install-drbd-secondary"></a>14.1.1.4. Setting Up a DRBD Secondary Node</h4></div></div></div><p>
The configuration process for setting up a secondary node is the
same as for the primary node, except that you do not have to
create the filesystem on the secondary node device, as this
information will automatically be transferred from the primary
node.
</p><p>
To set up a secondary node:
</p><div class="orderedlist"><ol type="1"><li><p>
Copy the <code class="filename">/etc/drbd.conf</code> file from your
primary node to your secondary node. It should already
contain all the information and configuration that you need,
since you had to specify the secondary node IP address and
other information for the primary node configuration.
</p></li><li><p>
Create the DRBD metadata on the underlying disk device:
</p><pre class="programlisting"># drbdadm create-md all</pre></li><li><p>
Start DRBD:
</p><pre class="programlisting"># /etc/init.d/drbd start</pre></li></ol></div><p>
Once DRBD has started, it will start the copy the data from the
primary node to the secondary node. Even with an empty
filesystem this will take some time, since DRBD is copying the
block information from a block device, not simply copying the
filesystem data.
</p><p>
You can monitor the progress of the copy between the primary and
secondary nodes by viewing the output of
<code class="filename">/proc/drbd</code>:
</p><pre class="programlisting"># cat /proc/drbd
version: 8.0.4 (api:86/proto:86)
SVN Revision: 2947 build by root@drbd-master, 2007-07-30 16:43:05
0: cs:SyncSource st:Primary/Secondary ds:UpToDate/Inconsistent C r---
ns:252284 nr:0 dw:0 dr:257280 al:0 bm:15 lo:0 pe:7 ua:157 ap:0
[==>.................] sync'ed: 12.3% (1845088/2097152)K
finish: 0:06:06 speed: 4,972 (4,580) K/sec
resync: used:1/31 hits:15901 misses:16 starving:0 dirty:0 changed:16
act_log: used:0/257 hits:0 misses:0 starving:0 dirty:0 changed:0</pre></div><div class="section" lang="en"><div class="titlepage"><div><div><h4 class="title"><a name="replication-drbd-install-drbd-using"></a>14.1.1.5. Monitoring and Managing Your DRBD Device</h4></div></div></div><p>
Once the primary and secondary machines are configured and
synchronized, you can get the status information about your DRBD
device by viewing the output from
<code class="filename">/proc/drbd</code>:
</p><pre class="programlisting"># cat /proc/drbd
version: 8.0.4 (api:86/proto:86)
SVN Revision: 2947 build by root@drbd-master, 2007-07-30 16:43:05
0: cs:Connected st:Primary/Secondary ds:UpToDate/UpToDate C r---
ns:2175704 nr:0 dw:99192 dr:2076641 al:33 bm:128 lo:0 pe:0 ua:0 ap:0
resync: used:0/31 hits:134841 misses:135 starving:0 dirty:0 changed:135
act_log: used:0/257 hits:24765 misses:33 starving:0 dirty:0 changed:33</pre><p>
The first line provides the version/revision and build
information.
</p><p>
The second line starts the detailed status information for an
individual resource. The individual field headings are as
follows:
</p><div class="itemizedlist"><ul type="disc"><li><p>
cs — connection state
</p></li><li><p>
st — node state (local/remote)
</p></li><li><p>
ld — local data consistency
</p></li><li><p>
ds — data consistency
</p></li><li><p>
ns — network send
</p></li><li><p>
nr — network receive
</p></li><li><p>
dw — disk write
</p></li><li><p>
dr — disk read
</p></li><li><p>
pe — pending (waiting for ack)
</p></li><li><p>
ua — unack'd (still need to send ack)
</p></li><li><p>
al — access log write count
</p></li></ul></div><p>
In the previous example, the information shown indicates that
the nodes are connected, the local node is the primary (because
it is listed first), and the local and remote data is up to date
with each other. The remainder of the information is statistical
data about the device, and the data exchanged that kept the
information up to date.
</p><p>
For administration, the main command is
<span><strong class="command">drbdadm</strong></span>. There are a number of commands
supported by this tool the control the connectivity and status
of the DRBD devices.
</p><p>
The most common commands are those to set the primary/secondary
status of the local device. You can manually set this
information for a number of reasons, including when you want to
check the physical status of the secondary device (since you
cannot mount a DRBD device in primary mode), or when you are
temporarily moving the responsibility of keeping the data in
check to a different machine (for example, during an upgrade or
physical move of the normal primary node). You can set state of
all local device to be the primary using this command:
</p><pre class="programlisting"># drbdadm primary all</pre><p>
Or switch the local device to be the secondary using:
</p><pre class="programlisting"># drbdadm secondary all</pre><p>
To change only a single DRBD resource, specify the resource name
instead of <code class="literal">all</code>.
</p><p>
You can temporarily disconnect the DRBD nodes:
</p><pre class="programlisting"># drbdadm disconnect all</pre><p>
Reconnect them using <code class="literal">connect</code>:
</p><pre class="programlisting"># drbdadm connect all</pre><p>
For other commands and help with <span><strong class="command">drbdadm</strong></span> see
the DRBD documentation.
</p></div><div class="section" lang="en"><div class="titlepage"><div><div><h4 class="title"><a name="replication-drbd-install-drbd-othercfg"></a>14.1.1.6. Additional DRBD Configuration Options</h4></div></div></div><p>
Additional options you may want to configure:
</p><div class="itemizedlist"><ul type="disc"><li><p>
<code class="literal">protocol</code> — specifies the level of
consistency to be used when information is written to the
block device. The option is similar in principle to the
<a href="storage-engines.html#option_mysqld_innodb_flush_log_at_trx_commit"><code class="literal">innodb_flush_log_at_trx_commit</code></a>
option within MySQL. Three levels are supported:
</p><div class="itemizedlist"><ul type="circle"><li><p>
<code class="literal">A</code> — data is considered written
when the information reaches the TCP send buffer and the
local physical disk. There is no guarantee that the data
has been written to the remote server or the remote
physical disk.
</p></li><li><p>
<code class="literal">B</code> — data is considered written
when the data has reached the local disk and the remote
node's network buffer. The data has reached the remote
server, but there is no guarantee it has reached the
remote server's physical disk.
</p></li><li><p>
<code class="literal">C</code> — data is considered written
when the data has reached the local disk and the remote
node's physical disk.
</p></li></ul></div><p>
The preferred and recommended protocol is C, as it is the
only protocol which ensures the consistency of the local and
remote physical storage.
</p></li><li><p>
<code class="literal">on-io-error</code> — specifies what should
happen when there is an error with the physical block
device. Suitable values include <code class="literal">panic</code>,
which causes the node to leave the DRBD cluster by causing a
kernel panic (and which would trigger automatic failover if
you were using Heartbeat); and <code class="literal">detach</code>,
which switches DRBD into a diskless mode (that is, data is
not written to the lower level block device).
</p></li><li><p>
<code class="literal">size</code> — if you do not want to use
the entire partition space with your DRBD block device then
you can specify the size of the DRBD device to be created.
The size specification can include a quantifier. For
example, to set the maximum size of the DRBD partition to
1GB you would use:
</p><pre class="programlisting">size 1G;</pre></li></ul></div><p>
With the configuration file suitably configured and ready to
use, you now need to populate the lower-level device with the
metadata information, and then start the DRBD service.
</p></div></div><div class="section" lang="en"><div class="titlepage"><div><div><h3 class="title"><a name="replication-drbd-install-mysql"></a>14.1.2. Configuring MySQL for DRBD</h3></div></div></div><p>
Once you have configured DRBD and have an active DRBD device and
filesystem, you can configure MySQL to use the chosen device to
store the MySQL data.
</p><p>
When performing a new installation of MySQL, you can either select
to install MySQL entirely onto the DRBD device, or just configure
the data directory to be located on the new filesystem.
</p><p>
In either case, the files and installation must take place on the
primary node, because that is the only DRBD node on which you can
mount the DRBD device filesystem as read/write.
</p><p>
You should store the following files and information on your DRBD
device:
</p><div class="itemizedlist"><ul type="disc"><li><p>
MySQL data files, including the binary log, and InnoDB data
files.
</p></li><li><p>
MySQL configuration file (<code class="filename">my.cnf</code>).
</p></li></ul></div><p>
To set up MySQL to use your new DRBD device and filesystem:
</p><div class="orderedlist"><ol type="1"><li><p>
If you are migrating an existing MySQL installation, stop
MySQL:
</p><pre class="programlisting">$ mysqladmin shutdown</pre></li><li><p>
Copy the <code class="filename">my.cnf</code> onto the DRBD device. If
you are not already using a configuration file, copy one of
the sample configuration files from the MySQL distribution.
</p><pre class="programlisting"># mkdir /mnt/drbd/mysql
# cp /etc/my.cnf /mnt/drbd/mysql</pre></li><li><p>
Copy your MySQL data directory to the DRBD device and mounted
filesystem.
</p><pre class="programlisting"># cp -R /var/lib/mysql /drbd/mysql/data</pre></li><li><p>
Edit the configuration file to reflect the change of directory
by setting the value of the <code class="literal">datadir</code> option.
If you have not already enabled the binary log, also set the
value of the <code class="literal">log-bin</code> option.
</p><pre class="programlisting">datadir = /drbd/mysql/data
log-bin = mysql-bin</pre></li><li><p>
Create a symbolic link from <code class="filename">/etc/my.cnf</code>
to the new configuration file on the DRBD device filesystem.
</p><pre class="programlisting"># ln -s /drbd/mysql/my.cnf /etc/my.cnf</pre></li><li><p>
Now start MySQL and check that the data that you copied to the
DRBD device filesystem is present.
</p><pre class="programlisting"># /etc/init.d/mysql start</pre></li></ol></div><p>
Your MySQL data should now be located on the filesystem running on
your DRBD device. The data will be physically stored on the
underlying device that you configured for the DRBD device.
Meanwhile, the content of your MySQL databases will be copied to
the secondary DRBD node.
</p><p>
Note that you cannot access the information on your secondary
node, as a DRBD device working in secondary mode is not available
for use.
</p></div></div></div><div class="navfooter"><hr><table width="100%" summary="Navigation footer"><tr><td width="40%" align="left"><a accesskey="p" href="storage-engines.html">Prev</a> </td><td width="20%" align="center"> </td><td width="40%" align="right"> <a accesskey="n" href="replication.html">Next</a></td></tr><tr><td width="40%" align="left" valign="top">Chapter 13. Storage Engines </td><td width="20%" align="center"><a accesskey="h" href="index.html">Home</a></td><td width="40%" align="right" valign="top"> Chapter 15. Replication</td></tr></table></div></body></html>
distrib
>
Mandriva
>
2008.1
>
x86_64
>
media
>
main-testing
>
by-pkgid
>
b1e2421f2416edfc24c5845fbc1c5a2e
>
files
>
11
