In this tutorial we will discuss about the Linux MultiPath Concepts and Implementation in detail.
What is a Path? And why we need Multipathing?
A Path or Multipath is a connection between a Server Nodes and Storage Arrays as shown in figure.
In Above configuration, there is one I/O path that goes through linux server HBA1, SAN switch and HBA of Storage controller. There are many possible single point of failure in this type of configuration such as:
- HBA failure
- FC cable failure
- SAN switch Failure
- Storage Array Port failure.
Due to failure of any component, server node loose connectivity to storage array and as a result end user/application wont able to get data which affect business. Hence, as a solution of such a single point of failures, Multipathing concepts comes in picture.
What is Multipathing?
- Multipathing allows you to configure multiple I/O paths between Server nodes and Storage arrays into a single device. These I/O paths are physical SAN Connections that can include separate cables, switches, and controllers. Multipathing aggregates the I/O paths and create new device that consists of the aggregated Paths.
- In above configuration, we have multiple I/O Path to avoid the single point of failure.
- Main benefits of Multipathing is to provide redundancy and increased throughput.
- Today many Multipathing solution provided by different vendor for Linux OS e.g.
- EMC power path
- Symantec’s VxDMP
What is DM-Multipathing?
Device Mapper Multipathing or DM-Multipathing is a Linux native multipath tool, which allows you to configure multiple I/O paths between server nodes and storage arrays into a single device. Regardless of the vendor hardware in use, device mapper creates a blockdevice under /dev/mapper/ for each LUN attached to the system. DM-Multipath is used to provides –
- Redundancy: DM multipath can provide failover in an active/passive configuration. In this type of configuration, only half the paths are used at any time for I/O. if any element of an I/O path such as cable, switch or controller fails, DM multipath switches to an alternate path.
- Improved Performance: DM multipath can be configured in active/active mode, where I/O is spread over the paths in a round-robin fashion. In some configurations, DM-Multipath can detect loading on the I/O paths and dynamically rebalance the load.
Before discussing DM-Multipath in details, let us explore the concepts of Device Mapper.
Figure: Device-mapper Kernel Architecture (taken from www.redhat.com)
Device mapper consists of three major components:
- Mapped device
- Mapping table
- Target device
- Mapped device is a logical device provided by device-mapper driver.
- It provides an interface to operate on.
- Device-mapper userspace facilities operate on it to create/remove mapped devices.
- g. logical volumes in LVM2 and pseudo disks in dm-multipath.
- Mapping table provides a mapping from a mapped device to target devices.
- One table can be seen as a group of variables including mapped device’s starting address, length and target device’s physical device, starting address, length. The unit used here is a sector (512 bytes).
- The command “dmsetup create” will create such a table with parameters provided.
- Target device is a modularized plugin. The device-mapper filters and redirects I/O requests with it. Some good examples for target devices are –
- linear for LVM2
- mirror for RAID
- stripped for LVM2
- snapshot for LVM2
- multipath for dm-multipath
Device-mapper Userspace Facilities
- Device-mapper works by processing data passed in from a virtual block device, that it itself provides, and then passing it on to another block device.
- Applications (like dm_multipath and LVM2) that want to create new mapped devices talk to the Device-mapper via the so shared library, which in turn issues ioctls to the /dev/mapper/control device node. Device-mapper is also accessible from shell scripts via the dmsetup tool.
- Device-mapper driver has been merged into Linux kernel since 2.6 version. Module soand dmsetup is the userspace facilities to communicate with the driver.
Now we know about what device mapper is and how it works. So let’s continue with our DM-Multipathing …
Now we are familiar with basic concepts of device mapper and dm-multipath and its related components. Let’s explore more…
How to check whether DM-Multipath installed and configured?
Step: 1 check whether device-mapper is installed or not as below.
Run the command: rpm -qa | grep -i device-mapper
Step: 2 verify that the device-mapper kernel modules are loaded if not loaded then load it.
Run the command: lsmod | grep -i dm_multipath #Lists the installed module
modprobe dm_multipath #loads the module
You will see the output below once you run the command: lsmod grep -i dm_multipath
Step: 3 check whether Multipathing enable for device in /etc/multipath.conf file. Uncomment lines in yellow-bold in order to enable device mapper.
Note: we can setup multipath with the mpathconf utility, which creates the multipath configuration file /etc/multipath.conf
Step: 4 check whether multipathd daemon is running or not.
Step: 5 print out the multipathd device.
Procedure for configuring the system with DM-Multipath:
- Install device-mapper-multipath rpm
- Edit the multipath.conf configuration file:
- comment out the default blacklist
- change any of the existing defaults as needed
- Start the multipathd daemons
- Create the multipath device with the multipath
Step: 1 Install the device-mapper-multipath package
Step: 2 create or Edit the multipath.conf configuration file
- The DM-Multipathing uses the configuration file /etc/multipath.conf for the configuration.
- If you make any changes to this file the multipath command must be run in order to reconfigure the multipathed devices.
- The easiest way to create this file is to use the mpathconf
- If there is an existing configuration file mpathconf will edit it, if no such file exists it will use /usr/share/doc/device-mapper-multipath-*/multipath.conf file as starting file.
- If /usr/share/doc/device-mapper-multipath-*/multipath.conf file does not exit then the mpathconf utility will create the /etc/multipath.conf file from scratch.
- If you don’t want to edit /etc/multipath.conf file manually then we can setup DM-Multipath for a basic failover configuration by running the following command which enables the multipath configuration file and starts the multipathd daemon.
Step: 3 manually editing of /etc/multipath.conf file
/etc/multipath.conf file divided into 5 main section as shown in table.
Step: 4 load the necessary kernel module for device-mapper and start the related services.
Step: 5 Configure device type in config file.
Step: 6 Configure multipath device in config file.
After removing the all the paths for a multipathed device, run the below command to remove the multipath device completely:
To flush all the multipathed device after stopping the multipathd daemon:
Thats all I have for this topic. if you have any questions or concerns let me know