Blog

Managed File Transfer and Network Solutions

How To Set Up A SFTP Active-Active High Availability Cluster

Posted by John Carl Villanueva on Thu, Sep 22, 2016 @ 10:53 PM

Overview

An active-active high availability SFTP cluster can greatly reduce the chances of downtime by allowing two or more SFTP servers share the load. If you'd like to learn an easy way of setting up this kind of cluster, check out this post.


What we're trying to achieve

 

sftp_active_active_high_availability_cluster.png

 

Why you would need a SFTP active-active HA cluster

An active-active SFTP high availability cluster primarily does two things:

1. It enables load balancing.

and

2. It automatically redirects traffic to 1 or more healthy server(s) if one of the servers go down.

Load balancing basically distributes network traffic among all members of the cluster. Although the manner of distribution varies depending on the type of load balancing algorithm used, they all work to prevent a single server from shouldering all of the traffic.

If only one server shoulders the load, that server can easily get overloaded and eventually fail. This can cause downtime. However, if multiple servers share the load, the chances of one server going down due to overload will be reduced substantially. It also has another benefit.

A SFTP HA cluster eliminates single points of failure. In the event that a server/node does fail, traffic will be automatically redirected to the remaining nodes in the cluster. They can then take up the load and your file transfers can continue as if nothing happened. 

 

Implement redundancy

One of the basic elements of any high availability cluster is redundancy. This means that each node in the cluster must have exactly the same configuration as the other nodes. If all your SFTP services are running on instances of JSCAPE MFT Server, then setting up redundancy is as easy as pie. 

All you have to do is point each MFT Server (with each running an SFTP service) to a shared global datastore like so:

mft_server_global_datastore-1.png

We already included a detailed discussion on that in the post "How To Set Up A SFTP Active-Passive Cluster", so I suggest you read that first. If you follow the steps for connecting to a global datastore as instructed in that post, all your SFTP servers will automatically acquire the same configurations. If you make changes to your settings in one node, those changes will be automatically applied to all other nodes of the cluster. 

We now proceed to the most crucial part of this tutorial.

 

Set up the SFTP cluster and apply load balancing

 

In order to balance loads across your cluster, you need a load balancer. This load balancer will also be responsible for accepting inbound connections as well as redirecting traffic to available nodes should one of the cluster's nodes fail. What we recommend is JSCAPE MFT Gateway, a software-based load balancer and reverse proxy that supports SFTP as well as several other file transfer protocols. 

Assuming you already have an instance of MFT Gateway powered up, the first thing you need to do is create a SFTP cluster.

Login to the MFT Gateway Manager and navigate to the Clusters menu. Click the Add button to add a new cluster.

 

add-high-availability-cluster.png

 

Give the cluster a name (e.g. SFTP Cluster) and then select  SFTP/SSH from the Protocol drop-down list. 

Choose a load balancing algorithm that suits your needs. 

 

add_sftp_cluster.png

 

 

Next, click the higher Add button (there are two on that screen) to add a host to your SFTP cluster. 

 

sftp_cluster_added.png

 

Enter the IP address of one of your SFTP nodes as well as its SFTP service's port number (in case it's not the default - 22). Click the Add button.

 

add_sftp_cluster_host.png

 

You should then see that node's connection settings in your list of hosts. Click the Add button again to add another host. Do this for all remaining hosts/nodes. 

 

added_host_to_sftp_cluster.png

 

Here's how my screen looks like after adding 4 nodes. 

When you're done adding nodes, click the lower Add button to add the cluster. 

To summarize, first you add a SFTP cluster, then you add nodes to the cluster, and then you add/finalize the cluster. 

 

hosts_in_a_sftp_high_availability_cluster.png

 

Here we have our newly added SFTP cluster. 

 

newly_added_sftp_cluster.png

 

The next step would be to add an SFTP service that would be listening for SFTP connection requests. So, for example, MFT Gateway would be sitting on your DMZ, those connection requests would be coming from an external network, and your SFTP nodes would be sitting inside your internal network. 

 

sftp_active_active_high_availability_cluster.png

 

To add an SFTP service, go to the Services menu and then click the Add button. 

 

add-reverse-proxy-service-to-high-availability-cluster.png

 

Select SFTP/SSH from the Protocol drop-down list.

 

add_proxy_service_protocol.png

 

Choose an IP address from the drop-down list. This would be the IP address of the network interface that would be listing to SFTP requests. For this example, we just chose 0.0.0.0, which stands for all network interfaces on our machine. We also left the port number to the default. 

Next, select the Cluster option button and then choose the cluster we created earlier from the drop-down list. 

When you're done, click Add

 

add_sftp_reverse_proxy_service_cluster.png

 

You should then see your newly created SFTP service with a setting indicating that it's associated with the SFTP Cluster. That means, all valid SFTP requests that connect to this service will be forwarded to the cluster named "SFTP Cluster".

 

newly_added_sftp_reverse_proxy_service_cluster.png

 

Connect all SFTP nodes to a shared storage

Note: This is different from the global datastore.

Last but not the least, you need to set up a shared storage system for your cluster. The purpose of having a shared storage is that it allows users and file transfer clients to store and retrieve files to/from the same storage system regardless which SFTP server they manage to connect to.

We already had an extensive discussion on this matter in the post "Setting Up a NAS Shared Storage for Your File Transfer Servers". That post already includes reasons for setting up as well as a detailed example on how to set up a shared storage, so I suggest you take the time to read it. 

 

mft_server1_down_shared_storage.png

 

Get started

Would you like to try this out? Download a free, fully-functional evaluation edition of JSCAPE MFT Gateway now.

 

Download Now 

 

Related posts

How To Install A SFTP Server on Windows

Configuring A High Availability Cluster for Various TCP/UDP Services

You Know It’s Time To Implement Server To Server File Transfer When..

Install A Linux SFTP Server Via Command Line and Configure Via Web UI

Introduction to Reverse Proxy Servers

Scheduling Automated File Synchronization Between Two Servers

How To Set Up An Automated SFTP Download 

The Quickstart Guide To Setting Up An AS2 Server

 

Want to be updated on posts like this? Connect with us...

 

 

 

 

Topics: JSCAPE MFT Server, High Availability, JSCAPE MFT Gateway, SFTP, Failover, Load Balancing