If your FTP/S service runs on JSCAPE MFT Server, it's possible to mirror files stored in that FTP server to an Amazon S3 bucket folder. We'll show you how to do that in this post.
Mirroring in this context is basically a type of synchronization process that ensures a target directory (in this case, the Amazon S3 bucket folder) ends up with exactly the same files as a source directory (in this case, your FTP server).
Some people don't want to call it synchronization because, strictly speaking, synchronization is a two way process, i.e., new and modified files from both directories are copied onto both paths. Mirroring is normally just one way, i.e., new and modified files are copied from source to target and redundant files in the target are deleted. As you'll see in a short while, JSCAPE MFT Server supports both processes.
Watch the video
Would you prefer to watch a video version of this tutorial instead? You can play the video below. Otherwise, just skip it if you wish to continue reading.
[this is on your Amazon S3 bucket]
I'm going to assume you already have an Amazon S3 bucket with a folder inside it. Here, for example, I have a bucket named 'jscapejohn' and inside it is a folder named 'folder1'. Take note of the region (e.g. US East (Ohio)) indicated for that bucket as you'll be needing that piece of information when you set up your Amazon S3 trading partner on JSCAPE MFT Server.
[this is on your FTP server]
I'm now on my JSCAPE MFT Server instance and, as you can see, I have an FTP service running on it. So, this basically serves as an FTP server. JSCAPE MFT Server supports several other file transfer protocols, so it can also be an SFTP server, an HTTP server, a WebDAV server, and so on, all at the same time.
Before we can set this up to mirror a local folder to our Amazon S3 folder, we need to create a couple of trading partner objects first. The first one will be a Local Directory trading partner, which will serve as a sort of reference point for any local directory on this JSCAPE MFT Server instance. Of course, the second one will be our Amazon S3 trading partner.
To create a trading partner, just go to the Trading Partners module and click the Add button.
We'll be creating out Local Directory trading partner first, so just expand the Protocol drop-down list and select Local Directory. Click OK to proceed.
Next, give this trading partner a name, e.g. tp-local, and enter any other pertinent information that you wish to specify for this trading partner. We'll just leave ours blank for now. Once you're done click OK.
You can then do the same process for the Amazon S3 trading partner.
The Amazon S3 trading partner will have a couple more fields that you'll need to fill up. In addition to the trading partner object's name...
... you also need to specify your Amazon S3 account's Username or Access Key as well as its corresponding Password or Secret Key.
Lastly, you'll need to specify the region. If you recall, I asked you take note of that earlier. In our case, our Amazon S3 bucket's region is US East (Ohio).
Click OK to proceed.
Once you're trading partner objects are ready, you can select them one by one and click the Test button to test if your connections to those trading partners are actually working.
If all tests succeed, you can then proceed to create a trigger that would enable an automated mirroring process from a directory you will soon specify to your Amazon S3 bucket folder.
To do that, just go to the Triggers module and click the Add button.
For those who are using JSCAPE MFT Server version 12.1 and higher, you'll be presented with a Trigger Template dialog where you can choose a template that best describes the workflow you're about to automate. Let's just leave that blank and click OK.
Give this trigger a name, e.g. 'sync local with s3'.
Now, you can specify an event that would initiate this mirroring process. We just want this trigger to execute at a particular schedule, so let's just choose the Current Time event type.
In the next screen, you can use the Expression Builder to build an expression that would define the schedule when this particular trigger should fire. I want this trigger to execute at 11:30 PM everyday, so I built the expression as shown on the screen.
Recommended read: Introducing the New Trigger Conditions Expression Builder
When you get to the next screen, click the Add button to add a new trigger action. This is the part of trigger settings where you would specify how the mirroring process should be carried out.
Select the Trading Partner Synchronization action from the drop-down list and then click OK.
Let me now walk you through the key settings for this Trading Partner Synchronization action.
The first ones you'll encounter are PartnerA and PartnerB. PartnerA is basically the source. It's the trading partner from which the target trading partner will be mirroring from. So, in our case, that would be the Local Directory trading partner.
PartnerB, on the other hand, is the target. It's the trading partner to which the source trading partner will be mirroring to. In our case, that would be the Amazon S3 trading partner.
Next up are PathA and PathB. PathA is the absolute local directory path in PartnerA which PathB will be mirroring from, and PathB is the relative path on PartnerB that PathA will be mirroring to.
The next setting we need to specify is the Copy Condition. This is the condition JSCAPE MFT Server will use to determine whether to commence copying (or mirroring) files each time the predefined schedule of this trigger is up.
If you select:
- different time, JSCAPE MFT Server will commence copying if it sees that file timestamps on A are different from the ones on B;
- different size, JSCAPE MFT Server will commence copying if it sees that file sizes on A are different from the ones on B;
- different content, JSCAPE MFT Server will commence copying if it sees that the content in A is different from the content in B
Let's just choose different time for now.
Another setting you need to specify is the Synchronization Mode. There are four options:
- mirror - New and modified files from A are copied to B; redundant files in B will be deleted;
- synchronize - New and modified files from both paths are copied to each other
- backup - All files from A are copied to B
- contribute - New and modified files from A are copied to B
Obviously, we need to select 'mirror' because that's what we're here for but at least you know there are other options to choose from.
Lastly, you need to specify the Result Directory. This is where the results of the mirroring process will be written to.
Click OK and then drag an arrow from the Start output of the Workflow node to the Trading Partner Synchronization Action node.
Recommended read: Introducing the Redesigned Trigger Action Workflow
Click OK to finalize the trigger creation process.
That's it. Now you know how to configure JSCAPE MFT Server so that you can mirror an FTP/S folder to an Amazon S3 bucket folder.
Try this yourself
Would you like to try this yourself? Download the FREE, fully-functional Starter Edition of JSCAPE MFT Server now.
Related content:
How to Download Newly Added Files from an AWS S3 Folder
Set Up Automated File Transfers with Windows SMB Share
Moving Data Across a Hybrid Cloud Doesn't Have To Cost An Arm and A Leg