Blog

Managed File Transfer and Network Solutions

How To Sync A Windows Folder With An AWS S3 Bucket

Posted by John Carl Villanueva on Sat, Oct 17, 2020 @ 03:10 AM

In previous posts, we taught you how to set up an Agent service and trading partner. We also taught you how to install an agent on a remote Windows host devoid of any file transfer service. Let's now put that agent to work. In this post, we're going to teach you how to configure JSCAPE MFT Server so that that remote Windows host can synchronize files with an Amazon S3 bucket. 

Note: To understand the concepts behind this solution, read the post: 

How to Transfer Data to a Host Without a Service

Watch the video 

Would you prefer to watch a video version of this tutorial instead? You can play the video below. Otherwise, just skip it if you wish to continue reading.

Before we start setting things up on JSCAPE MFT Server, let me first show you the two folders that will be involved in the synchronization process. The first one is the folder named 'syncfolder' under the C: drive of my Windows host. If you recall, this is the same folder we designated as the 'Root path' when we installed our agent in the tutorial How to Install MFT Server Agents on Windows.

 

[on my remote Windows host]

 

aws s3 sync windows - sync folder-1

 

If you recall also, the IP address of that remote Windows host was 172.31.12.181. 

 

aws s3 sync windows - windows host ip address

 

[on my S3 bucket folder]

The second folder that's going to take part in this synchronization process is the S3 bucket folder named 'folder1'. This folder is located inside the bucket named 'jscapejohn'.

 

aws s3 sync windows - s3 bucket-1

 

[on my JSCAPE MFT Server instance]

Now that we have those two folders ready, let's now head over to our JSCAPE MFT Server instance so we can start setting up our synchronization process. 

First, let's go to the domain where we created our agent service and our agent trading partner. If we go to Overview > Sessions, we should see an active session that represents the agent on our Windows host. You'll know from the session type and the IP address of the Client, which in our case should be 172.31.12.181. This means, the agent on our Windows host is actively connected to our JSCAPE MFT Server instance. 

 

aws s3 sync windows - agent session

 

If we also go to our Trading Partners module, we should also see the trading partner that corresponds to our agent on that remote Windows host as well as a trading partner for our Amazon S3 bucket. We recognize them by their designated names, tp-agent and tp-s3 respectively. 

If you don't have an Amazon S3 trading partner on hand yet, I suggest you read the post: How To Connect and Upload Files To an Amazon S3 Trading Partner.

Read the section 'Creating the Amazon S3 Trading Partner'.

 

aws s3 sync windows - trading partners

 

Now for the main part. Let's now start creating the trigger that will facilitate our synchronization process.

Go to the Triggers module and click the Add button.

 

aws s3 sync windows - add new trigger

 

For those who are using JSCAPE MFT Server version 12.1 and higher, you'll be presented with a Trigger Template dialog where you can choose a template that best describes the workflow you're about to automate. Let's just leave that blank and click OK

 

aws s3 sync windows - add new trigger template

 

Give the trigger a name, for example 'aws s3 sync windows', and the select the Current Time event type. You may select any event type that suits your needs but, for this example, I'd like this trigger to run at a certain time of the day. That's why I'm using the Current Time event type. 

Click Next to proceed.

 

aws s3 sync windows - add new trigger current time event type

 

In the next screen, you can use the Expression Builder to build an expression that would define the schedule when this particular trigger should fire. I want this trigger to execute at 12:05 AM everyday, so I built the expression as shown on the screen. 

Recommended read: Introducing the New Trigger Conditions Expression Builder

Click Next to proceed.

 

aws s3 sync windows - trigger condition

 

Once you get to the next screen, click the Add button to add a new trigger action and then select Trading Partner Synchronization from the Action drop-down list and then click OK.

 

aws s3 sync windows - trading partner synchronization action

 

Let me now walk you through the key settings for this Trading Partner Synchronization action. 

The first ones you'll encounter are PartnerA and PartnerB. Usually, PartnerA is the source. But because this is a synchronization process, which is a two-way process, the concept of a source and target is irrelevant.

Anyway, let's just select our Windows host agent trading partner for PartnerA and our Amazon S3 trading partner for PartnerB.  

Next up are PathA and PathB. PathA is the relative path in PartnerA that will be involved in the synchronization process and PathB is the relative path on PartnerB that PathA will be synchronizing with. Of course, those paths should already exist in the respective trading partners.

The forward slash (/) in PathA simply points to the root folder of our designated 'Root path' for the Windows host agent. If you recall, its absolute path on that Windows host is the directory with the path C:/syncfolder. 

For PathB, 'jscapejohn' is just the S3 bucket and 'folder1' is just the folder inside that bucket.

 

aws s3 sync windows - trading partner synchronization action parameters 1-1

 

The next setting we need to specify is the Copy Condition. This is the condition JSCAPE MFT Server will use to determine whether to commence copying (or synchronizing) files each time the predefined schedule of this trigger is up.

If you select:

  • different time, JSCAPE MFT Server will commence copying if it sees that file timestamps on A are different from the ones on B;
  • different size, JSCAPE MFT Server will commence copying if it sees that file sizes on A are different from the ones on B;
  • different content, JSCAPE MFT Server will commence copying if it sees that the content in A is different from the content in B

Let's just choose different time for now. 

Another setting you need to specify is the Synchronization Mode. There are four options:

  • mirror - New and modified files from A are copied to B; redundant files in B will be deleted;
  • synchronize - New and modified files from both paths are copied to each other
  • backup - All files from A are copied to B
  • contribute - New and modified files from A are copied to B

As you might have guessed, we'll be using synchronize for this example. 

Lastly, you need to specify the Result Directory. This is where the results of the synchronization process will be written to. 

 

aws s3 sync windows - trading partner synchronization action parameters 2

 

Click OK and then, when you get to the outer screen, drag an arrow from the Start output of the Workflow node to the Trading Partner Synchronization Action node. 

Recommended read: Introducing the Redesigned Trigger Action Workflow

Click OK to finalize the trigger creation process. 

 

aws s3 sync windows - new trading partner action

 

aws s3 sync windows - new created trigger

 

That's it. Now you know how to configure JSCAPE MFT Server so that you can synchronize a remote Windows folder to an Amazon S3 bucket folder. 

Give JSCAPE MFT Server Agents a test run

Interested in trying out JSCAPE MFT Agents? Download the FREE, fully-functional Starter Edition of JSCAPE MFT Server now. 

Download JSCAPE MFT Server

Agents can be downloaded from here:

Download an Agent

 

Topics: JSCAPE MFT Server, Managed File Transfer, Business Process Automation, Cloud Computing, Tutorials, Triggers