Blog

Managed File Transfer and Network Solutions

How To Transfer Files Uploaded To An FTP/S Server To An Amazon S3 Bucket

Posted by John Carl Villanueva on Thu, Jul 02, 2020 @ 04:15 AM

Let's say you have an FTP or FTPS server and you want to transfer files uploaded to it to an AWS S3 bucket. You can do that with just a few clicks if your FTP/S service is running on JSCAPE MFT Server. Let me show you how it's done.

The basic set up

What I have here is an instance of JSCAPE MFT Server running an FTP/S service on a host with an IP address of 172.31.12.181...

Watch the video 

Would you prefer to watch a video version of this tutorial instead? You can play the video below. Otherwise, just skip it if you wish to continue reading.

 

ftp service

 

... an FTP/S client configured to connect to that FTP/S service...

 

login via ftp

 

... and an Amazon S3 bucket named 'jscapejohn' with a folder in it named 'folder1'.

 

amazon s3 bucket

 

Our objective is that, when a file is uploaded from that FTP client to the FTP service on JSCAPE MFT Server, that file should not be stored on the FTP server. But rather, it will be sent directly to that Amazon S3 bucket folder. 

What we want to do

Here's an illustration of what we just described there.

upload ftp file to amazon s3

How to achieve that on JSCAPE MFT Server

Let me now show you how to do that on JSCAPE MFT Server. Note that JSCAPE MFT Server is platform-independent, so it doesn't matter if you have it running on Windows, Linux, Solaris, MacOS, UNIX, AIX, or just about any operating system out there that has a JVM. 

To start, let's go to the Network Storage module and click the Add button.

 

network storage add

N

ext, select Amazon S3 from the Protocol drop-down list and then click OK.

 

amazon s3 network storage protocol add

 

Enter pertinent details for that network storage object. 

  • Give it a name,
  • Enter the Access Key of your S3 bucket,
  • Enter that Access Key's corresponding secret key
  • Select the region that S3 bucket belongs to (e.g. US West (Oregon)). Make sure you have the right region. Otherwise you'll encounter some connection issues. 
  • Enter the S3 folder's path, e.g. jscapejohn/folder1. 

Once everything's set, click the Test Server button to test the connection. If all goes well, click OK to proceed.

 

amazon s3 network storage details

 

You should then see your newly created network storage object in the Network Storage tab.

 

newly created network storage

 

Now that you have that network storage object, the next step is to map it to an existing user account's virtual path. You can also map it to a group's virtual path but let's just map it to a user's virtual path for now. The purpose of mapping the S3 network storage to a user's virtual path is so that, when that user logs in to the FTP service, he/she would see the network storage object as a regular folder in his/her account.

To map, just go to the Users module, select the desired user account, and click Edit.

 

edit user-1

 

Next, go to the Paths tab and click Add to add a new virtual path.

 

add new path

 

Enter the desired path for this virtual path. This will appear as a folder in this user account’s virtual file system. So, for example, if you enter ‘/ans3bucketfolder’ (notice the forward slash), the folder ‘ans3bucketfolder’ will be added under this user’s virtual root directory.

After that, expand the Network Storage drop down list and select the Amazon S3 network storage object you created earlier to map the virtual path to the s3 network storage object.

Click OK to proceed.

 

ans3bucket virtual path

 

Click OK until you’re back at the main screen.

That’s all there is to it! So, now, moving forward, every time this user logs in to that remote FTP service,

 

login via ftp-1

 

... he/she will see that virtual path (that maps to S3) just like any other folder... 

 

s3 bucket folder virtual path

 

... and be able to upload files to it.

 

uploaded file

 

Unbeknownst to that user, those uploaded files are not stored on that FTP server. Rather, they're actually stored in an Amazon S3 bucket folder.

 

uploaded file to s3 via ftp

 

That's it. Now you know how to configure JSCAPE MFT Server so that files uploaded to the FTP/S service are stored to an Amazon S3 bucket.

Try this out

Would you like to try this out yourself? Download the FREE, Starter Edition of JSCAPE MFT Server now.

 

Download Now

Topics: JSCAPE MFT Server, Managed File Transfer, Cloud Computing, Tutorials