How To Transfer Files Uploaded To An FTP/S Server To An Amazon S3 Bucket
Let's say you have an FTP or FTPS server and you want to transfer files uploaded to it to an AWS S3 bucket. You can do that with just a few clicks if your FTP/S service is running on JSCAPE MFT Server. Let me show you how it's done.
The basic set up
What I have here is an instance of JSCAPE MFT Server running an FTP/S service on a host with an IP address of 172.31.12.181...
Watch the video
Would you prefer to watch a video version of this tutorial instead? You can play the video below. Otherwise, just skip it if you wish to continue reading.
... an FTP/S client configured to connect to that FTP/S service...
... and an Amazon S3 bucket named 'jscapejohn' with a folder in it named 'folder1'.
Our objective is that, when a file is uploaded from that FTP client to the FTP service on JSCAPE MFT Server, that file should not be stored on the FTP server. But rather, it will be sent directly to that Amazon S3 bucket folder.
What we want to do
Here's an illustration of what we just described there.
How to achieve that on JSCAPE MFT Server
Let me now show you how to do that on JSCAPE MFT Server. Note that JSCAPE MFT Server is platform-independent, so it doesn't matter if you have it running on Windows, Linux, Solaris, MacOS, UNIX, AIX, or just about any operating system out there that has a JVM.
To start, let's go to the Network Storage module and click the Add button.
Next, select Amazon S3 from the Protocol drop-down list and then click OK.
Enter pertinent details for that network storage object.
- Give it a name,
- Enter the Access Key of your S3 bucket,
- Enter that Access Key's corresponding secret key
- Select the region that S3 bucket belongs to (e.g. US West (Oregon)). Make sure you have the right region. Otherwise you'll encounter some connection issues.
- Enter the S3 folder's path, e.g. jscapejohn/folder1.
Once everything's set, click the Test Server button to test the connection. If all goes well, click OK to proceed.
You should then see your newly created network storage object in the Network Storage tab.
Now that you have that network storage object, the next step is to map it to an existing user account's virtual path. You can also map it to a group's virtual path but let's just map it to a user's virtual path for now. The purpose of mapping the S3 network storage to a user's virtual path is so that, when that user logs in to the FTP service, he/she would see the network storage object as a regular folder in his/her account.
To map, just go to the Users module, select the desired user account, and click Edit.
Next, go to the Paths tab and click Add to add a new virtual path.
Enter the desired path for this virtual path. This will appear as a folder in this user account’s virtual file system. So, for example, if you enter ‘/ans3bucketfolder’ (notice the forward slash), the folder ‘ans3bucketfolder’ will be added under this user’s virtual root directory.
After that, expand the Network Storage drop down list and select the Amazon S3 network storage object you created earlier to map the virtual path to the s3 network storage object.
Click OK to proceed.
Click OK until you’re back at the main screen.
That’s all there is to it! So, now, moving forward, every time this user logs in to that remote FTP service,
... he/she will see that virtual path (that maps to S3) just like any other folder...
... and be able to upload files to it.
Unbeknownst to that user, those uploaded files are not stored on that FTP server. Rather, they're actually stored in an Amazon S3 bucket folder.
That's it. Now you know how to configure JSCAPE MFT Server so that files uploaded to the FTP/S service are stored to an Amazon S3 bucket.
Try this out
Would you like to try this out yourself? Download the FREE, Starter Edition of JSCAPE MFT Server now.