Overview
Some modern file transfer servers have the built-in capability to store uploaded files in different places. For instance, in JSCAPE MFT Server, you can store uploaded files locally (i.e., on the same machine as the system running the file transfer service itself) or to a remote server or storage service. One of these storage services is Amazon S3. But why would you want to store uploaded files there?
In this post, we explore some pros and cons of using Amazon S3 as your MFT Server's file storage system compared to the default option, i.e., local storage.
Advantages of using Amazon S3 for file storage
Amazon S3, which stands for Simple Storage Service, is Amazon's cloud-based storage solution. You can connect to it from MFT Server by using MFT Server's reverse proxy module. Because it's a cloud-based solution, S3 comes with the benefits of the cloud, including:
High scalability
Most file transfer servers store their files in hard disks that are directly attached (or built-in) to the physical server. These disks naturally have limited space. That's why server administrators often set storage quotas or adopt regular file deletion or archiving practices to conserve disk space.
Once the maximum limit of these local disks is reached, they have to be replaced, or additional disks have to be added to the server. This can be time-consuming and can cause a substantial amount of downtime.
By comparison, Amazon's massive and highly scalable infrastructure enables file transfer servers to have unlimited storage. That means your users and trading partners can upload gigabytes or terabytes upon terabytes of files, and your server won't even bat an eyelash. An Amazon S3 bucket, where your uploaded files will be stored, has unlimited capacity. So, your users can upload as many files, regardless of size, as they want.
High availability and disaster recovery capabilities
One problem with local storage is that if the local hard disk crashes, your users cannot access their files. Similarly, if the hard disk works fine but the server crashes, the files won't be accessible either.
You won't be subjected to the same level of risk if you store your files on Amazon S3. That's because Amazon S3 is built on top of a reliable, fault-tolerant, and highly available infrastructure. The chances of an Amazon S3 bucket 'crashing' or becoming unavailable are extremely low.
Also, if your server crashes, you can still access your files on Amazon S3 by using another instance of JSCAPE MFT Server (e.g., running on a backup server) and then connecting to the Amazon S3 bucket using precisely the same S3 login credentials.
If you want, you can even set up two or more instances of JSCAPE MFT Server behind an MFT Gateway reverse proxy to form an active-active high availability cluster and then have those instances connect to the same Amazon S3 bucket to reduce the chances of an MFT Server going down due to overload.
Pay-as-you-go pricing
Heavily used file transfer servers not bound by strict storage quota restrictions or file deletion and archiving policies can rapidly gobble up storage resources. That can be a problem if your IT department runs on a tight budget.
Let's say you anticipate a large storage consumption in the next few years. And so, to meet that anticipated demand, you purchase an enterprise-class on-premise storage solution. That could easily cost north of a thousand $$$$. And because it's an upfront cost, you'll feel the full force of the expense right away, even if you won't be able to make full use of its maximum capacity yet.
An Amazon S3 subscription is more practical and affordable because you only need to pay as you go. Although the prices vary from region to region, they're usually just a couple of cents per GB.
So if you're starting small, you can certainly take advantage of the substantial cost savings. Then as your storage demands increase, you can gradually increase your spending. This rate of expense will be easier on your cash flow.
Regulatory compliance
Some data protection laws and regulations impose restrictions to where files should be stored. Some laws, for instance, have data localization provisions that prohibit files from being stored overseas or have stringent requirements if files have to be stored outside certain regions (the EU GDPR is one example).
In cases like this, storing files in Amazon S3 provides certain benefits. If the users of your file transfer server are scattered across the globe or located in different regions with varying data protection legislation, you could benefit from Amazon S3's capability to store data in different geographical locations, a.k.a. regions.
Basically, when you create an Amazon S3 bucket, you can specify which region that bucket will be hosted in. So, suppose you have a geographically dispersed user base. In that case, you can create multiple buckets and assign each user to whichever bucket (along with its associated region) keeps you in compliance with the law. Some of the supported regions include:
- US East, in North Virginia
- US West, in Oregon
- EU, in Ireland,
- Asia Pacific, in Tokyo
Managed file transfer servers like JSCAPE MFT Server allow you to connect to multiple buckets and map each bucket (via reverse proxy) to a virtual path. So, you can, for example, enable one set of users to store their files in the US and another set of users to store their files in EU. You can even enable certain users to access buckets in different regions. You can't have this flexibility if your server can only store files locally.
Highly secure
This isn't necessarily an advantage over local storage (because you can also make it equally secure), but it's worth mentioning.
During a file transfer session, i.e., when files are being uploaded to your server, files are protected via data-in-motion encryption technologies like SSL/TLS (in the case of FTPS or HTTPS) or SSH (in the case of SFTP). But once on your server, they must be secured with a data-at-rest encryption solution.
For locally stored files, JSCAPE MFT Server offers PGP encryption. But how about files uploaded to an Amazon S3 bucket? For this purpose, Amazon offers server-side encryption. Interested in seeing how seamlessly JSCAPE MFT Server integrates with Amazon S3 for secure and scalable file storage? Schedule a demo with us today and discover how to optimize your file storage strategy. Amazon S3 encrypts data using AES-256, a strong and widely accepted block cipher that is also a federal government standard.
Disadvantages of using Amazon S3 for file storage
Although there are many advantages of using Amazon S3 as the file storage system for your managed file transfer server, it also comes with a few disadvantages. We discuss some of them here.
Increased latency
When it comes to file transfers, latency is always a bad thing. High latency means slower uploads and downloads. Generally speaking, the longer the distance files travel, the greater the latency they'll get to experience.
Recommended read: Introducing Accelerated File Transfer (an article that discusses how to increase file transfer speeds even in high latency networks)
Storing files on Amazon S3 means you'll need to deal with additional latency because, after files are uploaded to your server, they'll still have to be forwarded to the Amazon S3 infrastructure. This means they'll have to go through the Internet again.
That's a much longer path than if the files only had to be stored in a disk directly attached to (or even in the same local area network) the file transfer server itself. Worse, if your S3 bucket is located in a region far from the geographical location of your server, you'll experience even greater latency.
Of course, if your MFT Server is running on an Amazon EC2 instance, then the latency might be reduced. However, it will probably still be higher than if you stored files in a (physical) directly attached disk.
Privacy concerns
In spite of the robust security measures implemented by Amazon on the S3 infrastructure, some companies just don't feel comfortable entrusting their sensitive data to third parties. In these situations, you're left with no other option but to store your files locally or in the same data center as the one hosting your file transfer server.
Related content
How To Use Amazon S3 As Storage For Your MFT Server
How to Access AWS S3 Storage with SFTP
How To Connect and Upload Files To an Amazon S3 Trading Partner
How to Download Newly Added Files from an AWS S3 Folder
Monitoring an Amazon S3 Folder and Transferring Newly Added Files To Azure
How To Transfer Files Uploaded To An FTP/S Server To An Amazon S3 Bucket
How To Sync a Linux Directory with an Amazon S3 Bucket
How To Copy Files From Azure Blob Storage To AWS S3 | JSCAPE