Blog

Managed File Transfer and Network Solutions

Rotating and Archiving Logs in JSCAPE MFT Server

Posted by John Carl Villanueva on Wed, Aug 13, 2014 @ 09:51 PM

Overview

A client wanted to know if there was a more efficient way of archiving and/or deleting old log files to clear up some storage space on their managed file transfer server. They were currently doing things manually and he understandably wanted to adopt some form of automation. Fortunately, JSCAPE MFT Server offers a solution and we'd like to detail the steps here so that anyone else interested may also follow it.

Rotating_and_archiving_logs

Warning

We do not recommended you to delete logs. These can be very important for audit purposes, especially for companies who are subject to regulatory compliance. Instead of deleting, we strongly suggest that you ZIP the files and then move them to another location. This way, you can review old logs if needed. Steps for archiving/zipping files are included in this article.

Log Rotation on JSCAPE MFT Server

Depending on the volume of requests your managed file transfer server gets daily, your log files can grow quite rapidly. Thus, to keep those logs more manageable and auditable, it would be a good idea to break them into smaller log files and to archive those files on a regular basis. To accomplish the first task (breaking into smaller log files), our client employed JSCAPE MFT Server's Log File Rotation feature, an automated process that rotates the log on a user defined basis. 

JSCAPE MFT Server allows log files to be rotated on a daily, weekly, or monthly basis. They can even be rotated as soon as the file reaches a certain size (in MB). These File Rotation settings can be configured during the domain creation process as shown in Figure 1.  

 

Figure 1 

01 mft server domain log service resized 600

 

If your domain has already been created, you can still change those settings by going to Logging > Service.

 

Figure 2 

02 mft server 9 logging service resized 600

 

Here are some sample log files of a domain whose file rotation was set to Daily. Some days were skipped because this was taken from a test server which didn't run every day. 

 

Figure 3

log files resized 600

 

Of course, once you have a server running in a production environment, you'll normally see one log file per day. Correspondingly, if the rotation is set to Weekly, you'll see one log file per week, and so on. Because these log files accumulate, our client wanted to automatically delete files that exceeded a certain period, say 60 days. The problem was there was no way of doing that through any of the settings shown earlier. 

To delete aged log files, you would need a combination of JSCAPE MFT Server's Directory Monitor and Trigger features. Here's an overview of the steps you need to take:

1. Create a Directory Monitor for the directory that stores your log files;
2. Check the "File exceeds age of" checkbox; specify the number of days (e.g. 60 days);
3. Create a Trigger that uses the "Directory Monitor File Aged" Event Type;
4. Specify the MonitorName variable to use the Directory Monitor you created in step 1

5A. Use the "Delete File"Trigger Action. Not recommended

5B. Use the "Zip File"Trigger Action. This is the recommended option.

Let me now go through the details.

1. Create the Directory Monitor

To create a Directory Monitor for the directory that stores your log files, go to Directory Monitor and then click the Add button.

 

Figure 4 

04 mft server 9 directory monitor add resized 600

 

Next, give this Directory Monitor a Name and then enter the path of the Directory in question. It's actually just the directory you specified in the screens shown on Figure 1 and Figure 2. 

 

Figure 5 

05 mft server 9 directory monitor parameters resized 600

 

2. Check the "File exceeds age of" checkbox

After that, tick the check box File exceeds age of and then specify the maximum number of days you want the log files to be stored before having it deleted. Click OK.

 

Figure 6 

06 mft server 9 directory monitor file exceeds age resized 600

 

You should then see your newly created Directory Monitor in the Directory Monitors tab. 

 

Figure 7 

07 mft server 9 directory monitor newly added resized 600

 

3. Create the Trigger

Next, you'll need to create a trigger that would fire as soon as a file in the monitored directory reaches the specified age. 

Start by going to Triggers and then click the Add button.

 

Figure 8 

08 mft server 9 add new trigger resized 600

 

Give this trigger a name and then select the "Directory Monitor File Aged" Event type. Click Next. 

 

Figure 9 

09 mft server 9 trigger event dm file aged resized 600

 

4. Specify the MonitorName variable

Since you might have many Directory Monitors in place, specify the Directory Monitor in question in the Trigger Conditions Expression box. In my case, that would be Monitor Log Directory. When you're done, click Next.

 

Figure 10 

10 mft server 9 trigger condition monitorname resized 600

 

5A. Use the "Delete File"Trigger Action

Finally, you need to add the Delete File Trigger Action. To do that, click the Add button...

 

Figure 11 

11 mft server 9 trigger action add resized 600

 

... and then select the Delete File Action. 

 

Figure 12 

12 mft server 9 trigger action delete file resized 600

 

Enter the %File% variable into the File field and then click OK.

 

Figure 13 

13 mft server 9 trigger action delete file parameters resized 600

 

5B. Use the "Zip File"Trigger Action.

As indicated earlier, the above option isn't recommended. Instead of using the Delete File Trigger Action, we suggest you archive aged log files by using the Zip File Trigger action like this:

 

Figure 14

14 mft server 9 trigger action zip file resized 600

 

Specify the Zip file trigger action parameters. If you want to delete the original file, set Delete Source to true. Click OK.

 

15 mft server 9 trigger action zip file parameters resized 600

 

Click the OK buttons until you're back at the main screen, where you should then see your newly created trigger.

 

Figure 14

16 mft server 9 trigger new resized 600

 

That's it! If you want to try this out yourself, download the latest version of JSCAPE MFT Server today.

 

 

Download Now