Skip to main content
  • EN

Hi All,

Running an automation to pull data from an external SFTP site to be post processed with multiple other workspaces.

I've set the time per my server at 4:15 am (and tried 4:30am, now trying 5am), have checked my server, it is set to EST (UTC - 4 since we are in DST now). Schedule trigger states that it uses the local time on the FME server). Looking at the SFTP thin client website, the data says that it was uploaded at 4am today. When looking at the same SFTP site via WinSCP, it says that it was uploaded at 12am today.

My automation appears to be working as it should by processing all of the data. However, it is consistently having the previous week's data (it is only uploaded to the SFTP site once a week and I process only once a week). What time should I really be using for getting this data? When I trigger the automation after 8am EST it picks up the currently shown data (although it is dropping it on the server with the 12am timestamp). I would have thought that if I had a time AFTER the date stamp on the SFTP site, that it would get the latest data.

Have taken a look at the FTP directory watch and it will not connect no matter how I enter the creds. The SFTP site does have a host key and I don't see where to add that in the directory watch setup.

Thanks,

Lorinda

Hi @lorinda​ ,

 

To confirm, when you schedule for 4:15am-5am, it runs with previous week's data but when you trigger the Automation at 8am, it does not get the latest data? I agree that having the schedule set up to run after the SFTP site should get you the latest data. Is the previous week's data removed from the SFTP? You could try adding some logic in your workbench to process the latest data or remove the previous week's data before your Automation is scheduled to run.

 

As for the FTP directory modified trigger within Automations, it currently only supports working with FTP, FTPES and FTPS. It does not support SFTP.

There is an Idea on our Ideas page that requests adding support for SFTP as well, so definitely add your vote to that if it's something you'd like to see added to FME Server: https://knowledge.safe.com/content/idea/47929/sftp-watcher.html

 

Hope this helps!

Reply


Community Stats

30,149
Posts
114,457
Replies
37,795
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.