Question

Best practices for monitoring folder of XML files and then uploading new ones to S3?

  • 24 July 2018
  • 1 reply
  • 1 view
E

I have a folder that has approximately 50 XML files in it. Between one and three new XML files are added to the folder each day. Each of these files needs to be sent to an AWS S3 bucket. Ideally, every morning FME would scan the folder for new XML files and upload only the new files to the S3 bucket.

 

 

Does anyone have an example workspace or best pracitces they can share? There seems to be many ways to do this, but I would love some guidance on the most efficient and reliable route.

I have both FME Desktop and FME Server.

Thanks!!

1 reply

stalknecht
Rank
Userlevel 1
Badge +18

In general you can use the next steps. Create a workspace that copies a XML file to S3 using S3Uploader transformer and publish it to server. On FME server add a publication with the protocol "Directory Watch" to point to the XML folder. Create a topic to publish to. Create a subscription that listens to the same topic with protocol "FME Workspace" and call the just created workspace to do the trick.

Reply


Community Stats

28,688
Posts
109,427
Replies
36,543
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.