Hi Rob

 

It might be easier to skip the Data Download Service and use the FME Server Notification framework instead. The basic steps are

 

1. Set up an Email Subscription in the WebUI with an associated topic for sending emails

 

2. Create a master workspace which:Runs you main "worker" workspace using the FMEServerJobSubmitter

 

uploads to the output using the S3Uploader

 

creates an email using the FMEServerEmailGenerator and send the email using the FMEServerNotifier3. Any parameters in the worker workspace would need to be created in the master as well so they are available in the web form. They have to be passed to the worker workspace the FMEServerJobSubmitter by using published parameter values for the parameter values shown in the transformer.

 

4. This master gets published to FME Server and the default webform will include the email address and any parameters exposed. Then users will fill in the form and run the master.

 

 

I made a quick video to show the key steps here:

 

http://screencast.com/t/EjhoPYMbd

 

 

There is a tutorial on sending emails with FME Server which will also help here:

 

https://knowledge.safe.com/articles/How_To/Email-in-FME-Server-A-Tutorial-Series

 

 

Hope this helps

 

?Ken

---

Thanks Ken,

 

That was really helpful and will definitely work for us, but...the worker workspace writes a raster output and a text file output.  I want to include both of these outputs in a zip file then upload the zip file to the S3 bucket.  How do I create that zip file with there doesn't seem to be a FME_SERVER_DEST_DIR parameter for the worker workspace.

 

I've been playing with a python shutdown script on the worker workspace to try and do the zipping (https://knowledge.safe.com/articles/How_To/Zip-your-Output-with-FME) but don't know where to point it.

 

These are examples of the output locations for the text file and grid:

 

/mnt/fme_temp/fmeengines/localhost_Engine4/FME_NEDF_DEMClipZipShip_Worker_1441339087140_20082/GENERIC_1/NEDF_test

 

/mnt/fme_temp/fmeengines/localhost_Engine4/FME_NEDF_DEMClipZipShip_Worker_1441339087140_20082/TEXTLINE_1/readme_metadata.txt

 

Ideally I'd like to zip the files under this directory

 

/mnt/fme_temp/fmeengines/localhost_Engine4/FME_NEDF_DEMClipZipShip_Worker_1441339087140_20082

 

but there doesn't seem to be a parameter defining this location.

 

 

I've noticed the ZipArchiver transformer but have an older build so don't have access to it yet.

 

 

Any ideas?

 

 

Thanks

 

Rob

 

 

---

Hi Rob,

 

Here is a suggestion I just tried. By the way can you please tell me what version of FME Desktop and Server you are working with?

 

 

All file and directory writers in FME have the ability to set a zip file as ouput. You can set multiple writers to the same zip file name and all of the output will go into the one zip file. (To create a more complex zip file structure you'll need to use Dataset Fanout but we can come back to that.)

 

You can write this zip file to the FME Server Resources temp directory using an existing FME Server system parameter which points to it (the true path is hidden). And then the S3Uploader can get it from there. To answer your questions the FME_SERVER_DEST_DIR parameter is only created if using the actual Data Download Service.

 

 

Here are the basic steps:

 

 

1. In the Worker workspace any writers should have their destination data set parameter set like this:

 

$(FME_SHAREDRESOURCE_TEMP)\\output.zip

 

This way all output goes to a single file in your shared resources temp directory.

 

 

2. Then you will have to change a couple of small things in the master The setting in the FMEServerJobSubmitter which says "Output Data Location" needs to be changed to "As Directed in the Workspace".  Now you need to change the path to the file for the S3 uploader. Do this by creating a FilePath attribute with an AttributeCreator and set it to $(FME_SHAREDRESOURCE_TEMP)/output.zip

 

In the S3Uploader change the Source File name to this new FilePath attribute and change the key to output.zip

 

 

You'll likey need to create a hashed zip file name so multiple users aren't writing to the same zip file and the same time but we can come back to that if you think this approach will work for you.

 

Hope this helps

 

Ken

---

Thanks Ken,

 

That worked perfectly.  Great little tip about the ability of FME writers to generate zip file...didn't know that!

 

For now used an attribute creater in the master workspace to create a unique output path to the zip file $(FME_SHAREDRESOURCE_TEMP)DataExtract$(FME_JOB_ID).zip

 

and pass its value to the worker workspace via a published paramter.  I've added the job ID to the file name to make it unique.

 

Thanks again for all your help Ken,

 

Rob

---