Skip to main content
Solved

Getting temporary folder of Data Download Service

jneujens
Forum|alt.badge.img

Hi all,

Workspace A is uploaded to FME Server as a Data Download and Job Submitter Service. Several files are written in the workspace (excel, sdf, json, txt) and -logically- they appear in the output .zip package of the data download service.

 

 

Problem: A FileCopy writer is used to copy a .dwg file to the output folder. Because the files for the .zip package of the download service are stored an a temporary folder on FME Server, it is not yet possible to have to .dwg also in the download package. The Job Submitter does its job and all my files are written correctly (including the .dwg file). The data download .zip-package does not have this .dwg file and that is my problem. I just need the URL or the location of where the files for the download package are temporarily stored on FME Server, so I could use this URL as a parameter for the output location in the FileCopy Writer.

The documentation online told me this should be the 'FME_SERVER_DEST_DIR' parameter? But when I use this parameter in the FileCopy Writer, it does not work?

 

 

I clearly could use some advice on this... Thanks in advance!

 

 

FME Version: 2017.0.0.0 (Build 17259, 64 bits)

Best answer by rylanatsafe

Hi @jneujens - without seeing your workspace, I am not exactly sure what is occurring (or rather, not occurring)... It may be that we needed to populate the filecopy_dest_filename attribute for the File Copy Writer?

Anyway, I did manage to get this working with the attached workspace in FME Build 17291. Please note it is set to reference the attached text file from FME Server Resources > Temp - this file should be placed there before publishing and running the workspace (or the respective filepath changed).

In this example workspace, everything (Text File, JSON, Excel, File Copy) is correctly placed inside the Data Download .zip file.

 

Note: The File Copy seems to place the file into a 'dataset' subfolder (or named as anything you enter into the DestDataset_FILECOPY Published Parameter) - but this should get you one step closer!

*And a shoutout to @david_r - that's a great template to have in your tool belt!

datadownload-filecopy-test.zip

View original
Did this help you find an answer to your question?

13 replies

david_r
Celebrity
  • david_rCelebrity
  • June 27, 2017

I don't have a specific answer, but one way of trying to understand what's happening could be to publish and run the attached workspace. The log file will contain a dump of all the published, private and internal parameters available to you.

dump-fme-macrovalues.fmwt

Sample dump with parameters shown in blue:

jneujens
Forum|alt.badge.img
  • jneujens
  • Author
  • June 27, 2017
david_r wrote:

I don't have a specific answer, but one way of trying to understand what's happening could be to publish and run the attached workspace. The log file will contain a dump of all the published, private and internal parameters available to you.

dump-fme-macrovalues.fmwt

Sample dump with parameters shown in blue:

Great! I will definitely try it.

 

 

jneujens
Forum|alt.badge.img
  • jneujens
  • Author
  • June 27, 2017

I have been looking into this more:

I was able to write the .dwg (from the FileCopy Writer), .sdf, .json and .xlsx file to a temporary location by making use of the Job Submitter service. So there is no problem, everything's fine.

 

 

Now, when looking at the Data Download Service, my log file shows that my .dwg file from the FileCope Writer is being zipped into the data download package. However, it does not appear in this package when I download this package. I only get the .xlsx, .sdf and .json file. When registering this service I specifically checked that all writers are included in the data download package.

 

 

Someone any suggestions why the FileCopy writer does not work with a Data Download service?

 

 

 

david_r
Celebrity
  • david_rCelebrity
  • June 27, 2017

Try using a "Data File" reader/writer combo rather than the File Copy, that ought to work.

rylanatsafe
Safer
Forum|alt.badge.img+13
  • rylanatsafeSafer
  • Safer
  • Best Answer
  • June 27, 2017

Hi @jneujens - without seeing your workspace, I am not exactly sure what is occurring (or rather, not occurring)... It may be that we needed to populate the filecopy_dest_filename attribute for the File Copy Writer?

Anyway, I did manage to get this working with the attached workspace in FME Build 17291. Please note it is set to reference the attached text file from FME Server Resources > Temp - this file should be placed there before publishing and running the workspace (or the respective filepath changed).

In this example workspace, everything (Text File, JSON, Excel, File Copy) is correctly placed inside the Data Download .zip file.

 

Note: The File Copy seems to place the file into a 'dataset' subfolder (or named as anything you enter into the DestDataset_FILECOPY Published Parameter) - but this should get you one step closer!

*And a shoutout to @david_r - that's a great template to have in your tool belt!

datadownload-filecopy-test.zip

mark2atsafe
Safer
Forum|alt.badge.img+44

One - maybe obvious - suggestion: when you publish the workspace, are you assigning both writers to the Data Download? Otherwise you wouldn't get anything.

When you assign the writer but get no output I think it's usually because no output was sent to the writer. So check that the source file is available and that there are no errors in the log.

mark2atsafe
Safer
Forum|alt.badge.img+44
mark2atsafe wrote:

One - maybe obvious - suggestion: when you publish the workspace, are you assigning both writers to the Data Download? Otherwise you wouldn't get anything.

When you assign the writer but get no output I think it's usually because no output was sent to the writer. So check that the source file is available and that there are no errors in the log.

Also - although I never put this into production - I made a prototype of a data download tool that used FileCopy and wrote a blog about it. See this page: https://blog.safe.com/2013/07/fmeevangelist117/

 

 

jneujens
Forum|alt.badge.img
  • jneujens
  • Author
  • June 28, 2017
mark2atsafe wrote:

One - maybe obvious - suggestion: when you publish the workspace, are you assigning both writers to the Data Download? Otherwise you wouldn't get anything.

When you assign the writer but get no output I think it's usually because no output was sent to the writer. So check that the source file is available and that there are no errors in the log.

Thanks for the suggestions, but those were the first things I checked. All writers are assigned to the data download. Also, the job submitter works fine and without errors.
jneujens
Forum|alt.badge.img
  • jneujens
  • Author
  • June 28, 2017
rylanatsafe wrote:

Hi @jneujens - without seeing your workspace, I am not exactly sure what is occurring (or rather, not occurring)... It may be that we needed to populate the filecopy_dest_filename attribute for the File Copy Writer?

Anyway, I did manage to get this working with the attached workspace in FME Build 17291. Please note it is set to reference the attached text file from FME Server Resources > Temp - this file should be placed there before publishing and running the workspace (or the respective filepath changed).

In this example workspace, everything (Text File, JSON, Excel, File Copy) is correctly placed inside the Data Download .zip file.

 

Note: The File Copy seems to place the file into a 'dataset' subfolder (or named as anything you enter into the DestDataset_FILECOPY Published Parameter) - but this should get you one step closer!

*And a shoutout to @david_r - that's a great template to have in your tool belt!

datadownload-filecopy-test.zip

Thanks! I tried your workspace (and even simplified it) but still no results. If I run this as a Job Submitter service, my file is copied to my preferred location (a folder within the temp folder of the server). But when using it as a Download Service, the download package is empty (but my file is still written to the folder within the temp folder, as with the Job Submitter).

 

 

rylanatsafe
Safer
Forum|alt.badge.img+13
jneujens wrote:
Thanks! I tried your workspace (and even simplified it) but still no results. If I run this as a Job Submitter service, my file is copied to my preferred location (a folder within the temp folder of the server). But when using it as a Download Service, the download package is empty (but my file is still written to the folder within the temp folder, as with the Job Submitter).

 

 

@jneujens - The plot thickens! I did my due diligence and tested on FME Server 2017 Build 17259 Win64 - and observed the same successful translation... I have attached the output dataset from Data Download and the log file of the translation.

 

1. Can you provide your full log file?

 

2. What is your FME Server architecture? Are there any UNC paths in the mix? (The FME Server 2017 I have tested with is an Express Installation with all default paths.)

 

 

If you cannot share this data publicly, please feel free to contact support referencing this Knowledge Center Q&A.;

 

archive-log-17259.zip

 

jneujens
Forum|alt.badge.img
  • jneujens
  • Author
  • June 29, 2017

RylanAtSafe , thanks for your help! I will check everything again and try to make your example work.

 

 

I will get back to you as soon as I have all the information. Thanks!
mark2atsafe
Safer
Forum|alt.badge.img+44
jneujens wrote:
Thanks for the suggestions, but those were the first things I checked. All writers are assigned to the data download. Also, the job submitter works fine and without errors.
I figured. But sometimes we have to ask!

 

 

jneujens
Forum|alt.badge.img
  • jneujens
  • Author
  • August 3, 2017
rylanatsafe wrote:
@jneujens - The plot thickens! I did my due diligence and tested on FME Server 2017 Build 17259 Win64 - and observed the same successful translation... I have attached the output dataset from Data Download and the log file of the translation.

 

1. Can you provide your full log file?

 

2. What is your FME Server architecture? Are there any UNC paths in the mix? (The FME Server 2017 I have tested with is an Express Installation with all default paths.)

 

 

If you cannot share this data publicly, please feel free to contact support referencing this Knowledge Center Q&A.;

 

archive-log-17259.zip

 

 

A quick update: your workspace (and other examples) finally worked. The problem is not related to FME Server but an error in the workspace itself. Thanks again!

Reply


Helpful Members This Week

Community Stats

31,141
Posts
118,067
Replies
38,761
Members

Latest FME

Terms & ConditionsCookie settings

Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.