Solved

Getting temporary folder of Data Download Service

jneujens
Badge

Hi all,

Workspace A is uploaded to FME Server as a Data Download and Job Submitter Service. Several files are written in the workspace (excel, sdf, json, txt) and -logically- they appear in the output .zip package of the data download service.

 

 

Problem: A FileCopy writer is used to copy a .dwg file to the output folder. Because the files for the .zip package of the download service are stored an a temporary folder on FME Server, it is not yet possible to have to .dwg also in the download package. The Job Submitter does its job and all my files are written correctly (including the .dwg file). The data download .zip-package does not have this .dwg file and that is my problem. I just need the URL or the location of where the files for the download package are temporarily stored on FME Server, so I could use this URL as a parameter for the output location in the FileCopy Writer.

The documentation online told me this should be the 'FME_SERVER_DEST_DIR' parameter? But when I use this parameter in the FileCopy Writer, it does not work?

 

 

I clearly could use some advice on this... Thanks in advance!

 

 

FME Version: 2017.0.0.0 (Build 17259, 64 bits)

icon

Best answer by rylanatsafe 27 June 2017, 22:28

View original

13 replies

david_r
Rank
Userlevel 4

I don't have a specific answer, but one way of trying to understand what's happening could be to publish and run the attached workspace. The log file will contain a dump of all the published, private and internal parameters available to you.

dump-fme-macrovalues.fmwt

Sample dump with parameters shown in blue:

jneujens
Badge

I don't have a specific answer, but one way of trying to understand what's happening could be to publish and run the attached workspace. The log file will contain a dump of all the published, private and internal parameters available to you.

dump-fme-macrovalues.fmwt

Sample dump with parameters shown in blue:

Great! I will definitely try it.

 

 

jneujens
Badge

I have been looking into this more:

I was able to write the .dwg (from the FileCopy Writer), .sdf, .json and .xlsx file to a temporary location by making use of the Job Submitter service. So there is no problem, everything's fine.

 

 

Now, when looking at the Data Download Service, my log file shows that my .dwg file from the FileCope Writer is being zipped into the data download package. However, it does not appear in this package when I download this package. I only get the .xlsx, .sdf and .json file. When registering this service I specifically checked that all writers are included in the data download package.

 

 

Someone any suggestions why the FileCopy writer does not work with a Data Download service?

 

 

 

david_r
Rank
Userlevel 4

Try using a "Data File" reader/writer combo rather than the File Copy, that ought to work.

rylanatsafe
Rank
Badge +11

Hi @jneujens - without seeing your workspace, I am not exactly sure what is occurring (or rather, not occurring)... It may be that we needed to populate the filecopy_dest_filename attribute for the File Copy Writer?

Anyway, I did manage to get this working with the attached workspace in FME Build 17291. Please note it is set to reference the attached text file from FME Server Resources > Temp - this file should be placed there before publishing and running the workspace (or the respective filepath changed).

In this example workspace, everything (Text File, JSON, Excel, File Copy) is correctly placed inside the Data Download .zip file.

 

Note: The File Copy seems to place the file into a 'dataset' subfolder (or named as anything you enter into the DestDataset_FILECOPY Published Parameter) - but this should get you one step closer!

*And a shoutout to @david_r - that's a great template to have in your tool belt!

datadownload-filecopy-test.zip

mark2atsafe
Rank
Userlevel 4
Badge +25

One - maybe obvious - suggestion: when you publish the workspace, are you assigning both writers to the Data Download? Otherwise you wouldn't get anything.

When you assign the writer but get no output I think it's usually because no output was sent to the writer. So check that the source file is available and that there are no errors in the log.

mark2atsafe
Rank
Userlevel 4
Badge +25

One - maybe obvious - suggestion: when you publish the workspace, are you assigning both writers to the Data Download? Otherwise you wouldn't get anything.

When you assign the writer but get no output I think it's usually because no output was sent to the writer. So check that the source file is available and that there are no errors in the log.

Also - although I never put this into production - I made a prototype of a data download tool that used FileCopy and wrote a blog about it. See this page: https://blog.safe.com/2013/07/fmeevangelist117/

 

 

jneujens
Badge

One - maybe obvious - suggestion: when you publish the workspace, are you assigning both writers to the Data Download? Otherwise you wouldn't get anything.

When you assign the writer but get no output I think it's usually because no output was sent to the writer. So check that the source file is available and that there are no errors in the log.

Thanks for the suggestions, but those were the first things I checked. All writers are assigned to the data download. Also, the job submitter works fine and without errors.
jneujens
Badge

Hi @jneujens - without seeing your workspace, I am not exactly sure what is occurring (or rather, not occurring)... It may be that we needed to populate the filecopy_dest_filename attribute for the File Copy Writer?

Anyway, I did manage to get this working with the attached workspace in FME Build 17291. Please note it is set to reference the attached text file from FME Server Resources > Temp - this file should be placed there before publishing and running the workspace (or the respective filepath changed).

In this example workspace, everything (Text File, JSON, Excel, File Copy) is correctly placed inside the Data Download .zip file.

 

Note: The File Copy seems to place the file into a 'dataset' subfolder (or named as anything you enter into the DestDataset_FILECOPY Published Parameter) - but this should get you one step closer!

*And a shoutout to @david_r - that's a great template to have in your tool belt!

datadownload-filecopy-test.zip

Thanks! I tried your workspace (and even simplified it) but still no results. If I run this as a Job Submitter service, my file is copied to my preferred location (a folder within the temp folder of the server). But when using it as a Download Service, the download package is empty (but my file is still written to the folder within the temp folder, as with the Job Submitter).

 

 

rylanatsafe
Rank
Badge +11
Thanks! I tried your workspace (and even simplified it) but still no results. If I run this as a Job Submitter service, my file is copied to my preferred location (a folder within the temp folder of the server). But when using it as a Download Service, the download package is empty (but my file is still written to the folder within the temp folder, as with the Job Submitter).

 

 

@jneujens - The plot thickens! I did my due diligence and tested on FME Server 2017 Build 17259 Win64 - and observed the same successful translation... I have attached the output dataset from Data Download and the log file of the translation.

 

1. Can you provide your full log file?

 

2. What is your FME Server architecture? Are there any UNC paths in the mix? (The FME Server 2017 I have tested with is an Express Installation with all default paths.)

 

 

If you cannot share this data publicly, please feel free to contact support referencing this Knowledge Center Q&A.;

 

archive-log-17259.zip

 

jneujens
Badge

RylanAtSafe , thanks for your help! I will check everything again and try to make your example work.

 

 

I will get back to you as soon as I have all the information. Thanks!
mark2atsafe
Rank
Userlevel 4
Badge +25
Thanks for the suggestions, but those were the first things I checked. All writers are assigned to the data download. Also, the job submitter works fine and without errors.
I figured. But sometimes we have to ask!

 

 

jneujens
Badge
@jneujens - The plot thickens! I did my due diligence and tested on FME Server 2017 Build 17259 Win64 - and observed the same successful translation... I have attached the output dataset from Data Download and the log file of the translation.

 

1. Can you provide your full log file?

 

2. What is your FME Server architecture? Are there any UNC paths in the mix? (The FME Server 2017 I have tested with is an Express Installation with all default paths.)

 

 

If you cannot share this data publicly, please feel free to contact support referencing this Knowledge Center Q&A.;

 

archive-log-17259.zip

 

 

A quick update: your workspace (and other examples) finally worked. The problem is not related to FME Server but an error in the workspace itself. Thanks again!

Reply


Community Stats

28,670
Posts
109,366
Replies
36,531
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.