How to download and save multiple DTM zip files from CSV with URLs. Need help with settings for extracting individual files | Community

How to download and save multiple DTM zip files from CSV with URLs. Need help with settings for extracting individual files

Page 1 / 1

I think you can read URLs from the CSV file with CSV reader, extract filename part from each URL with FielnamePartExtractor, and download the DTM zip file with HTTPCaller into a specific folder.

HTTPCaller parameters setting looks like this. Assuming that "url" stores a URL read from CSV, "DONWLOD_FOLDER" is a user parameter spacifies a folder path to which you would like to downlaod the DTM zip files, and "_filename" stores a zip filename extracted from URL with FilenamePartExtractor.

I think you can read URLs from the CSV file with CSV reader, extract filename part from each URL with FielnamePartExtractor, and download the DTM zip file with HTTPCaller into a specific folder.

HTTPCaller parameters setting looks like this. Assuming that "url" stores a URL read from CSV, "DONWLOD_FOLDER" is a user parameter spacifies a folder path to which you would like to downlaod the DTM zip files, and "_filename" stores a zip filename extracted from URL with FilenamePartExtractor.

Thanks. I did the same thought. In my case the csv already contains a FILENAME column, but I’m not able to set this FILENAME in the Output Filename field.

Thanks. I did the same thought. In my case the csv already contains a FILENAME column, but I’m not able to set this FILENAME in the Output Filename field.

Could you please share a few rows of the CSV table?

Thanks. I did the same thought. In my case the csv already contains a FILENAME column, but I’m not able to set this FILENAME in the Output Filename field.

This is a simplified version of my csv with only 2 rows, for my tests

Thanks. I did the same thought. In my case the csv already contains a FILENAME column, but I’m not able to set this FILENAME in the Output Filename field.

I was able to download the files from urls described in the CSV table.

However, it took very long time. Possibly you may have to set a large value to the Transfer Timeout Length parameter of the HTTPCaller.

csv-reader-httpcaller

Thanks. I did the same thought. In my case the csv already contains a FILENAME column, but I’m not able to set this FILENAME in the Output Filename field.

I still receive translation FAILED message as:

2023-12-02 21:47:43|      |     |INFORM|Starting translation...
2023-12-02 21:47:44|      |     |INFORM|Undefined macro `C:\Users\Francesco\Desktop\dtm austria' dereferenced in file `C:\Users\Francesco\Documents\FME\Workspaces\wb-xlate-1701550060797_22284'
2023-12-02 21:47:44|      |     |INFORM|Program Terminating
2023-12-02 21:47:44|      |     |INFORM|Translation FAILED.

I have the workspace shown below:

Screenshot 2023-12-02 214707

Thanks. I did the same thought. In my case the csv already contains a FILENAME column, but I’m not able to set this FILENAME in the Output Filename field.

"C:\Users\Francesco\Desktop\dtm austria" is a folder path, is not a User Parameter. You don't need to quote it by $( ).

Try setting this string to the Output Filename parameter.

C:\Users\Francesco\Desktop\dtm austria\@Value(FILENAME)

See here to learn more aboud User Parameters.

https://docs.safe.com/fme/html/FME-Form-Documentation/FME-Form/Workbench/published_private_parameters.htm

Thanks. I did the same thought. In my case the csv already contains a FILENAME column, but I’m not able to set this FILENAME in the Output Filename field.

Thanks! That worked fine. Now i understand where the problem was.

Reply