Hello everyone, I would really appreciate your support.
I have a CSV file that contains a column named "LINK" (file attached), which includes the URLs to download orthophotos (.tif).
I’d like to understand how to download them all using FME:
How should the HTTPCaller transformer be properly configured?
Is it necessary to create an attribute to assign a name to the orthophoto files?
Could someone share an example script or workflow?
Thank you very much for your help and support!
The most basic solution will only need a CSV reader and HTTPCaller.
The CSV reader will output a feature per row of the file with an attribute called LINK
Connect the reader to an HTTPCaller. Set the Request URL as the LINK attribute, the HTTP Method as GET.
Under Response Handling, save the response body to a folder and specify the folder you want to save the images to.
You’ll note that this approach automatically assigns a file name, it doesn’t take the file name from the url.
This is easy to resolve with a slight change.
Between the Reader and the HTTPCaller, add a FileNamePartExtractor (leave it as default settings). In the HTTPCaller, change it to save to a File, and set the output path using the _filename attribute from the FIleNamePartExtractor.
This will result in the files being downloaded and saved with their correct filename.
Your resulting workspace should look similar to:
A helpful transformer to use when building and testing this, is the Sampler. Place that right after the reader and it will allow you to sample (eg first 5 features) the incoming data so you can test with a smaller subset before running it across the entire dataset.