HTTP Caller and renaming of response files on-the-fly

Hi @eshirinyan,

If you set the HTTPCaller to save the response body to an attribute, you can follow it with an AttributeFileWriter to write the contents of that attribute to a file. This transformer will allow you to set the output filename from an attribute, and will write every feature to its own file.

Thank you, @DaveAtSafe!

Just checked, it works fine! Hope there is no limitations for large feature sets too.

@takashi Thanks a lot! This is the method I was searching for. FME is very flexible

@takashi 

I have a problem similar to one you've addressed previously.

I am retrieving text files/CSVs from a remote server of a consulting company using HTTPCaller. My goal is to create a copy of each file in my local database to provide access for other data engineers in my company and to process the data further for various purposes.

Currently, I achieve this by saving the files to my local disk, then using a FeatureReader/Writer to write them to my local database, and a FeatureDeleter to delete the downloaded files immediately after they are written to the database.

However, this method is not optimal and poses security risks since some of the data is sensitive. Therefore, I want to directly write the incoming text file data to my local database with minor attribute renaming.

How can I connect my HTTPCaller, which downloads to a file, directly to my writer?

As this is a 6 year old question, the preferred route is to create a new one.

One way to solve what you need is to use the TempPathnameCreator to create a temp folder where the files from the httpcaller can be stored and read. This takes away the need to clean up because the files are written to temp files.

Another way is to set the HTTPCaller to save response body to Attribute. You can then use that to send the _response_body to a Text File FeatureWriter. _response_body will nee to be renamed to text_line_data.