Skip to main content

Hi,

Newbie type question. I am currently loading in files to a output on a weekly basis. Issue has been I have to edit the file in the workbench with the new filename.

The file I upload changes on a daily basis with its name. How can FME pick up the latest file that has been updated without having to change the file path on a daily basis?

I have tried to use this https://knowledge.safe.com/questions/65203/read-only-the-newest-file-from-a-directory.html method but unfortunately I am not getting it to work. I also do not need to generate any further data to my existing table, just was interested in the function where it can automatically pick up the latest file at folder level by date modified.

 

Thank You

1. As your source, use a Directory and File Pathnames reader.

 

2. A Sorter that sorts descending on path_modified_date.

 

3. A Sampler that only lets the first feature pass.

 

4. A FeatureReader where Dataset is set to path_windows.

1. As your source, use a Directory and File Pathnames reader.

 

2. A Sorter that sorts descending on path_modified_date.

 

3. A Sampler that only lets the first feature pass.

 

4. A FeatureReader where Dataset is set to path_windows.

This is why I believe the FeatureReader and FeatureWriter are the biggest game changing transformers in a long time.


1. As your source, use a Directory and File Pathnames reader.

 

2. A Sorter that sorts descending on path_modified_date.

 

3. A Sampler that only lets the first feature pass.

 

4. A FeatureReader where Dataset is set to path_windows.

path2none.fmwThanks Dan - I have put this together, is this correct in the way its working. In terms of then adding the rest of my workbench in terms of filtering out records, does this get attached towards the schema.

path2none.fmw


path2none.fmwThanks Dan - I have put this together, is this correct in the way its working. In terms of then adding the rest of my workbench in terms of filtering out records, does this get attached towards the schema.

path2none.fmw

Almost perfect I think. 🙂

You just need to change from ascending to descending in the Sorter, to get the newest file instead of the oldest.


Almost perfect I think. 🙂

You just need to change from ascending to descending in the Sorter, to get the newest file instead of the oldest.

Thanks Dan, i got this working now. In regards to seeing data after the featurereader, is there any specific changes that need to be made? At the featurereader I only get information in regards to the file it is will pick up. Thanks again.


Thanks Dan, i got this working now. In regards to seeing data after the featurereader, is there any specific changes that need to be made? At the featurereader I only get information in regards to the file it is will pick up. Thanks again.

Have you set the Format in the FeatureReader so that FME knows what it's looking at? If the Format is set correctly, and the Dataset points to a valid file (through the path_windows attribute) FME should be able to read the file and extract the features. If it still doesn't work, please post a screen dump of the FeatureReader settings or the .fmw.


Have you set the Format in the FeatureReader so that FME knows what it's looking at? If the Format is set correctly, and the Dataset points to a valid file (through the path_windows attribute) FME should be able to read the file and extract the features. If it still doesn't work, please post a screen dump of the FeatureReader settings or the .fmw.

Hi Dan, apologise for delay in response. I am getting the records to come through it seems as the file count of the file is 227. But I am getting a empty table with no attributes. It is a excel file. Capture.JPG path2none.fmw


Hi Dan, apologise for delay in response. I am getting the records to come through it seems as the file count of the file is 227. But I am getting a empty table with no attributes. It is a excel file. Capture.JPG path2none.fmw

At the bottom of the FeatureReader parameters dialog box you've got "<Generic> Port". You might need to go in there and expose your attributes to get them to show.


At the bottom of the FeatureReader parameters dialog box you've got "<Generic> Port". You might need to go in there and expose your attributes to get them to show.

That seems to be the issue. I have manually setup the attributes, so this now has picked up the attributes. This process is good as long as the source file does not change.


Reply