Skip to main content
Solved

read only the newest file from a directory


vki
Contributor
Forum|alt.badge.img+7
  • Contributor

In a directory I have some textfiles and their names contain a date (xy... 2017-12-13.txt). Each month there's a new file. I want to read only the newest file, but I don't want to change the name in the reader each month. So my idea is to read them all (or at least one record to get the fme_basename), compare the dates and then read only the newest file.

So I tried various approaches (first extracting the date out of the filename, then use Aggregator, Sorter, VariableSetter...) but I don't manage to compare the dates of the features as I can't see a way to set the initial value...

Any ideas?

Best answer by taojunabc

If your text files have a fix format (xy... 2017-12-13.txt), You can use [Direcotry and File Pathnames] reader to read file's infomation(path_rootname:file name,path_windows:file path etc.) . Then creat a new attribute (@Right(@Value(path_rootname),10)) to get date part. then sort it by Alphabetic,Descending. Get top 1 record by sampler, You should get the newest file name(path_rootname).

getlastdatefile.fmw

View original
Did this help you find an answer to your question?

6 replies

debbiatsafe
Safer
Forum|alt.badge.img+20

Hi @vki

You use a Directory and Path reader to read only .txt files from your directory. This reader outputs a feature for each file matching your criteria along with feature attributes such as path_filename (file name) and path_windows/paths_unix (file path), etc.

Assuming your file naming is consistent (name YYYY-MM-DD.txt), you can create a new date attribute by using the substring extractor function of the AttributeManager text editor to extract the date part of file name without the file extension (eg. 2017-12-13). Use the StringReplacer to replacer non-numeric characters (dashes in your case) and then use a Tester to verify the attribute values are numbers only. Attach a Sorter to sort by descending numeric values and use the Sampler to get the first feature which should contain the file path of the newest file. Use the path_windows attribute as the input for dataset parameter with a FeatureReader or WorkspaceRunner.

Hope this helps,

Debbi


taojunabc
Participant
Forum|alt.badge.img+6
  • Participant
  • Best Answer
  • March 3, 2018

If your text files have a fix format (xy... 2017-12-13.txt), You can use [Direcotry and File Pathnames] reader to read file's infomation(path_rootname:file name,path_windows:file path etc.) . Then creat a new attribute (@Right(@Value(path_rootname),10)) to get date part. then sort it by Alphabetic,Descending. Get top 1 record by sampler, You should get the newest file name(path_rootname).

getlastdatefile.fmw


vki
Contributor
Forum|alt.badge.img+7
  • Author
  • Contributor
  • March 3, 2018

Thanks a lot to both of you. I didn't know the Directory and Path-Reader! Now it works like a charm!

Vera


fmelizard
Contributor
Forum|alt.badge.img+17
  • Contributor
  • March 4, 2018
vki wrote:

Thanks a lot to both of you. I didn't know the Directory and Path-Reader! Now it works like a charm!

Vera

Glad it works. In case others come across this and do not have their files named as nicely as you did, it is possible to ask the PATH reader to give back a bunch of metadata on each file, including its last modified date `path_modified_date' -- just set the PATH reader to "Retrieve File Properties" in its parameters when you create it.

 

 

 


ksu_geographer
Participant
Forum|alt.badge.img

Rather than going to the trouble of parsing out the date from the filename, wouldn't it be easier to rely on the path_created_date, or path_modified_date from the Directory and Path reader? You will have to check the option to Retrieve file properties to 'Yes', but that would be an easier way to determine the latest file.


billybob
Contributor
Forum|alt.badge.img+6
  • Contributor
  • May 21, 2020

Does anyone know how to use this same or similar technique to find the latest file from a One Drive folder. I'm able to select a file from the One Drive folder, but I haven't figured out how to "search" through the list of files for the latest one. This might not be the correct approach, but if you know of one I'd greatly appreciate any insight.

Thanks,

Frank


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings