Solved

read multiple files from a folder that all contain a specific text string in the title

  • 18 October 2021
  • 4 replies
  • 51 views
O

I am trying to read in multiple files from a folder at once which contain a specific text string in the title and not any files that don't contain that string. I want it to be dynamic, such that if the files in the folder change, the reader will update as I want to put this at the start of a workflow which will put all the features from each individual source file into a single file and perform some transformations with that data. I have managed to figure out making a reader that looks at a folder and all its sub folders, but it reads in all the files in the folder, some of which I don't need/want. Is there a way I can apply a search filter to the reader so it only brings in files that have certain terms in the title?

icon

Best answer by ebygomm 18 October 2021, 18:10

View original

4 replies

ebygomm
Rank
Userlevel 1
Badge +21

What reader are you using? You can use wildcards in the path to only include files that contain certain terms

O

Sorry for my ignorance I've only just started using FME. How would I know which reader I am using? I am just using the "add reader" function in the workbench. I am using Mapinfo Tab as the format and the dataset is a folder on my server which I am selecting using the "select multiple folders/files" option and I am including all sub folders as the files are split up into sub folders. I'm using "single Merged Feature type" for the workflow option.

ebygomm
Rank
Userlevel 1
Badge +21

Sorry for my ignorance I've only just started using FME. How would I know which reader I am using? I am just using the "add reader" function in the workbench. I am using Mapinfo Tab as the format and the dataset is a folder on my server which I am selecting using the "select multiple folders/files" option and I am including all sub folders as the files are split up into sub folders. I'm using "single Merged Feature type" for the workflow option.

No problem, if you are reading mapinfo files you are using a mapinfo reader. If you have added it and chosen a folder it should have created a published parameter called something like SourceDataset_MAPINFO

This will look something like 

C:\folders\myfolder\*.tab

If you want it for example to only read tab files that contain the word test you could amend to

C:\folders\myfolder\*test*.tab

 

 

O

No problem, if you are reading mapinfo files you are using a mapinfo reader. If you have added it and chosen a folder it should have created a published parameter called something like SourceDataset_MAPINFO

This will look something like 

C:\folders\myfolder\*.tab

If you want it for example to only read tab files that contain the word test you could amend to

C:\folders\myfolder\*test*.tab

 

 

Thanks this has done the trick. Just for reference, since I am also using sub folders the path ended up looking like:

 

 C:\folders\myfolder\**\*test*.tab

 

Reply


Community Stats

28,700
Posts
109,456
Replies
36,549
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.