Skip to main content
  • EN

I have a bunch of ZIP files which we download the contain data in 3 formats, shape, csv and file gdb. The zips also contain numerous other files such as pdf and xml, which we don't need to read.

 

Now the tricky bit, I am using a PATH reader to check a directory to see what ZIPs are there and to only load the ones that have arrived in the last 24hrs. The load is triggered in a Sub workbench where the Path is passed to a FeatureReader set to Generic. I can add I file type filter to the path that is passed by adding **\\*.shp], for example to the path of the zip found by the PATH reader and can read the ZIP 3 times with a different filter each time by building this string

""@Value(path_windows))**\\*.shp]" "@Value(path_windows))**\\*.csv]" "@Value(path_windows))**\\*.shp]""

 

but can't work out how to add filters for csv and gdb to read the file once and look for shp, csv or gdb.

 

The above works, but I get error messages for the iterations that don't find the file type filtered for, which is messy.

 

I could hard code the file type for each file, but want to future proof for possible changes in the future

I think this should work here

"@Value(path_windows)\**\*.{shp,csv,gdb}"

 

I think this should work here

"@Value(path_windows)\**\*.{shp,csv,gdb}"

 

Thanks @ebygomm​ , I knew there was some way to specify more that one extension, though I found I had to add the square brackets as well

@Value(path_windows)s**\*.{shp,csv,gdb}]

This only works for me whenever I choose the input format as design (i.e Esri shapefile). if chosen “<Guess format name from extension>” it won’t work out

Hi @felipeverdu 

I may have gotten it to work on my end.


Could you share a screenshot of how you set up your reader? 
Also, which version of FME are you using? 

 

Crystal 

Hi @crystalatsafe 

 

I do better than that so i upload my own workspace (fmwt) :)

I see that it works fine for most input formats but Json and OGC Geopackage.

Let me know how it should be done.

Br,

Felipe Verdú

Hi @felipeverdu 

 

I was able to reproduce the same issue with the generic format FeatureReader. I see that json and Geopackage files are being filtered through the rejected port.

I have filed a ticket for our team to take a closer look. ointernal: FMEENGINE-82928]

 

I apologize for any inconvenience and I will make sure to keep you updated.

 

Crystal 

Hi @felipeverdu 

 

I’ve been looking into finding a workaround for this. I did find a way to filter those records through the generic port of the Generic Feature Reader. However, many files are missing from the final output when compared to the JSON Feature Reader you have set up.

It looks like the workaround you have found is the best option for your workspace but I wanted to share what I found just in case it helps. 

I was able to filter records through the generic port by creating a new attribute with conditional values where the value is set to the fme format short name.

 

Then in the parameters of the Generic Feature Reader I selected that newly created attribute as the input format.  

This was able to filter records through the generic port but doesn’t read in the files as well as the JSON Feature Reader does. 

There have been some ideas around adding the option to select multiple formats in the Feature Reader. If you would like, please feel free to add a comment to this idea: Limit or chose preference of which Feature Types to read in Generic FeatureReader as our team reviews these. 

Please let me know if you have any questions. 

Crystal

Hi @crystalatsafe 

Nice worksaround!!!. I like it and it solved pretty much the challenges here. I found just one json file not be able to be read but I believe there is something else.

I have added a commet to the idea so it can be taken into account.

Hi @felipeverdu 

I’m happy to hear that the workaround worked well for you! 

A huge thanks to @jdh for proving this workaround in the community post: Can I change the default formats read by the Generic Reader? 

Reply


Community Stats

30,351
Posts
115,336
Replies
38,078
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.