@kalbert Hi there, thanks for your question. If I understand your ask correctly, I believe you can point the FeatureReader to the .zip, and then use the "Feature Types to Read" parameter to filter out all of the files which contain "POINT_".
If you set up the FeatureReader for Format and Dataset first, and then click the ellipses next to Feature Types to Read, a new window will pop open displaying the contents of the .zip. Make sure to deselect the "Select All" option. Next, at the bottom left of this window is a Filter input box. If you enter "POINTS_" into the filter, that should result in the FeatureReader picking up just those points shapefiles. Now you can select the "Select All" option to grab just those "POINTS_" shapefiles.
Here are a couple of screenshots showing the process:
- Set up the FeatureReader and click the ellipses next to "Feature Types to Read.
- Now deselect "Select All" and enter your file name filter into the "Filter" text input, at bottom left.
Give this a try and let us know how it goes! Hopefully it will do the trick 🙂
Hi natalie,
No . Your solution is the default way when the content of the zip is always the same. As I wrote I have a monthly run, where I don't know the exact content of the zip. Known is only the naming convention of the files so that I can distinguish point and line classes by name.
The process runs automated on FME server, so I can't let the Feature Reader read the zip first and then manually select the classes. That's why I tried to work with wild cards but with no success.
@kalbert My apologies, I forgot the FME Server context! I don't think the parameters of the FeatureReader can accept wildcards or other similar special characters like it. I'm not certain on this, but I've tried several different methods, and cannot get the wildcard to work. The only exception to this seems to be in the Dataset parameter, which can accept and use the wildcard.
When you feed a .zip into the FeatureReader Dataset parameter, it can be treated just like any other folder, where you can go a couple levels beyond the .zip and get right into the compressed folder contents. This is where you can use the wildcard to get at those POINTS_ shapefiles. I'll demonstrate.
I have a zipped file called "Data.zip." It contains two shapefiles and a CAD drawing file. One of my shapefiles starts with "Forward", and I'm interested in processing just that shapefile. My shapefile of interest is not stored in any subfolders within my .zip. My .zip is located at the following path:
D:\\MyDrive\\MyProject\\MyWorkingFolder\\Inputs\\Data.zip
I can get right into that .zip file and access just my "Forward" shapefile by inputting the following into the FeatureReader Dataset parameter:
D:\\MyDrive\\MyProject\\MyWorkingFolder\\Inputs\\Data.zip\\Data\\Forward*.shp
I ran the above FeatureReader and received only my expected shapefile as output.
As long as your shapefiles are not stored in subfolders within the .zip, you can repeat the name of the .zip as if it's an extracted folder, and then put in your POINTS_*.shp wildcard portion at the end.
If they are all in the same subfolder, you can just append the subfolder name to the path. If they're in separate subfolders, you'd likely have to use the "Select Multiple Folders / Files" option from the drop-down beside the Dataset parameter, and specify each file path with subfolders to grab those, or play with the Subfolders option therein.
I know you mentioned that you don't know the exact contents of the .zip, but if you do know the folder structure / path to your shapefiles within the .zip, then this method may work for your use case. You'd just need to adjust the published parameter that you're using for "Dataset" to manually add-on that extra path portion after the .zip, so that you can use the wildcard.
Please don't hesitate to ask for clarification on this if needed, and let me know if this works for you!
@kalbert My apologies, I forgot the FME Server context! I don't think the parameters of the FeatureReader can accept wildcards or other similar special characters like it. I'm not certain on this, but I've tried several different methods, and cannot get the wildcard to work. The only exception to this seems to be in the Dataset parameter, which can accept and use the wildcard.
When you feed a .zip into the FeatureReader Dataset parameter, it can be treated just like any other folder, where you can go a couple levels beyond the .zip and get right into the compressed folder contents. This is where you can use the wildcard to get at those POINTS_ shapefiles. I'll demonstrate.
I have a zipped file called "Data.zip." It contains two shapefiles and a CAD drawing file. One of my shapefiles starts with "Forward", and I'm interested in processing just that shapefile. My shapefile of interest is not stored in any subfolders within my .zip. My .zip is located at the following path:
D:\\MyDrive\\MyProject\\MyWorkingFolder\\Inputs\\Data.zip
I can get right into that .zip file and access just my "Forward" shapefile by inputting the following into the FeatureReader Dataset parameter:
D:\\MyDrive\\MyProject\\MyWorkingFolder\\Inputs\\Data.zip\\Data\\Forward*.shp
I ran the above FeatureReader and received only my expected shapefile as output.
As long as your shapefiles are not stored in subfolders within the .zip, you can repeat the name of the .zip as if it's an extracted folder, and then put in your POINTS_*.shp wildcard portion at the end.
If they are all in the same subfolder, you can just append the subfolder name to the path. If they're in separate subfolders, you'd likely have to use the "Select Multiple Folders / Files" option from the drop-down beside the Dataset parameter, and specify each file path with subfolders to grab those, or play with the Subfolders option therein.
I know you mentioned that you don't know the exact contents of the .zip, but if you do know the folder structure / path to your shapefiles within the .zip, then this method may work for your use case. You'd just need to adjust the published parameter that you're using for "Dataset" to manually add-on that extra path portion after the .zip, so that you can use the wildcard.
Please don't hesitate to ask for clarification on this if needed, and let me know if this works for you!
@natalieatsafe thanks for feedback. I played with your approach and figured out that it works with the wildcard setting as below:
$(path_to_zip)/**/LINE_*.zip
Although the zip file has no subfolders I have to add the /**/ part in the Dataset string.
My $(path_to_zip) parameter is of type Filename(existing) (e.g. C:\\dev\\delivery\\month\\Myshapes.zip).
(Side note: I also tried it as parameter type Folder assuming a zip is treated as a folder, but this was failing).
Next step: Test the approach on FME Server and provide the result here for the community.
@kalbert Happy to hear it's looking like a possible solution! And nice use of the recursive wildcard, I myself overlook it too often as an option in paths. I look forward to seeing if this works on the FME Server side.