Skip to main content
  • EN

Hello,

I have a bit of a problem that I can't find a solution to.

I have a list of directories and sub directories. Some contain shapefiles. Some contain tables, gdbs etc. Eg:

S:\\NEW\\170530\\test.shp

S:\\NEW\\170529\\NewData\\NewTest.shp

S:\\OLD\\170428\\test.xlsx

S:\\OLD\\170428\\MoreData\\MoreTest.shp

There are more folders within those dated directories. The root drive is always the same "S:\\"

The root data directories (eg, OLD, NEW) will be provided by users as a user parameter. There won't be any navigating to those specific directories. How can I have a workbench that reads all the shapefiles nested in all those different directories, once the root data directory has been passed to it?

Everything from the PATH reader up to the FeatureReader only specifies up to the dated directory based on user input. Eg:

S:\\NEW\\170530

I saved the directory path and passing the following to the FeatureReader: S:\\NEW\\170530\\**

Now it works.. but with a couple of drawbacks. For one, if it comes across a directory that has no shapefiles, the process terminates. I want it to skip and continue when that happens.

Next, it is very slow. Even though I specified in the FeatureReader to only read shapefiles, it reads dbf and a bunch of other types and it reads really slowly. Something that a simple shape file reader takes 1 minute to read, this one took 10 minutes or more. I need to speed up the process and if a workaround can be found, I would really appreciated.

Any help will be appreciated and thanks!

Hi @fariyafarhad, if you create a user parameter (e.g. called 'ROOT') to allow the users to select a possible root folder name ('NEW', 'OLD', etc.), you can set this expression to the Dataset parameter of the FeatureReader to read all Shapefile files under the specified root folder and its sub folders.

S:\$(ROOT)\**\*.shp

0684Q00000ArKE9QAN.png

Hi @fariyafarhad, if you create a user parameter (e.g. called 'ROOT') to allow the users to select a possible root folder name ('NEW', 'OLD', etc.), you can set this expression to the Dataset parameter of the FeatureReader to read all Shapefile files under the specified root folder and its sub folders.

S:\$(ROOT)\**\*.shp

0684Q00000ArKE9QAN.png

And, there are two possible ways to avoid the termination when there were no Shapefile files under the specified root directory.

 

1. Connect a transformer (e.g. Logger or Junction) to the <Rejected> port of the FeatureReaderor.

 

or 

 

2. Select "Continue to Translation" in the Rejected Feature Handling parameter of the workspace (Navigator > Workspace Parameters).

 

 

Hi @fariyafarhad, if you create a user parameter (e.g. called 'ROOT') to allow the users to select a possible root folder name ('NEW', 'OLD', etc.), you can set this expression to the Dataset parameter of the FeatureReader to read all Shapefile files under the specified root folder and its sub folders.

S:\$(ROOT)\**\*.shp

0684Q00000ArKE9QAN.png

Thank you so much @takashi 

 

That simple tip worked perfectly for me!

Reply


Community Stats

30,092
Posts
114,247
Replies
37,720
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.