Solved

Search Time optimization in FeatureReader Files

  • 10 February 2020
  • 2 replies
  • 2 views
C
Badge +1

Hi! I have an Excel file with ID numbers of different train stops, 20 of them. I wanted to find all DWG files that are in the subfolder Z\\Modell\\.*03.dwg but it takes around 1 h to run, at this point is it faster just to find them manually and run the workspace. Is there any way to improve the search time?

 

The "root" folder is a drive, but all files are within a folder is called "4\\_CAD" so I limit the search to only the subfolders of my starting folder. Trying to make it faster my only use the subfolder I know contain the other folders and files.

I was thinking maybe change it to have 2 feature writers after eachother instead, one that only searches for the correct folder then look for the files within the found folders. Change allowed path type to only folders then only files.

 

Other suggestions?

Also I figured if I have 60 subfolders, for every feature read it will look in all 60 of them 20 times over, that's probably why it takes such a long time, is there any smarter way?

 

root_folder:

"R:\\5410\\10285959\\4_CAD"

One Example subfolder containing a DWG:

"R:\\5410\\10285959\\4_CAD\\04\\0480\\04800800\\04800800R009\\Z\\Modell"

Example TrainID from the Excel File provided, so its [ID]+\\Z\\Modell

"04800800R009"

 

My Current settings:

root is: "R:\\5410\\10285959\\4_CAD"

 

 

My current filter

*\\@Value(BussStopID)*\\Z\\Modell\\*03.dwg

icon

Best answer by takashi 10 February 2020, 15:16

View original

2 replies

takashi
Rank
Userlevel 2
Badge +17

Hi @clowncutie, I guess that the three parts "04", "0480" and "04800800" in the directory path can be extracted from the ID. i.e.

"04" = @Substring(@Value(ID),0,2)
"0480" = @Substring(@Value(ID),0,4)
"04800800" = @Substring(@Value(ID),0,8)

If so, you can set this expression to the Dataset in the PATH reader, in order to directly specify the folder containing target dwg files,

$(root_folder)\@Substring(@Value(ID),0,2)\@Substring(@Value(ID),0,4)\@Substring(@Value(ID),0,8)\@Value(ID)\Z\Modell

and just set *03.dwg to the Path Filter.

C
Badge +1

Hi @clowncutie, I guess that the three parts "04", "0480" and "04800800" in the directory path can be extracted from the ID. i.e.

"04" = @Substring(@Value(ID),0,2)
"0480" = @Substring(@Value(ID),0,4)
"04800800" = @Substring(@Value(ID),0,8)

If so, you can set this expression to the Dataset in the PATH reader, in order to directly specify the folder containing target dwg files,

$(root_folder)\@Substring(@Value(ID),0,2)\@Substring(@Value(ID),0,4)\@Substring(@Value(ID),0,8)\@Value(ID)\Z\Modell

and just set *03.dwg to the Path Filter.

I went from this:

 

0684Q00000ArMLhQAN.png

to this:

 

0684Q00000ArMnrQAF.png

Not the prettiest solution but the first FeatureReader only find the all directories containing "Modell\Z", aroung 100 of them, then uses the FeatureMerger to match with the substring in the path_name with the ID from Excel, then moves on as usual but without recursing into the subfolders, so now it only looks for .dwg in the 20 found folders which reduced the runtime with like 95%. 

Reply


Community Stats

28,650
Posts
109,267
Replies
36,523
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.