Hello, the FeatureHolder in your workspace holds all 5 features coming from your reader, and then releases them one by one to your Tester. It seems to be working like expected.

If you want to hold your features until all paths are checked by your Tester, you have to put your FeatureHolder somewhere behind your Tester.

Why do you want to hold your features?

Hi @joepk 

The tester passes 1 feature only and not the others.

Simply put, the revised version should read the extracted archive files performed by the user:

However I’m applying breakpoints as usual to intercept user error.

1. Beakpoint 1
   Ensure the file names contain number 3 and if that is the case then carry on else notify user and terminate 
    
2. Breakpoint 2
   Files are correct but check if file extension has value shp. If yes read the Shape file else notify user and terminate

There’s a reason for the above as the source provider is changing the file names. One of the archive does not even contain a .shp file. Might have been a corruption during the download process their end.

The reason for using the FeatureHolder was because I thought the workflow was not processing all the features when reaching the tester. Only 1 feature passes through whereby I’m expecting more files. (Extracted ESRI Zip file)

Hope that helps

So here as you have set up the Terminator it will fail for every feature / file which does not have a .shp extension. For each file you have one feature and only one of them will have the .shp extension.

Another thing you can try and use is a NoFeatureTester and / or a FeatureFlowValve.

The NoFeatureTester will output a feature from the NoInput port - This you could run into the Terminator (in the case where no shapefile was found.

 

The output port will just output the shp when there is a shp path to output and will only trigger the feature reader then

5 features were on their way to the Tester, but since the first feature already reached the Terminator, the translation was terminated and they never reached the Tester. 

I think you want to collect all the 5 paths from all the 5 files making up your Esri Shapefile in 1 feature, and then test on that.

That could be tricky as today it might contains 5 files but in the future it might contain just 3 or more. But I get your point. I think my approach is wrong.

Hi @joepk 

The tester passes 1 feature only and not the others.

Simply put, the revised version should read the extracted archive files performed by the user:

However I’m applying breakpoints as usual to intercept user error.

1. Beakpoint 1
   Ensure the file names contain number 3 and if that is the case then carry on else notify user and terminate 
    
2. Breakpoint 2
   Files are correct but check if file extension has value shp. If yes read the Shape file else notify user and terminate

There’s a reason for the above as the source provider is changing the file names. One of the archive does not even contain a .shp file. Might have been a corruption during the download process their end.

 

The reason for using the FeatureHolder was because I thought the workflow was not processing all the features when reaching the tester. Only 1 feature passes through whereby I’m expecting more files. (Extracted ESRI Zip file)

Hope that helps

So here as you have set up the Terminator it will fail for every feature / file which does not have a .shp extension. For each file you have one feature and only one of them will have the .shp extension.

Another thing you can try and use is a NoFeatureTester and / or a FeatureFlowValve.

The NoFeatureTester will output a feature from the NoInput port - This you could run into the Terminator (in the case where no shapefile was found.

 

The output port will just output the shp when there is a shp path to output and will only trigger the feature reader then

The NoFeatureTester is not a bad idea. The Terminator is used because the workspace has to fail otherwise when published to FME Server the workspace will run as normal with no indication of a failure. It becomes a false positive if no breakpoints are applied if you see what I mean.

I’ll try that and also review my existing workflow. I’ll keep you posted.

You could try to use a ListBuilder to collect all those (5 or 3) path_windows attributes in a single list attribute on a single feature, and then use a ListConcatenator to collect all those (5 or 3) path_windows attributes in a single attribute.

Then use a StringSearcher to check if the .shp extension is in the concatenated attribute. 

Yeah 100% You are trying to ensure that there is a .shp file - it doesn't really matter what the others are (I assume?) but I guess you want to probably also make sure that the .dbf file is also uploaded if the attributes are important.

You don’t need to connect the terminator directly to the Tester (for the shp) because you don’t care that there are other files - This will indeed create the false positive you mentioned. The NoFeatureTester > Terminator performs the same logic (will fail the workspace if no shp file is found)

If I understand correctly, what you want to do is:

• If one of the features gets through the Passed port of a Tester
  • Then all features should continue.
  • Else all features should be stopped.

How I solve this:

• Test using a Tester.
• Sampler to let only one through.
• AttributeKeeper to remove all attributes.
• FeatureMerger merging 1 =1.

Thank you all for your inputs!

@joepk Your idea about creating a list was useful and inspirational. However I’ve used an Aggregator instead to simply things. Ultimately that’s what led me to fix my problem.

Final workspace below which will connect to the remaining part.

Extracted files by user (Source Files)

Using the PATH Reader, I’ve added path_filename as initially I only exposed path_windows & path_extension.

path_filename is used to check if the files contain number 3 (Aggregator / Tester)

path_extension is used to check if “shp” exists (Aggregator / Tester)

path_windows is used as descriptive note for the Emailer

I intentionally sabotaged the file names and extensions and it worked as intended.

Finally the FeatureReader makes use of a UserParameter, which was initially created for the user to upload the ESRI Zip file. I’ve modified it so that the Reader pickups up the SHP file directly from the source location using a wild card and extension filtered to shp.

By setting the “Output Ports” to “Single Output Port” it ensures that no matter how the files are named, the features will be read.

@virtualcitymatt Yes you were right in assuming that I was not interested in the other files. Attributes will be created during the transformation stage. (i.e. Owner, Counter as Primary Key, Date_Created, etc...). The NoFeatureTester could also have been a good alternative.

This whole issue was caused by the Source Provider changing the name of the archive as well as the filenames. My FeatureReader was set initially to specifically read these features. So I wanted this workflow to be a bit more dynamic. I had to amend my workflow several times to accommodate for their changes previously.

@nielsgerrits Your workspace is great. Interesting approach. I will keep it for future use.

Thanks all!