I'd use a "Directory and File Pathnames" reader to check for the presence of the ffs (set the path filter to the ffs filename), a NoFeaturesTester (FME Hub) and a Terminator connected to the NoInput port:
I'd use a "Directory and File Pathnames" reader to check for the presence of the ffs (set the path filter to the ffs filename), a NoFeaturesTester (FME Hub) and a Terminator connected to the NoInput port:
I don't
think this will work for what I need, or I'm not understanding it correctly.
In my
understanding the solution you provided would allow for a translation to abort
if an ffs file is not found. I do want the translation to abort if one of the
required ffs files does not exist, but not for the optional ones. If you pass into
an FFS Reader a file path that does not exist it aborts the translation, even
if the Ignore Failed Readers is set to Yes. So it seems like I need to have an
ffs file to pass to the workbench for the optional readers to allow for the
translation to continue.
I would use the NoFeaturesTester as David_r suggested, however I would only attach it to the required feature types.
Let's say you have 4 feature types, 2 of which are required, 2 of which are optional. You can have one reader for each feature type, or a single reader can produce multiple feature types, it's irrelevant.
In the example the required features share a common terminator, but you can create a custom message depending on which required feature is missing
If you like you can even design a workflow where if the option features don't exist you can read in a default or dummy file with the FeatureReader
I would use the NoFeaturesTester as David_r suggested, however I would only attach it to the required feature types.
Let's say you have 4 feature types, 2 of which are required, 2 of which are optional. You can have one reader for each feature type, or a single reader can produce multiple feature types, it's irrelevant.
In the example the required features share a common terminator, but you can create a custom message depending on which required feature is missing
If you like you can even design a workflow where if the option features don't exist you can read in a default or dummy file with the FeatureReader
I thought about this approach, but at least as far as I can tell, FME errors-out if you pass a non-existent file path to an FFS Reader. Even when Ignore Failed Features is set to Yes. So I still cannot have optional ffs inputs.
Am I missing something?
Hi @_trent, how about reading the required ffs files by the FeatureReader(s)? If you do so, you can check whether specified files exist or not beforehand with the FileExistenceChecker transformer from FME Hub, or a script.
I don't
think this will work for what I need, or I'm not understanding it correctly.
In my
understanding the solution you provided would allow for a translation to abort
if an ffs file is not found. I do want the translation to abort if one of the
required ffs files does not exist, but not for the optional ones. If you pass into
an FFS Reader a file path that does not exist it aborts the translation, even
if the Ignore Failed Readers is set to Yes. So it seems like I need to have an
ffs file to pass to the workbench for the optional readers to allow for the
translation to continue.
I still think that you could accomplish that using a combination of PATH readers, NoFeaturesTesters and FeatureReaders.
Although I think Takashi's solution is as good as you're likely to get, another way would be for the first workspace to write a dummy record to FFS if no features exist (again, the NoFeaturesTester will help).
Then the second workspace tests for the dummy record. If it appears, you know there is no real input.
The dummy record could be the same structure as the proper data, but it has (for example) an ID of -999, which is easy to test for and not something a true record would have.
I still think Takashi's solution is more elegant, but this might be a little easier.
It seems to be a known issue that the Ignore Failed Readers option doesn't always work as expected. If you want to contact our support team about the issue (safe.com/support) then please quote PR#33121 and hopefully we'll be able to increase the priority of the problem.
I thought about this approach, but at least as far as I can tell, FME errors-out if you pass a non-existent file path to an FFS Reader. Even when Ignore Failed Features is set to Yes. So I still cannot have optional ffs inputs.
Am I missing something?
Your're right, there is a bug with the FFS reader. I've only had occasion to do this with shapefiles.
Hi @_trent, how about reading the required ffs files by the FeatureReader(s)? If you do so, you can check whether specified files exist or not beforehand with the FileExistenceChecker transformer from FME Hub, or a script.
I didn't really know about the FeatureReader. I flipped your suggestion to use the FeatureReader to read the optional files, which I think will do the job. So I use a Directory and File Pathnames reader and pass the expected ffs file path to the Path Filter. If the file exists it outputs a single feature which then acts as the initiator for the FeatureReader.
See any potential problems with this?
I still think that you could accomplish that using a combination of PATH readers, NoFeaturesTesters and FeatureReaders.
Yes, I see that now, outlined in the reply to the input from @takashi.
fyi I'm told that the Failed Reader issue occurs when ALL readers in the workspace fail. In that case there is no data at all entering and so... what should happen? Basically if no features enter the workspace then the Failed Reader parameter is ignored.
This seems to be what you mentioned:
If you pass into an FFS Reader a file path that does not exist it aborts
the translation, even if the Ignore Failed Readers is set to Yes.
So... I'm wondering if you are using one reader to read all those files. If so, perhaps split it into two readers. One reader for the required data, one for the optional. That might help.
fyi I'm told that the Failed Reader issue occurs when ALL readers in the workspace fail. In that case there is no data at all entering and so... what should happen? Basically if no features enter the workspace then the Failed Reader parameter is ignored.
This seems to be what you mentioned:
If you pass into an FFS Reader a file path that does not exist it aborts
the translation, even if the Ignore Failed Readers is set to Yes.
So... I'm wondering if you are using one reader to read all those files. If so, perhaps split it into two readers. One reader for the required data, one for the optional. That might help.
Yes, I see this would work. I am using multiple readers. So if I set Ignore Failed Readers to Yes and put a NoFeaturesTester on the required readers output, chained to a Terminator for NoInput, that would give me the required and optional datasets I need.
I didn't really know about the FeatureReader. I flipped your suggestion to use the FeatureReader to read the optional files, which I think will do the job. So I use a Directory and File Pathnames reader and pass the expected ffs file path to the Path Filter. If the file exists it outputs a single feature which then acts as the initiator for the FeatureReader.
See any potential problems with this?
The combination of a PATH reader and FeatureReader might be a frequently used method but the PATH reader is not essential for the FeatureReader use. For example, if the required FFS file paths would be given by an external CSV table, you can read the paths with a CSV reader, check existence of each file with the FileExistenceChecker, and then send the path to the FeatureReader only if it exists. There can be various ways depending on how the required FFS paths will be given.
