Skip to main content
Question

Generate Empty FFS

_
Forum|alt.badge.img

Is it possible to generate an empty ffs file?

I have a data conversion process where some optional datasets used in a workbench might not exist. I do not want to use the Ignore Failed Readers option, as I want the translation to error-out if the required datasets are not present. A predecessor workbench will normally generate these optional ffs files if the input exists, but if no input exists, no ffs is created. I cannot find an option for the workbench to write the ffs file even when it has no input. I could also potentially copy an empty ffs file into the expected reader location, but I cannot figure out how to generate an empty ffs file.

Or could this be handled a better way?

Using FME Desktop 2016.1.1.0

14 replies

david_r
Celebrity
  • david_rCelebrity
  • September 8, 2016

I'd use a "Directory and File Pathnames" reader to check for the presence of the ffs (set the path filter to the ffs filename), a NoFeaturesTester (FME Hub) and a Terminator connected to the NoInput port:

_
Forum|alt.badge.img
  • _trent
  • Author
  • September 8, 2016
david_r wrote:

I'd use a "Directory and File Pathnames" reader to check for the presence of the ffs (set the path filter to the ffs filename), a NoFeaturesTester (FME Hub) and a Terminator connected to the NoInput port:

I don't think this will work for what I need, or I'm not understanding it correctly.

 

In my understanding the solution you provided would allow for a translation to abort if an ffs file is not found. I do want the translation to abort if one of the required ffs files does not exist, but not for the optional ones. If you pass into an FFS Reader a file path that does not exist it aborts the translation, even if the Ignore Failed Readers is set to Yes. So it seems like I need to have an ffs file to pass to the workbench for the optional readers to allow for the translation to continue.

 

 

 

jdh
Contributor
Forum|alt.badge.img+28
  • jdhContributor
  • Contributor
  • September 8, 2016

I would use the NoFeaturesTester as David_r suggested, however I would only attach it to the required feature types.

 

 

Let's say you have 4 feature types, 2 of which are required, 2 of which are optional. You can have one reader for each feature type, or a single reader can produce multiple feature types, it's irrelevant.

 

 

In the example the required features share a common terminator, but you can create a custom message depending on which required feature is missing

If you like you can even design a workflow where if the option features don't exist you can read in a default or dummy file with the FeatureReader

_
Forum|alt.badge.img
  • _trent
  • Author
  • September 8, 2016
jdh wrote:

I would use the NoFeaturesTester as David_r suggested, however I would only attach it to the required feature types.

 

 

Let's say you have 4 feature types, 2 of which are required, 2 of which are optional. You can have one reader for each feature type, or a single reader can produce multiple feature types, it's irrelevant.

 

 

In the example the required features share a common terminator, but you can create a custom message depending on which required feature is missing

If you like you can even design a workflow where if the option features don't exist you can read in a default or dummy file with the FeatureReader

I thought about this approach, but at least as far as I can tell, FME errors-out if you pass a non-existent file path to an FFS Reader. Even when Ignore Failed Features is set to Yes. So I still cannot have optional ffs inputs.

 

Am I missing something?

 

takashi
Influencer
  • takashiInfluencer
  • September 9, 2016

Hi @_trent, how about reading the required ffs files by the FeatureReader(s)? If you do so, you can check whether specified files exist or not beforehand with the FileExistenceChecker transformer from FME Hub, or a script.

david_r
Celebrity
  • david_rCelebrity
  • September 9, 2016
_trent wrote:
I don't think this will work for what I need, or I'm not understanding it correctly.

 

In my understanding the solution you provided would allow for a translation to abort if an ffs file is not found. I do want the translation to abort if one of the required ffs files does not exist, but not for the optional ones. If you pass into an FFS Reader a file path that does not exist it aborts the translation, even if the Ignore Failed Readers is set to Yes. So it seems like I need to have an ffs file to pass to the workbench for the optional readers to allow for the translation to continue.

 

 

 

I still think that you could accomplish that using a combination of PATH readers, NoFeaturesTesters and FeatureReaders.

 

 

mark2atsafe
Safer
Forum|alt.badge.img+44
  • mark2atsafeSafer
  • Safer
  • September 9, 2016

Although I think Takashi's solution is as good as you're likely to get, another way would be for the first workspace to write a dummy record to FFS if no features exist (again, the NoFeaturesTester will help).

Then the second workspace tests for the dummy record. If it appears, you know there is no real input.

The dummy record could be the same structure as the proper data, but it has (for example) an ID of -999, which is easy to test for and not something a true record would have.

I still think Takashi's solution is more elegant, but this might be a little easier.

mark2atsafe
Safer
Forum|alt.badge.img+44
  • mark2atsafeSafer
  • Safer
  • September 9, 2016
It seems to be a known issue that the Ignore Failed Readers option doesn't always work as expected. If you want to contact our support team about the issue (safe.com/support) then please quote PR#33121 and hopefully we'll be able to increase the priority of the problem.

 

 

jdh
Contributor
Forum|alt.badge.img+28
  • jdhContributor
  • Contributor
  • September 9, 2016
_trent wrote:
I thought about this approach, but at least as far as I can tell, FME errors-out if you pass a non-existent file path to an FFS Reader. Even when Ignore Failed Features is set to Yes. So I still cannot have optional ffs inputs.

 

Am I missing something?

 

Your're right, there is a bug with the FFS reader. I've only had occasion to do this with shapefiles.
_
Forum|alt.badge.img
  • _trent
  • Author
  • September 9, 2016
takashi wrote:

Hi @_trent, how about reading the required ffs files by the FeatureReader(s)? If you do so, you can check whether specified files exist or not beforehand with the FileExistenceChecker transformer from FME Hub, or a script.

I didn't really know about the FeatureReader. I flipped your suggestion to use the FeatureReader to read the optional files, which I think will do the job. So I use a Directory and File Pathnames reader and pass the expected ffs file path to the Path Filter. If the file exists it outputs a single feature which then acts as the initiator for the FeatureReader.

 

 

See any potential problems with this?

 

_
Forum|alt.badge.img
  • _trent
  • Author
  • September 9, 2016
david_r wrote:
I still think that you could accomplish that using a combination of PATH readers, NoFeaturesTesters and FeatureReaders.

 

 

Yes, I see that now, outlined in the reply to the input from @takashi.

 

mark2atsafe
Safer
Forum|alt.badge.img+44
  • mark2atsafeSafer
  • Safer
  • September 9, 2016

fyi I'm told that the Failed Reader issue occurs when ALL readers in the workspace fail. In that case there is no data at all entering and so... what should happen? Basically if no features enter the workspace then the Failed Reader parameter is ignored.

This seems to be what you mentioned:

If you pass into an FFS Reader a file path that does not exist it aborts the translation, even if the Ignore Failed Readers is set to Yes.

So... I'm wondering if you are using one reader to read all those files. If so, perhaps split it into two readers. One reader for the required data, one for the optional. That might help.

_
Forum|alt.badge.img
  • _trent
  • Author
  • September 9, 2016
mark2atsafe wrote:

fyi I'm told that the Failed Reader issue occurs when ALL readers in the workspace fail. In that case there is no data at all entering and so... what should happen? Basically if no features enter the workspace then the Failed Reader parameter is ignored.

This seems to be what you mentioned:

If you pass into an FFS Reader a file path that does not exist it aborts the translation, even if the Ignore Failed Readers is set to Yes.

So... I'm wondering if you are using one reader to read all those files. If so, perhaps split it into two readers. One reader for the required data, one for the optional. That might help.

Yes, I see this would work. I am using multiple readers. So if I set Ignore Failed Readers to Yes and put a NoFeaturesTester on the required readers output, chained to a Terminator for NoInput, that would give me the required and optional datasets I need.

 

 

takashi
Influencer
  • takashiInfluencer
  • September 9, 2016
_trent wrote:
I didn't really know about the FeatureReader. I flipped your suggestion to use the FeatureReader to read the optional files, which I think will do the job. So I use a Directory and File Pathnames reader and pass the expected ffs file path to the Path Filter. If the file exists it outputs a single feature which then acts as the initiator for the FeatureReader.

 

 

See any potential problems with this?

 

The combination of a PATH reader and FeatureReader might be a frequently used method but the PATH reader is not essential for the FeatureReader use. For example, if the required FFS file paths would be given by an external CSV table, you can read the paths with a CSV reader, check existence of each file with the FileExistenceChecker, and then send the path to the FeatureReader only if it exists. There can be various ways depending on how the required FFS paths will be given.

 

Reply


Helpful Members This Week

Community Stats

31,378
Posts
119,006
Replies
39,007
Members

Latest FME

Terms & ConditionsCookie settings

Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.