Skip to main content
  • EN

I have a scenario where I need to dynamically read in some data (FeatureReader), do a bit of filtering and write it out dynamically with the FeatureWriter and then continue processing.

 

 

Since there's no reader to point to in the Dynamic Schema Definition Schema Sources, I'm using "Schema From Schema Feature". This creates the correct output file with the correct number of features.

 

However, the Summary port feature is including the schema feature in the _total_features_written attribute, so is one more than the number of features recorded in the log, and also outputs it through the Feature Output Port.

 

Is this the desired behaviour?

 

 

What happens if you skip the DataProcessing transformer, i.e. just read-write with nothing in between? Do you still get the same feature counts?

What happens if you skip the DataProcessing transformer, i.e. just read-write with nothing in between? Do you still get the same feature counts?

Yes.

 

 

It's the presence of the schema feature. If I use the FeatureWriter dynamically, with the schema source set to a resource and don't attach the schema feature then the _total_features_written count matches the value from the log file.

 

 

Now I can remove the schema feature from the output port by testing for fme_schema_handling = schema_only, but the summary attribute is more concerning.

 

 

If all my FeatureWriters had a schema feature I could just subtract one from the _total_features_written, but I have a mix of static and dynamic writers and keeping tract of which ones should be adjusted will be make future development interesting.

 

 

Yes.

 

 

It's the presence of the schema feature. If I use the FeatureWriter dynamically, with the schema source set to a resource and don't attach the schema feature then the _total_features_written count matches the value from the log file.

 

 

Now I can remove the schema feature from the output port by testing for fme_schema_handling = schema_only, but the summary attribute is more concerning.

 

 

If all my FeatureWriters had a schema feature I could just subtract one from the _total_features_written, but I have a mix of static and dynamic writers and keeping tract of which ones should be adjusted will be make future development interesting.

 

 

Actually, I now remember having to write some Python snippet to "fix" that particular value, probably in FME 2017, before doing some reporting based on it.

 

I would class it as a small bug, consider sending it to Safe.

In case anybody else stumbles upon this issue, here's some code for a PythonCaller that can be used just after the FeatureWriter to adjust the count values to not include the schema feature:

def FeatureProcessor(feature):
    count_list = feature.getAttribute('_feature_type{}.count')
    adjusted_counts = eint(x)-1 for x in count_list]
    feature.setAttribute('_feature_type{}.count',  str(x) for x in adjusted_counts])

In short, all the values in the list attribute _feature_type{}.count will be subtracted by 1.

In case anybody else stumbles upon this issue, here's some code for a PythonCaller that can be used just after the FeatureWriter to adjust the count values to not include the schema feature:

def FeatureProcessor(feature):
    count_list = feature.getAttribute('_feature_type{}.count')
    adjusted_counts = eint(x)-1 for x in count_list]
    feature.setAttribute('_feature_type{}.count',  str(x) for x in adjusted_counts])

In short, all the values in the list attribute _feature_type{}.count will be subtracted by 1.

If I write 10 features use Dynamic Schema mode, then I get _feature_type{0}.count=10, _feature_type{1}.count=1, both of these results are correct, so I don't think will be It is correct to subtract 1 from these two values.

 

 

What confuses the user is the _total_features_written attribute, which has a value of 11 in this case, but the value itself is correct because 10 features and 1 schema feature are actually entered. When schema source is applied, _total_features_written has a value of 10 because no schema feature is actually entered, so I think FME behaves correctly.

 

 

If you want to get the correct number from summary ports, you can use ListSearcher to search for the exported Shapefile'name in _feature_type{}.name, and then use ListIndexer to get the corresponding _feature_type{}.count.

 

 

You can use the following method, first add an flag attribute to the input feature, and then add it to the Additional Summary Attributes of the FeatureWriter. You can then use the ListElementFilter to filter out the statistics of the schema so that you can get the count of the feature outside of the schema feature.

Reply


Community Stats

30,159
Posts
114,480
Replies
37,807
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.