Question

Expose TestFilter output port names or write them to attribute?

  • 24 October 2016
  • 6 replies
  • 29 views
tim_wood
Rank
Badge +7

I have a dataset containing highway defects that I want to fanout to separate layers for potholes, blocked drains, etc. However there is no attribute which can immediately be used for fanout. The information required for fanout is part of the value of one of the attributes, but the whole value is not always the same e.g. "aaaa Pothole xyz", "bbbb Pothole abc", etc.

I have used a TestFilter to create output ports for different defect types e.g. If "@value(fieldname) CONTAINS Pothole" then output through "Pothole" port,

If "@value(fieldname) CONTAINS Blocked Drain" then output through "Blocked Drain" port. I then realised what I wanted to do was pick up the output port name somehow and use that to fanout out the data (e.g. in the writer set the table name to something like "highway_defects_@value(output_port_name)".

I could have a separate writer for each defect type, but the solution I'm going with at the moment is creating a DefectType attribute before the TestFilter, then having an AttributeCreator attached to each output port from the TestFilter and setting the DefectType value accordingly. Then everything goes to one writer which sets the table name to highway_defects_@value(DefectType).

If anyone has any better ideas, I'd be interested to hear them. Maybe TestFilter is the wrong starting point?

6 replies

david_r
Rank
Userlevel 4

You could use an AttributeCreator with a conditional value, then use that value for the fanout, e.g.

itay
Rank
Badge +16

would be my also my take on it

ebygomm
Rank
Userlevel 1
Badge +21

You could also use a StringSearcher with a regular expression to set the fanout value. I find this easier to manage than a lot of conditional statements

tim_wood
Rank
Badge +7

You could also use a StringSearcher with a regular expression to set the fanout value. I find this easier to manage than a lot of conditional statements

Thanks. I might give that a go. The Matched and NotMatched ports would filter out only the records that had the defect types I wanted - there may be records in the data that don't fall into any of the defect categories.

 

 

david_r
Rank
Userlevel 4
Thanks. I might give that a go. The Matched and NotMatched ports would filter out only the records that had the defect types I wanted - there may be records in the data that don't fall into any of the defect categories.

 

 

The AttributeCreator above will give you the same possibility, just use a Tester with "_fanout" = "Unknown", which will be identical to the NotMatched of the StringSearcher.
M

Putting in the ability to setup attributes based upon test values would be a nice straightforward way to do this and it has my vote! I can see that doing it with a transformer like the AttributeManager would work also but it isn't immediately obvious on how to do it and when you need a way to do it right now, then this would provide it. @tim_wood

Reply


Community Stats

28,688
Posts
109,427
Replies
36,543
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.