So... until my new job starts in October, I have some time to kill! I'd love to build a useful custom transformer or format and share it on FME Hub, but the problem is: I'm out of ideas...Do you have an idea for a transformer/format that would really be helpful to the community? So not something that only applies to your unique situation, but can be used by everyone? Post it below and who knows... it might appear on FME Hub the next coming weeks! Your name will be mentioned of course :)

Need an idea for a new useful custom transformer or format!

Userlevel 4

+25

I've been thinking about doing a custom transformer to implement my PedanticSorter idea but haven't had time.

https://knowledge.safe.com/idea/41576/sorter-options-for-handling-null-missing-empty-str.html

Well @tim_wood and @sander_s - I'm not sure if it's the most efficient, but it seems to work OK. Have a go and let me know what you think. You can make the non-values appear at the top/bottom of the pile, and you can choose the order of Null/Missing/Empty too.

https://hub.safe.com/transformers/controllednullsorter

+22

jdh
Contributor
1961 replies
6 years ago
24 August 2017

@jdh: I think I'm going with the ListAppender for now! As usual, the GUI is the hardest nut to crack... In order to decide how I'm going to build it, could you tell me if the transformer should be able to deal with nested lists as well? And if that is the case, should it append attributes to all "sublists" or only a selected one?

For instance, when appending to parent{}.child{}.value, should it append new child.values for every parent (which seems a bit odd to me)? Or should you be able to select parent{n}.child{}.value, where n is a selected parent list node, so that new child.values are appended for a specific parent?

The latter would make more sense, but GUI-wise, it's a nightmare. I also had to create a hacky solution for my ListKeyValuePairExtractor (that also supports nested lists), so it would be cleaner to only support non-nested lists.

The version I wrote (which unfortunately cannot be shared) handles complex lists(_list{}.a) but not nested lists (_list{}.x{}), for pretty much the GUI reason.

+22

jdh
Contributor
1961 replies
6 years ago
24 August 2017

Hi, @sander_s. If there are any of these ideas that you don't want to work on, let me know - I have had some free time lately as well, and am looking for FME brain-food. :-)

Go with the orthogonal generalization :). It's probably the most complex, but also the most useful.

+7

Well @tim_wood and @sander_s - I'm not sure if it's the most efficient, but it seems to work OK. Have a go and let me know what you think. You can make the non-values appear at the top/bottom of the pile, and you can choose the order of Null/Missing/Empty too.

https://hub.safe.com/transformers/controllednullsorter

Nice work! Next version should support multiple attribute sorting though ;)

By the way, I noticed this when I first opened the parameters dialog:

Probably needs a little clean up.

Oh and nothing important, but in the bookmark descriptions it says "...Appendage" but in the private parameters it says "...Suffix".

But other than that, good stuff!

Userlevel 4

+25

Nice work! Next version should support multiple attribute sorting though ;)

By the way, I noticed this when I first opened the parameters dialog:

Probably needs a little clean up.

Oh and nothing important, but in the bookmark descriptions it says "...Appendage" but in the private parameters it says "...Suffix".

But other than that, good stuff!

Yes, multiple attribute sorting presents a bit of a challenge. Actually a lot of a challenge. The Sorter only accepts one attribute at a time, so I'd need to know in advance how many attributes to sort by. Or switch to python, which I'm very loathe to do.

The "unusedCfg" part of the label is a mystery. I didn't add that. Filed it with the developers. And ParkName is just the default I used in my test data.

Ha! I used "appendage" at first rather than suffix, until I realized that gave me not just NullAppendage, but MissingAppendage! Missed changing that part though.

+7

The version I wrote (which unfortunately cannot be shared) handles complex lists(_list{}.a) but not nested lists (_list{}.x{}), for pretty much the GUI reason.

Alright, I'll have it support nested lists then, since I already wrote the code for that in the ListKeyValuePairExtractor.

Meanwhile, I posted an idea for a new/enhanced GUI type, so please vote! ;)

+7

Yes, multiple attribute sorting presents a bit of a challenge. Actually a lot of a challenge. The Sorter only accepts one attribute at a time, so I'd need to know in advance how many attributes to sort by. Or switch to python, which I'm very loathe to do.

The "unusedCfg" part of the label is a mystery. I didn't add that. Filed it with the developers. And ParkName is just the default I used in my test data.

Ha! I used "appendage" at first rather than suffix, until I realized that gave me not just NullAppendage, but MissingAppendage! Missed changing that part though.

A transformer that sorts missing appendages, wow...! :D

+2

pratap
Contributor
594 replies
6 years ago
25 August 2017

Hi, @sander_s. If there are any of these ideas that you don't want to work on, let me know - I have had some free time lately as well, and am looking for FME brain-food. :-)

Hi @courtney_m, I would suggest to kindly post your requirement. Out of the suggestions posted orthogonal generation seems to be interesting for me to try, so I'm trying it. I don't know how much time it will take :). If your requirement is taking less time then normally I can change the focus and complete.

Hope @sander_s don't have objection since he is the owner of the post :)

+7

Hi @courtney_m, I would suggest to kindly post your requirement. Out of the suggestions posted orthogonal generation seems to be interesting for me to try, so I'm trying it. I don't know how much time it will take :). If your requirement is taking less time then normally I can change the focus and complete.

Hope @sander_s don't have objection since he is the owner of the post :)

Sure, no problem @pratap! I might also give it a go, but not sure how long it's going to take me either. We could turn it into a contest! ;)

+7

tim_wood
Contributor
311 replies
6 years ago
25 August 2017

Well @tim_wood and @sander_s - I'm not sure if it's the most efficient, but it seems to work OK. Have a go and let me know what you think. You can make the non-values appear at the top/bottom of the pile, and you can choose the order of Null/Missing/Empty too.

https://hub.safe.com/transformers/controllednullsorter

Thanks :-) I'll try and dig out the data I was using at the time this came up and give it a go. I prefer my name for the transformer though ;-)

+22

jdh
Contributor
1961 replies
6 years ago
25 August 2017

Hi @courtney_m, I would suggest to kindly post your requirement. Out of the suggestions posted orthogonal generation seems to be interesting for me to try, so I'm trying it. I don't know how much time it will take :). If your requirement is taking less time then normally I can change the focus and complete.

Hope @sander_s don't have objection since he is the owner of the post :)

I don't have time, which is why I sent the papers to Marc originally, but if multiple people are going to give it a go, why don't you each pick a different paper to try and implement.

+11

A visual compare and diff between two workspaces. Which transformers, readers and writers are added, modified or deleted. See: https://knowledge.safe.com/idea/51439/visual-compare-two-workspaces.html

Ah, yes! I was working on something myself in off hours.. I used the FMW Reader (in FeatureReader) and way too many FeatureMergers – this was a very messy approach so I simplified it to just checking transformers.

It is not fully tested but might be useful,.. so I have attached!

@sander_s

workspacechangedetector.fmw

+22

jdh
Contributor
1961 replies
6 years ago
28 August 2017

Ah, yes! I was working on something myself in off hours.. I used the FMW Reader (in FeatureReader) and way too many FeatureMergers – this was a very messy approach so I simplified it to just checking transformers.

It is not fully tested but might be useful,.. so I have attached!

@sander_s

workspacechangedetector.fmw

I feel your pain, when I was toying with this back in february, I ended up with 6 feature mergers, and that was just comparing readers and writers and their featuretypes.