Question

Need an idea for a new useful custom transformer or format!


Badge +7

So... until my new job starts in October, I have some time to kill! I'd love to build a useful custom transformer or format and share it on FME Hub, but the problem is: I'm out of ideas...

Do you have an idea for a transformer/format that would really be helpful to the community? So not something that only applies to your unique situation, but can be used by everyone? Post it below and who knows... it might appear on FME Hub the next coming weeks! Your name will be mentioned of course :)


40 replies

Userlevel 5
Badge +25

The DoWhatIThinker would be a nice one ;)

But if you're looking for serious suggestions... One thing that I often have to do as a cartographer is determine the relation between map scale, map size and area covered.

Say I have a 50 x 100 km area that I want to map at 1:100.000 scale, how much space on paper would this be? (actually, that's an easy one: 50 x 100 cm)

This also ties in with the resolution of raster- or elevation datasets. For example:

I have an elevation dataset with a resolution of 3 arc-second, customer wants shaded relief at scale 1:50.000 and it has to be printed at 300 dpi, does my dataset have the resolution to accomodate that?

My calculation: 3 arc second is 3/3600th of a degree, so 90 meters (at the equator... we really should incorporate projections into this as well). But 1:50.000 scale at 300 dpi means every pixel is about 5 meters (300 dpi means it's 11.8 dots per millimeter and 1 millimeter at 1:50.000 scale is 50 meters)

So the answer in this case is: no, the elevation data does not have the resolution to accomodate this (you can render the shaded relief at the required resolution, that's no problem, but you'll see artifacts)

Anyway... I have been trying to get something set up for this but haven't gotten very far yet. Ideally it'd be an FME Cloud thing that people could just access online: enter a few parameters, press go and get their results.

If this is something you think you could help out with, let me know (and if I'm not making any sense please let me know too)

Badge +7

The DoWhatIThinker would be a nice one ;)

But if you're looking for serious suggestions... One thing that I often have to do as a cartographer is determine the relation between map scale, map size and area covered.

Say I have a 50 x 100 km area that I want to map at 1:100.000 scale, how much space on paper would this be? (actually, that's an easy one: 50 x 100 cm)

This also ties in with the resolution of raster- or elevation datasets. For example:

I have an elevation dataset with a resolution of 3 arc-second, customer wants shaded relief at scale 1:50.000 and it has to be printed at 300 dpi, does my dataset have the resolution to accomodate that?

My calculation: 3 arc second is 3/3600th of a degree, so 90 meters (at the equator... we really should incorporate projections into this as well). But 1:50.000 scale at 300 dpi means every pixel is about 5 meters (300 dpi means it's 11.8 dots per millimeter and 1 millimeter at 1:50.000 scale is 50 meters)

So the answer in this case is: no, the elevation data does not have the resolution to accomodate this (you can render the shaded relief at the required resolution, that's no problem, but you'll see artifacts)

Anyway... I have been trying to get something set up for this but haven't gotten very far yet. Ideally it'd be an FME Cloud thing that people could just access online: enter a few parameters, press go and get their results.

If this is something you think you could help out with, let me know (and if I'm not making any sense please let me know too)

Thanks for the suggestion! That's a cool idea, although I'm not sure how many people would really need this... So @ everyone: upvote Hans' idea if you like it!

 

Userlevel 4
Badge +30

Hi @sander_s, that is a good Idea.

I'll think about a useful transformer for the comunity. :)

Thanks,

Userlevel 4
Badge +25

Interesting. If I can think of anything specific I'll let you know. In the meantime there's always this to contribute to: https://github.com/safesoftware/BestPractice

Userlevel 4
Badge +30

Interesting. If I can think of anything specific I'll let you know. In the meantime there's always this to contribute to: https://github.com/safesoftware/BestPractice

Great local to contribute in GitHub. Has 74 repositories :)

 

 

Badge +22

Based on ideas with popular votes

PDF reader - some sort of wrapper using www.foolabs.com/xpdf as was shown in the UC.

 

Schema modifier - https://knowledge.safe.com/idea/19730/introduce-new-transformer-to-modify-schema-easily.html

 

PolygonSplitter - https://knowledge.safe.com/idea/46814/transformer-to-split-a-polygon-by-line.html and https://knowledge.safe.com/idea/36673/polygonsplitter.html

3DSlicer - https://knowledge.safe.com/idea/18906/3d-slicer.html

 

ListAppender - https://knowledge.safe.com/idea/38231/listappender.html

 

Badge +22

I personally would love to see a transformer that calculates the maximal area inscribed rectangle for a given polygon.

 

 

See

https://d3plus.org/blog/behind-the-scenes/2014/07/08/largest-rect/

http://www.drdobbs.com/database/the-maximal-rectangle-problem/184410529 (axis-aligned)

https://geonet.esri.com/thread/182008-how-to-find-the-maximum-rectangle-contained-within-a-polygon

Userlevel 4
Badge +25
I think PDF reading is already planned for 2018, so you might want to hold off on that one.

 

 

Userlevel 5
Badge +25
Thanks for the suggestion! That's a cool idea, although I'm not sure how many people would really need this... So @ everyone: upvote Hans' idea if you like it!

 

Hey, I like being special ;)

 

 

Badge +2

Orthogonal Generalization:

Some papers on the subject are:

Haunert and Wolff. Optimal and Topologically Safe Simplification of Building Footprints https://www.researchgate.net/publication/208973342_Optimal_and_Topologically_Safe_Simplification_of_Building_Footprints Lupa, Koziol and Lesniak. An Attempt to Automate the Simplification of Building Objects in Multiresolution Databases https://www.researchgate.net/publication/280923583_An_Attempt_to_Automate_the_Simplification_of_Building_Objects_in_Multiresolution_Databases Bayer. Automated building simplification using recursive approach http://gisak.vsb.cz/GIS_Ostrava/GIS_Ova_2009/sbornik/Lists/Papers/033.pdf
Userlevel 1
Badge +18

A visual compare and diff between two workspaces. Which transformers, readers and writers are added, modified or deleted. See: https://knowledge.safe.com/idea/51439/visual-compare-two-workspaces.html

Badge +7

Orthogonal Generalization:

Some papers on the subject are:

Haunert and Wolff. Optimal and Topologically Safe Simplification of Building Footprints https://www.researchgate.net/publication/208973342_Optimal_and_Topologically_Safe_Simplification_of_Building_Footprints Lupa, Koziol and Lesniak. An Attempt to Automate the Simplification of Building Objects in Multiresolution Databases https://www.researchgate.net/publication/280923583_An_Attempt_to_Automate_the_Simplification_of_Building_Objects_in_Multiresolution_Databases Bayer. Automated building simplification using recursive approach http://gisak.vsb.cz/GIS_Ostrava/GIS_Ova_2009/sbornik/Lists/Papers/033.pdf
Some very interesting stuff and definitely useful, so thanks for the suggestions @MarkAtSafe! Although I'm not too sure if this can be achieved within a month and if it performs well enough. I believe that a native transformer made by Safe would be a better solution. But I'll look into it :)
Badge +7
Also a very interesting idea. Thanks @jdh! What could be the purpose of such a transformer?

 

Badge +7
The PolygonSplitter seems to be available already on FME Hub (sort of) as the PolygonCutter by @lars_de_vries. But the ListAppender might be a good one! Thanks again :)

 

Badge +7

A visual compare and diff between two workspaces. Which transformers, readers and writers are added, modified or deleted. See: https://knowledge.safe.com/idea/51439/visual-compare-two-workspaces.html

Thanks @stalknecht, I'll look into it!

 

Badge +22

Orthogonal Generalization:

Some papers on the subject are:

Haunert and Wolff. Optimal and Topologically Safe Simplification of Building Footprints https://www.researchgate.net/publication/208973342_Optimal_and_Topologically_Safe_Simplification_of_Building_Footprints Lupa, Koziol and Lesniak. An Attempt to Automate the Simplification of Building Objects in Multiresolution Databases https://www.researchgate.net/publication/280923583_An_Attempt_to_Automate_the_Simplification_of_Building_Objects_in_Multiresolution_Databases Bayer. Automated building simplification using recursive approach http://gisak.vsb.cz/GIS_Ostrava/GIS_Ova_2009/sbornik/Lists/Papers/033.pdf
Those papers somehow sound familiar ;)

 

 

Badge +22
Also a very interesting idea. Thanks @jdh! What could be the purpose of such a transformer?

 

The most common use is label placement, but it can also be used for texture orientation.

 

 

Badge +22
Thanks @stalknecht, I'll look into it!

 

You may want to talk with @larry who was working on one in april. https://knowledge.safe.com/questions/37858/fme-best-practice-validation-project-you-can-help.html?childToView=42374#comment-42374

 

Badge +7
@jdh: I think I'm going with the ListAppender for now! As usual, the GUI is the hardest nut to crack... In order to decide how I'm going to build it, could you tell me if the transformer should be able to deal with nested lists as well? And if that is the case, should it append attributes to all "sublists" or only a selected one?

 

For instance, when appending to parent{}.child{}.value, should it append new child.values for every parent (which seems a bit odd to me)? Or should you be able to select parent{n}.child{}.value, where n is a selected parent list node, so that new child.values are appended for a specific parent?

 

The latter would make more sense, but GUI-wise, it's a nightmare. I also had to create a hacky solution for my ListKeyValuePairExtractor (that also supports nested lists), so it would be cleaner to only support non-nested lists.

 

Badge +7

I've been thinking about doing a custom transformer to implement my PedanticSorter idea but haven't had time.

https://knowledge.safe.com/idea/41576/sorter-options-for-handling-null-missing-empty-str.html

Badge

Hi, @sander_s. If there are any of these ideas that you don't want to work on, let me know - I have had some free time lately as well, and am looking for FME brain-food. :-)

Badge +7

Hi, @sander_s. If there are any of these ideas that you don't want to work on, let me know - I have had some free time lately as well, and am looking for FME brain-food. :-)

Hi @courtney_m, I'd like to work on everything, but there's only so much I can do :)

 

I've started working on the ListAppender now and might do another one after that. Not sure which one. Just pick something and let me know!

 

Badge +7

I've been thinking about doing a custom transformer to implement my PedanticSorter idea but haven't had time.

https://knowledge.safe.com/idea/41576/sorter-options-for-handling-null-missing-empty-str.html

Hmm... not sure if this can be done efficiently in a custom transformer..? Should become an option on the regular Sorter transformer, I think. You have my vote!

 

 

Userlevel 4
Badge +25

I've been thinking about doing a custom transformer to implement my PedanticSorter idea but haven't had time.

https://knowledge.safe.com/idea/41576/sorter-options-for-handling-null-missing-empty-str.html

I like this one. I'll have a go at it this morning.

 

 

Userlevel 4
Badge +25
I like this one. I'll have a go at it this morning.

 

 

At which point I suddenly discover it's harder than I expected! Still, I'll give it a try.

 

 

Reply