Solved

What are the FME best practices for modularization?

F

Hello people,

 

I am new to FME (about 3 months) and this is my first post on this community. So, please be patient, if this question sounds ridiculous to you ;-)

The background of my question is as follows:

 

I have a big major workspace and want to split this into a few smaller, less complex, workspaces. In addition, I would like to re-use some of these new small workspaces in another context. Therefore, my idea was to build a few FME-workspaces as modules and use these modules to build bigger major workspaces.

I understand that I can use custom-transformers and/or workspace-runner for this, but both do have some disadvantages on its own.

Workspace-runner on the one hand, can just run another workspace (as the name says) and it can transfer data from the parent-workspace to the child-workspace via published parameter. However, the workspace-runner cannot give you an output from the child-workspace back to the parent-workspace (if I am right here?). So the workspace-runner is limited in its use.

Custom-Transformer on the other hand, are basically small workspaces on its own, which can get input from its (parent-)workspace and can give an output back to it, too. Therefore, the disadvantage of the workspace-runner is not present here. However, since custom-transformer are not real independent workspaces, you can't use this transformer as a module in other workspaces (if I am right here?).

The only way, I could find out, was to copy this custom-transformers to the other workspaces. Problem with this solution: If I do modify the custom-transformer (i.e. I do fix an issue in its logic) in one workspace, it will not be updated in other workspaces.

So, I try to figure out, when to use which and if there are some best practices for this.

Best greetings and thanks for help

 

Florian

 

 

 

icon

Best answer by redgeographics 1 July 2020, 19:34

View original

5 replies

redgeographics
Rank
Userlevel 5
Badge +25

I think custom transformers are the way to go:

If you create them in FME Workbench and then save them as a .fmx file you can re-use them easily in multiple workspaces, there's also an option to pick whether they should be embedded or linked, linked will have the benefit that one change will be propagated easily to all workspaces using that custom transformer and they will have that independence you are looking for.

I highly recommend you check out the chapter on Custom Transformers in the FME Desktop Advanced training course, it covers the creation of custom transformers, linking/embedding and versioning.

F

I think custom transformers are the way to go:

If you create them in FME Workbench and then save them as a .fmx file you can re-use them easily in multiple workspaces, there's also an option to pick whether they should be embedded or linked, linked will have the benefit that one change will be propagated easily to all workspaces using that custom transformer and they will have that independence you are looking for.

I highly recommend you check out the chapter on Custom Transformers in the FME Desktop Advanced training course, it covers the creation of custom transformers, linking/embedding and versioning.

Oh! Cool! Thanks! I wasn't aware of the use of linked transformers. I will check out the tutorial and will post a feedback, if this is solving all my problems... but it sounds like it :-)

F

I think custom transformers are the way to go:

If you create them in FME Workbench and then save them as a .fmx file you can re-use them easily in multiple workspaces, there's also an option to pick whether they should be embedded or linked, linked will have the benefit that one change will be propagated easily to all workspaces using that custom transformer and they will have that independence you are looking for.

I highly recommend you check out the chapter on Custom Transformers in the FME Desktop Advanced training course, it covers the creation of custom transformers, linking/embedding and versioning.

Hey. Sorry for the delayed response!

 

 

I tested the solution with linked custom transformers, but I have trouble with two issues:

 

(1) I was not able to use Readers and Writers inside the custom transformer, which would be a nice feature, if you want to have a true modularization of a workspace.

 

(2) It is sadly not possible to run custom transformers on its own (like a full featured FME workspace) and therefore an “feature caching run” inside a custom transformer is not possible. But this would be a great easement for debugging.

 

 

Maybe you (or someone else) can me help out with this issues, too :-)

Best greetings

Florian

jdh
Rank
Badge +22

Hey. Sorry for the delayed response!

 

 

I tested the solution with linked custom transformers, but I have trouble with two issues:

 

(1) I was not able to use Readers and Writers inside the custom transformer, which would be a nice feature, if you want to have a true modularization of a workspace.

 

(2) It is sadly not possible to run custom transformers on its own (like a full featured FME workspace) and therefore an “feature caching run” inside a custom transformer is not possible. But this would be a great easement for debugging.

 

 

Maybe you (or someone else) can me help out with this issues, too :-)

Best greetings

Florian

You can use FeatureReaders/FeatureWriters inside of custom transformers.

 

If you embed the custom transformer (assuming there is not a loop) you should be able to debug it at lot easier.
F

Hey. Sorry for the delayed response!

 

 

I tested the solution with linked custom transformers, but I have trouble with two issues:

 

(1) I was not able to use Readers and Writers inside the custom transformer, which would be a nice feature, if you want to have a true modularization of a workspace.

 

(2) It is sadly not possible to run custom transformers on its own (like a full featured FME workspace) and therefore an “feature caching run” inside a custom transformer is not possible. But this would be a great easement for debugging.

 

 

Maybe you (or someone else) can me help out with this issues, too :-)

Best greetings

Florian

Hi @jdh​ ,

thanks for your reply!

This hints were really usefull! I would love to have the partial run and feature caching functionality also available in linked custom transformers. But i can work with this!

Best greetings

Florian

 

Reply


Community Stats

28,731
Posts
109,530
Replies
36,565
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.