• EN
Question

Best practices for large workbenches

  • 1 April 2024
  • 3 replies
  • 54 views
timotheebecker
Rank
Badge +3

Hi,
I’ve been working on a pretty large export workbench.

The subject is related to real estate. We’ve got a relational database and we drill into about 20 external layers to get some info and to filter our own data.

The workbench has grown a lot over time and now it’s really hard to maintain or yet expand. I’m afraid it might break anywhere.

Are there best practices on how to go about many processing jobs ?

Is it sometimes best to split a workbench into several separate workbenches ? If yes, how to structure the processing parts ?

Where to start the cleaning up chore ?

3 replies

oscard
Rank
Userlevel 2
Badge +19

Are there any parts of the workspace that could be reusable and therefore candidates for a Custom Transfomrer?

 

Even if there isn’t any, I would create Custom Transformers to group functionality to make the main project easier to read and to mantain.

timotheebecker
Rank
Badge +3

Hi Oscard,

Thank you.

More Custom Transformers sounds like a good idea.

However, there are many fields which get used in different parts of the workbench. Sometimes for an evolution I need to change the name of an attribute, for instance with BulkAttributeRenamer. Might it be more difficult to maintain for such cases if processes are isolated in CustomTransformers ?

Would you have any recommandations to manage large numbers of attributes ? Can one think of attributes a little like the concept of API ? At such a point, I should have these attributes and not more… I suppose that would be the AttributeKeeper.

Kind regards,

Timothée

milo89
Rank
Userlevel 2
Badge +11

My first comment would be to always back up and save what works to begin with before making any changes. Working in a new version helps should things go wrong.

The custom transformers is a great idea, i’m also a fan of bookmarks especially now that you can minimise a bookmark to only show the input and output transformers.

With regards to attributes, to help keep workbenches efficient, only reading required attributes and removing attributes which are not longer needed is always a good idea.

Generally, AttributeKeeper (keeping less than you’re removing) or AttributeRemover (removing less than you’re keeping). Otherwise AttributeManagers can also help you do more in a single transformer. There’s many ways to skin a cat on this one.

Reply


Community Stats

28,807
Posts
109,801
Replies
36,622
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.