Skip to main content
  • EN

I recently inherited a workspace (sorry for all my other questions related and thanks for your time and patience 🙂)

The problem with the workspace is that it has over 1400 transformers, at least according to this command of my provenience which might be incorrect:

cat my_workspace.fmw | grep \<TRANSFORMER |wc -l

I can hardly orientate myself in all the logic, since the canvas won't fit to my screen even at the lowest zoom. I only managed to oversight it printing to A0 pdf:

huge-workspaceAnyway, I don't know if this is common in real life FME projects. Are there any best practices regarding the complexity and extent of the workspace? Are there any possible performance drawbacks with bigger workspaces?

My intuition would be to divide it and execute the resulting flows separately with WorkspaceRunner, but I cannot find a clear recommendation on it in the docs.

Ooof... I've never had a workspace this big myself (but have made workspaces with hundreds of transformers), I think 1400 is not a record but definitely a big one!

 

You'll get a count of the number of transformers in the Navigator as well by the way, and I think that includes transformers inside custom transformers as well, I see lots of those in your screenshot.

 

You'll notice FME Workbench getting sluggish when handling such a large workspace, other than that I don't think that there's a downside, performance-wise, for the actual processing. Managing the workspace will be difficult though, it does look like it's set up with best practices in mind (though I can't really tell if there's any annotations being used), but it's just really really complex and it'll take time to get your head around.

OK, thanks redgeographics!

My grep command is incorrect. In the Navigator view I see there's "only" 826 transformers.

I still have to get to know the tool. Thanks for pinpointing the Navigator, I had it closed.

Cheers

OK, thanks redgeographics!

My grep command is incorrect. In the Navigator view I see there's "only" 826 transformers.

I still have to get to know the tool. Thanks for pinpointing the Navigator, I had it closed.

Cheers

826 transformers is still a lot! Looking at your other questions in relation to this one it looks like you have a pretty hefty task ahead of you. Good luck!!

I have maintained and built many large workbenches that were originally built in FME2010/2011 timeframe. Pre-AttributeManager, in some instances it was required to string together many attribute transformers such as StringConcatenator, AttributeCreator, StringReplacer, AttributeKeeper, AttributeRemover, etc. With the advent of AttributeManager, my file sizes have drastically been reduced since this transformers can handle so many tasks in itself. This may be applicable in your case.

 

My biggest problem with large files seems to be physical .fmw file size compared to the actual number of transformers. Anything north of 20mb seems to be become sluggish to navigate, regardless of best practices used. I will usually split when possible with WorkspaceRunner as you mentioned.

I have maintained and built many large workbenches that were originally built in FME2010/2011 timeframe. Pre-AttributeManager, in some instances it was required to string together many attribute transformers such as StringConcatenator, AttributeCreator, StringReplacer, AttributeKeeper, AttributeRemover, etc. With the advent of AttributeManager, my file sizes have drastically been reduced since this transformers can handle so many tasks in itself. This may be applicable in your case.

 

My biggest problem with large files seems to be physical .fmw file size compared to the actual number of transformers. Anything north of 20mb seems to be become sluggish to navigate, regardless of best practices used. I will usually split when possible with WorkspaceRunner as you mentioned.

Thanks ddbrocato, this workspace is from 2020 and it's already using AttributeManager.

The fmw is only 8MB, so I guess I'm lucky :-)

Reply


Community Stats

30,171
Posts
114,531
Replies
37,828
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.