Skip to main content
Solved

Best Practices for Workbench Maintenance

agelfert
Contributor
Forum|alt.badge.img+8

I'm running into issues with ageing workbenches that are running as workspaces on FME Server. Most of these I did not build, many I've never published, some I've had to republish to make minor configuration changes. But I've not made any effort of somehow bringing them all up to the same "standard". Currently, I'm also still on server/Desktop 2018.

I'm familiar with the ability to upgrade readers/writers in place when upgrades are available. But is there any guidance for keeping a clean house with regard to your portfolio of workspaces? I'd like to avoid having various workspaces of different vintage, i.e. created in different versions of FME. But I'm reluctant to just update every reader/writer and republish out of fear I will break too many things.

Any guidance, or list of caveats for said approach would be helpful. - THANKS.

Best answer by jdh

Generally, but with a few significant exceptions, republishing a workspace with a newer version of FME does not change the functionality of the workspace.

 

 

In many cases the performance benefits are in the underlying code, so you can take advantage of them without upgrading the transformers. This can lead to the significant exceptions, for instance, the way the Buffer treated features that either collapse or become multipart with a negative buffer value, changed somewhere between 2013 and 2018, and that caused about a dozen of our workspaces to crash or produce erroneous results.

 

 

I think the best practice is to test all the workspaces (with as many edge cases as you can) when a new version of FME is available, but only update them if there is a behaviour change, or a known performance improvement. (ex bulk mode).

 

 

If however you are editing a workspace anyhow, it's worth seeing if you can get hours for refactoring. When I need to update a workspace for functionality reasons, I request an additional 10-20 hours for refactoring and improving annotations.

 

 

Use the history table in the workspace description.

If at all possible use a version control system. If it's not possible, back up your workspace before starting to edit, preferably with a datestamp in the filename.

 

 

Keep an external record of when workspaces were published to FME Server. We once had a bug creep in that was not caught for 8 months. Knowing the exact dates that version of the workspace was live, gave us the ability to identify all datasets affected by the bug and correct them automatically.
View original
Did this help you find an answer to your question?

9 replies

david_r
Evangelist
  • david_rEvangelist
  • April 23, 2020

My philosophy is to not change a winning team, i.e. if it works and I don't have to change anything else, then I won't touch the workspace to do any upgrades just for the sake of it. The risk of changes in the result is simply too high and I cannot justify the effort.

However, once I have some maintenance work to do that requires validating the whole workspace, then I usually spend a little time to also update any old transformers, in particular if I think that here's something to gain in functionality or performance.

 

salvaleonrp
1 person upvoted this
  • salvaleonrp
    salvaleonrp
redgeographics
Celebrity
Forum|alt.badge.img+48

In addition to @david_r's comment, I would generally only update workspaces if I think there's a performance gain to be had and then only if they need to be run on a regular basis.

If it works, don't touch it :)

salvaleonrp
1 person upvoted this
  • salvaleonrp
    salvaleonrp
agelfert
Contributor
Forum|alt.badge.img+8
  • agelfertContributor
  • Author
  • Contributor
  • April 23, 2020
david_r wrote:

My philosophy is to not change a winning team, i.e. if it works and I don't have to change anything else, then I won't touch the workspace to do any upgrades just for the sake of it. The risk of changes in the result is simply too high and I cannot justify the effort.

However, once I have some maintenance work to do that requires validating the whole workspace, then I usually spend a little time to also update any old transformers, in particular if I think that here's something to gain in functionality or performance.

 

@david_r, @redgeographics - So I can sympathize with the "if it ain't broke" approach. But this also quickly leads to you being landlord over a giant house of cards that works until all the sudden it doesn't. I mean I'm sure it's all a function of how you're set up within your company and what role or niche FME occupies. But I have countless processes reading from and writing to various versioned SDE feature classes. This can make tweaking one little thing to something that works a risky proposition. I'm in the process of building a more robust test environment mirroring production. Hopefully that can be my first line of defense.

Is there an easy to identify or predict what will change when I republish a workspace using a newer version of FME Desktop? That's what I'm most afraid of right now: overwriting a workspace created in 2017 or older with a more recent version of Desktop.

david_r
Evangelist
  • david_rEvangelist
  • April 23, 2020
agelfert wrote:

@david_r, @redgeographics - So I can sympathize with the "if it ain't broke" approach. But this also quickly leads to you being landlord over a giant house of cards that works until all the sudden it doesn't. I mean I'm sure it's all a function of how you're set up within your company and what role or niche FME occupies. But I have countless processes reading from and writing to various versioned SDE feature classes. This can make tweaking one little thing to something that works a risky proposition. I'm in the process of building a more robust test environment mirroring production. Hopefully that can be my first line of defense.

Is there an easy to identify or predict what will change when I republish a workspace using a newer version of FME Desktop? That's what I'm most afraid of right now: overwriting a workspace created in 2017 or older with a more recent version of Desktop.

You raise several very good questions, so let me give you my take, one by one:

So I can sympathize with the "if it ain't broke" approach. But this also quickly leads to you being landlord over a giant house of cards that works until all the sudden it doesn't.

I have several clients working with workspaces authored in FME 2013, still running without a problem in FME 2019. So personally I have never encountered this issue. Safe has a very good track record for backwards compatibility, so I don't think it's necessary to worry about it.

I mean I'm sure it's all a function of how you're set up within your company and what role or niche FME occupies.

Oh, definitely. I'll be the first to disclose that I work as an external consultant, which of course imposes limitations on how proactive you can be when it comes to doing maintenance work "just because" you want to, unless you have a (rare) client that will pay for this. Although I'm not sure I'd do things very differently if I was working as an employee for one specific client.

Is there an easy to identify or predict what will change when I republish a workspace using a newer version of FME Desktop?

There are never any hard guarantees, neither in FME or in life in general. There is always the risk that something unforeseen and out of your control will move the goalposts or throw a stick in your machine. This is why any critical functionality requires a separate test environment and why proper integration testing and quality control before each new release is so important.

That's what I'm most afraid of right now: overwriting a workspace created in 2017 or older with a more recent version of Desktop.

This is the role of a dedicated version control system. Be it git, subversion or whatever, use it to your advantage and never look back!

jdh
Contributor
Forum|alt.badge.img+28
  • jdhContributor
  • Contributor
  • Best Answer
  • April 23, 2020

Generally, but with a few significant exceptions, republishing a workspace with a newer version of FME does not change the functionality of the workspace.

 

 

In many cases the performance benefits are in the underlying code, so you can take advantage of them without upgrading the transformers. This can lead to the significant exceptions, for instance, the way the Buffer treated features that either collapse or become multipart with a negative buffer value, changed somewhere between 2013 and 2018, and that caused about a dozen of our workspaces to crash or produce erroneous results.

 

 

I think the best practice is to test all the workspaces (with as many edge cases as you can) when a new version of FME is available, but only update them if there is a behaviour change, or a known performance improvement. (ex bulk mode).

 

 

If however you are editing a workspace anyhow, it's worth seeing if you can get hours for refactoring. When I need to update a workspace for functionality reasons, I request an additional 10-20 hours for refactoring and improving annotations.

 

 

Use the history table in the workspace description.

If at all possible use a version control system. If it's not possible, back up your workspace before starting to edit, preferably with a datestamp in the filename.

 

 

Keep an external record of when workspaces were published to FME Server. We once had a bug creep in that was not caught for 8 months. Knowing the exact dates that version of the workspace was live, gave us the ability to identify all datasets affected by the bug and correct them automatically.
agelfert
Contributor
Forum|alt.badge.img+8
  • agelfertContributor
  • Author
  • Contributor
  • April 24, 2020
david_r wrote:

You raise several very good questions, so let me give you my take, one by one:

So I can sympathize with the "if it ain't broke" approach. But this also quickly leads to you being landlord over a giant house of cards that works until all the sudden it doesn't.

I have several clients working with workspaces authored in FME 2013, still running without a problem in FME 2019. So personally I have never encountered this issue. Safe has a very good track record for backwards compatibility, so I don't think it's necessary to worry about it.

I mean I'm sure it's all a function of how you're set up within your company and what role or niche FME occupies.

Oh, definitely. I'll be the first to disclose that I work as an external consultant, which of course imposes limitations on how proactive you can be when it comes to doing maintenance work "just because" you want to, unless you have a (rare) client that will pay for this. Although I'm not sure I'd do things very differently if I was working as an employee for one specific client.

Is there an easy to identify or predict what will change when I republish a workspace using a newer version of FME Desktop?

There are never any hard guarantees, neither in FME or in life in general. There is always the risk that something unforeseen and out of your control will move the goalposts or throw a stick in your machine. This is why any critical functionality requires a separate test environment and why proper integration testing and quality control before each new release is so important.

That's what I'm most afraid of right now: overwriting a workspace created in 2017 or older with a more recent version of Desktop.

This is the role of a dedicated version control system. Be it git, subversion or whatever, use it to your advantage and never look back!

Thank you, @david_r. All good points. I'm actually in the process of "populating" and really start using a dedicated test environment that's been lingering unused. So hopefully that along with version control will make my life easier going forward. Much of my frustration goes back to unscrambling the inner working and sometimes purpose of workspaces that I have inherited. So I wasn't trying to suggest FME isn't working. When some of my observations seemingly don't line up with how I think FME should be working, it's probably because I don't know where to look.

I've been using git with other development initiatives and have had it on my projects list for a while to set something up for workbench development. Any specific flavor or product for version control that's worked out particularly well for you? Thanks.

agelfert
Contributor
Forum|alt.badge.img+8
  • agelfertContributor
  • Author
  • Contributor
  • April 24, 2020
jdh wrote:

Generally, but with a few significant exceptions, republishing a workspace with a newer version of FME does not change the functionality of the workspace.

 

 

In many cases the performance benefits are in the underlying code, so you can take advantage of them without upgrading the transformers. This can lead to the significant exceptions, for instance, the way the Buffer treated features that either collapse or become multipart with a negative buffer value, changed somewhere between 2013 and 2018, and that caused about a dozen of our workspaces to crash or produce erroneous results.

 

 

I think the best practice is to test all the workspaces (with as many edge cases as you can) when a new version of FME is available, but only update them if there is a behaviour change, or a known performance improvement. (ex bulk mode).

 

 

If however you are editing a workspace anyhow, it's worth seeing if you can get hours for refactoring. When I need to update a workspace for functionality reasons, I request an additional 10-20 hours for refactoring and improving annotations.

 

 

Use the history table in the workspace description.

If at all possible use a version control system. If it's not possible, back up your workspace before starting to edit, preferably with a datestamp in the filename.

 

 

Keep an external record of when workspaces were published to FME Server. We once had a bug creep in that was not caught for 8 months. Knowing the exact dates that version of the workspace was live, gave us the ability to identify all datasets affected by the bug and correct them automatically.

Thanks, @jdh! You're really giving me some ammo for the kinds of initiatives I've been implementing around documenting workspaces and wanting to set up source control. I'll have another post on the documentation part here shortly. Good to know I'm on the right track.

Quoting from response to @david_r below: " Any specific flavor or product for version control that's worked out particularly well for you?"

Thanks again.

jdh
Contributor
Forum|alt.badge.img+28
  • jdhContributor
  • Contributor
  • April 27, 2020
agelfert wrote:

Thanks, @jdh! You're really giving me some ammo for the kinds of initiatives I've been implementing around documenting workspaces and wanting to set up source control. I'll have another post on the documentation part here shortly. Good to know I'm on the right track.

Quoting from response to @david_r below: " Any specific flavor or product for version control that's worked out particularly well for you?"

Thanks again.

Use whatever version control system your organisation already has. Given that FME workspaces mix layout and function, you're not going to be using advanced features like "integrate changes", or a text based "find differences".

 

If your starting from scratch go with github because once you upgrade to Server2019 there is native support for linking to a github repository.

david_r
Evangelist
  • david_rEvangelist
  • April 28, 2020
jdh wrote:

Use whatever version control system your organisation already has. Given that FME workspaces mix layout and function, you're not going to be using advanced features like "integrate changes", or a text based "find differences".

 

If your starting from scratch go with github because once you upgrade to Server2019 there is native support for linking to a github repository.

Yes, definitely use whatever is already in place, whenever possible.

I'm a bit more reserved about Github, however, as lots of organizations don't allow hosting source on cloud servers, but your mileage my vary.

Reply


Helpful Members This Week

Community Stats

30,940
Posts
117,251
Replies
38,742
Members

Latest FME

Terms & ConditionsCookie settings

Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.