Upgrading FME versions - How do you do it?

I’ve completed a few upgrades across a number of different organisations. Generally, workspaces are regression tested but not upgraded, unless issues come to light during testing.

Workspaces may get updated as part of other work, but it’s not tied to the FME Flow upgrade cycle

Okay seems like an efficient approach. Have you encountered any problems with older workspaces running on a newer version of FME Flow?

Generally, no. I’d say about 95% of workspaces run without any change.

Where there are issues they tend to be related to ArcGIS upgrades that have happened in parallel or they’re issues that aren’t related to the workspace themselves, but related to changes around security in FME Flow.

The below issue, whereby just opening a workspace in FME Form resets settings, whilst it just works if you leave it on Flow and don’t touch it has reinforced for the ‘if it ain’t broke don’t fix it’ approach for me.

Known Issue: ArcGIS Online FeatureWriter - older workspaces opened in FME 2022.0 or newer have their parameters reset – FME Support Center

Thanks for the input!

Indeed, upgrading workspaces is not necessary in the vast majority of cases. FME is backwards compatible and older workspaces can be expected to run as before, even on newer FME versions.

That said, I agree that doing regression testing is necessary.

Currently very busy with upgrading workspaces, which indeed takes a lot of time. But I’ve found that it is necessary, because I‘ve found several cases where FME’s behavior in the new version is different from that in the old version, causing multiple data issues. One issue turned out to be a bug (see this thread), but in my opinion a new version requires a lot of work to make sure everything still works ok. And backwards compatibility is all very fine, but if you have workspaces that were originally created 7, 9 or even 10 years ago, I’m not sure how far back that compatibility goes.

So we follow your scenario: upgrade every other year, and accept that it takes a lot of work. But sometimes we need a newer version (using OpenAPI or Sharepoint for example).

Sounds like we do it in a similar way then, and for the same reasons. It does seem to be a safe approach at least, since all workspaces get tested before the new instance gets deployed but it is a challenge to manage the project. It tends to drag out over a couple months, all while other developers keep working on the workspaces being upgraded. Since they work in the old instance running the previous version, it is a lot of work to keep track of changes in the different versions. 

We managed the upgraded workspaces in a separate branch in our Git repository, but merging changes between the main branch and the 2025 branch was tough. 

Have you also had to deal with those kind of challenges?

No, since we do not use Git, and there are only 3 of us that actively create and maintain workspaces. So each of us manages their own stuff. Our biggest challenge is indeed time: since the shop must stay open, you also get a lot of other requests/demands on your time. Add to that the fact that upgrading, then testing is not the most interesting/exciting task… 😄

Lucky you :-)
No it is not very exciting haha

It should work in theory, but we found out too many times that there is something that breaks and needs to be fixed anyway. This year was pretty smooth though.. :-) 

We have some testing in place but we generally find that testing FME workspaces is more difficult compared to “traditional” coding. It is hard to implement unit tests for the workspace, so we usually perform tests on the resulting data instead
For the majority of our dataflows we write to a temporary table, compare the table with the current table and replace the tables if the tests are OK. 

This generally works, but in many cases it is pretty much just a test if the number of objects does not change above a certain threshold.

Do you do it differently? It feels like this topic should be well explored, since testing is such an important part of the development workflow. But i find it difficult to get hold of any “best practice” articles about this.

This...

I use quite extensive custom logging, so I tend to compare my own log-files between versions as well, besides the numbers check (I try to run the old-version and the new version back-to-back, so numbers should be the same with little to no margin). And I’ve learned the hard way to check samples of actual data as well (during one test the attribute name in was used to fill a new attribute, instead of the actual value of the old one… and because of reasons I was forced to test that one on Production...)

Even that isn’t foolproof. In a previous role, we had a workspace that was producing a shape file. Count of features and content had been checked against a previous file and were identical, the file had been opened in QGIS and looked fine. However, a defect in shape file index creation meant it didn’t display correctly in ArcGIS. Luckily spotted in UAT but more by accident than design.

Similarly, a single instance of a “ in a few hundred thousand rows of data, meant a csv supplied to an external partner didn’t load correctly due to a change/bug in how qualify field values if needed worked (didn’t work!).

The main pain point i have around testing, is testing automations. Once they become more complex with automations writers etc. it’s not really sufficient to test the workspaces in isolation. But sometimes it can be really difficult to create the data in source systems to properly test, especially with regards to failures/error handling.