Solved

When is success not success?

  • 21 November 2019
  • 13 replies
  • 36 views
markdmclean
Badge

Hi,

 

I have an issue with a workspace that reads from AGOL and, among other things, writes to an SDE.

 

The workspace completes successfully, but then returns an error – as below:

 

Translation was SUCCESSFUL

Error running translation.

 

This isn’t so much of an issue on desktop – as the features are written correctly, but on Server (it’s destined to be a nightly scheduled task) that final error is enough to restart the workspace – ad infinitum.

 

I have narrowed the issue down to the SDE Writer and have tried many ‘solutions’, without success. These include:

  • Replacing the SDE Writer with a Feature Writer
  • Converting the SDE Feature Class to non-relational
  • Committing transactions at the end of write
  • Various transaction types
  • Audit logging (no additional details on the final error)
  • etc

 

Currently running FME Desktop 2018.1.2.1 (32bit) on Windows 7.

 

Any help and advice would be welcome.

 

Thanks,

Mark

icon

Best answer by markdmclean 28 November 2019, 05:26

View original

13 replies

nic_ran
Rank
Badge +11

Mark,

without seeing your data it's hard to know exactly what's going on but it looks to me like it might be something to do with the data you're writing to SDE.

Perhaps there are features with an incorrect geometry type, or invalid geometries being written? Another time I've seen similar issues is if the source data is defining a feature dataset that's not being cleared before attempting to write (expose the geodb_feature_dataset format attribute and make sure it's empty).

It might be worth trying running the process with subsets of your data to see if you can narrow down which feature(s) are causing the error.

On a more general note -- I do feel that this type of behaviour with FME is not very helpful. Either the translation is successful (in which case do not issue an error) or it has an error and should be failed. If this is not a true error, then it should be issued as a warning. Something for the developers to look into I think.

Nic

virtualcitymatt
Rank
Userlevel 4
Badge +26

This sounds like a bug that was was fixed - I'm just trying to find the article

virtualcitymatt
Rank
Userlevel 4
Badge +26

This sounds like a bug that was was fixed - I'm just trying to find the article

OK well here's the article: https://knowledge.safe.com/articles/63592/fme-exe-has-stopped-working-after-updating-windows.html

 

 

But it mentions that no one has had the problem with 32 bit. I would still look into it as it looks like the fme.exe process is crashing, this is why FME Server is resubmitting it.

 

 

Did you recently update Windows 7 or a new version of ArcGIS?
virtualcitymatt
Rank
Userlevel 4
Badge +26

OK well here's the article: https://knowledge.safe.com/articles/63592/fme-exe-has-stopped-working-after-updating-windows.html

 

 

But it mentions that no one has had the problem with 32 bit. I would still look into it as it looks like the fme.exe process is crashing, this is why FME Server is resubmitting it.

 

 

Did you recently update Windows 7 or a new version of ArcGIS?

Also check that you're not running any shutdown scripts - I would expect a different message if the python scripts failed, however, it's worth checking anyway.

mark2atsafe
Rank
Userlevel 4
Badge +25

You could try turning on Log Debug (under Tools > FME Options). If you save the workspace in Workbench with that option on I believe it will be carried through and would cause Server to log more information.

Having said that, I don't know if it will reveal much. It sounds to me like it completes the translation and then fails when shutting down the engine (maybe?) - are there any Server-specific log files that you can check into? You don't mention them, so I thought I'd ask.

markdmclean
Badge

Hi all, thanks for the comments and suggestions.

Mark: I did run with the log debug on, but, as the error occurs at the very end of the transformation(?) it didn't reveal any additional information.

Matt: SAFE sent me that links (as did our local vendor), but we are 32bit all the way (FME Desktop and Server 2018.1.2.1, ArcGIS and ArcFM 10.2.1) the only 64bit is the W7 OS. I did apply the ESRI patch to my desktop install, but no difference. FME doesn't stop working either - just sits there happily waiting for the next instruction...

Nic: This could be worthy of further checks - I know a few of the test points are located way beyond Jupiter, but I am not working with spatial extents.

More background: The purpose of the workspace is to retrieve contractor hydrant inspection point data from AGOL, sift out new records (using cyclic redundancy checks), massage attribution to fit the SDE Schema, upload photo attachments to our document management system, deposit new point features in the SDE (versioned FC with relationship tables).

There are two versions of the workspace:

1] Fire service - has additional section for handling urgent inspections (email notifications).

This (despite generating the final error) works on FME Server 2018.1.2.1 (not on 2019.1.3)....

2] Private company - this, despite being slightly less complex, doesn't work on server - restarting several times before giving up. Perhaps the underlying data, as per Nic's suggestion, is the issue - I'll investigate further.

Keep the comments and suggestions coming.

Thanks,

Mark

 

virtualcitymatt
Rank
Userlevel 4
Badge +26

Hi all, thanks for the comments and suggestions.

Mark: I did run with the log debug on, but, as the error occurs at the very end of the transformation(?) it didn't reveal any additional information.

Matt: SAFE sent me that links (as did our local vendor), but we are 32bit all the way (FME Desktop and Server 2018.1.2.1, ArcGIS and ArcFM 10.2.1) the only 64bit is the W7 OS. I did apply the ESRI patch to my desktop install, but no difference. FME doesn't stop working either - just sits there happily waiting for the next instruction...

Nic: This could be worthy of further checks - I know a few of the test points are located way beyond Jupiter, but I am not working with spatial extents.

More background: The purpose of the workspace is to retrieve contractor hydrant inspection point data from AGOL, sift out new records (using cyclic redundancy checks), massage attribution to fit the SDE Schema, upload photo attachments to our document management system, deposit new point features in the SDE (versioned FC with relationship tables).

There are two versions of the workspace:

1] Fire service - has additional section for handling urgent inspections (email notifications).

This (despite generating the final error) works on FME Server 2018.1.2.1 (not on 2019.1.3)....

2] Private company - this, despite being slightly less complex, doesn't work on server - restarting several times before giving up. Perhaps the underlying data, as per Nic's suggestion, is the issue - I'll investigate further.

Keep the comments and suggestions coming.

Thanks,

Mark

 

One other thing you might want to look at is the windows ProcessMonitor https://docs.microsoft.com/en-us/sysinternals/downloads/procmon . If you set it up to monitor the fme.exe process you can see what calls it is making before it dies. This could help see what's happening. You mentioned ArcFM - I suspect that this could be part of the problem somehow although I have no idea how.

 

 

markdmclean
Badge

So... after doing some more side by side comparison of the working and non-working workspaces and a few minor tweaks (of no perceived benefit) I now have the following situation:

Both workspaces run on FME Dev Server (2018.1.2.1)...

The previously not working workspace also runs on our FME PRD Server (2019.1.3), but the Fire Service workspace still doesn't...

As with most development issues, the cause will be something simple/trivial, but it would be good to get to the bottom of the issue to avoid repetition in future.

Thanks to all the Contributors for assistance to date.

 

markdmclean
Badge

Also check that you're not running any shutdown scripts - I would expect a different message if the python scripts failed, however, it's worth checking anyway.

Hi Matt, no shutdown scripts. I do have a pythonCaller at the end to remove temporary files - after successful upload.

markdmclean
Badge

Again - thanks for the inputs, suggestions and ideas.

It boiled down to the fact that the Fire Service Workspace issues email notifications when urgent inspections are required and we haven't yet configured email notifications on our PRD Server (Doh!).

Disabling that section allowed the workspace to run successfully...

It would still be nice if the logs would indicate this issue, but at least the issue is solved for now.

markdmclean
Badge

Again - thanks for the inputs, suggestions and ideas.

It boiled down to the fact that the Fire Service Workspace issues email notifications when urgent inspections are required and we haven't yet configured email notifications on our PRD Server (Doh!).

Disabling that section allowed the workspace to run successfully...

It would still be nice if the logs would indicate this issue, but at least the issue is solved for now.

Further narrowed this down to a ListBuilder in the email configuration section - the mere existence of this transformer, even if the email section isn't accessed, stops the workspace on Server. I have replaced the ListBuilder with a ListPopulator and all looks promising so far - just waiting for IT to open up the email port (25).

nic_ran
Rank
Badge +11

Further narrowed this down to a ListBuilder in the email configuration section - the mere existence of this transformer, even if the email section isn't accessed, stops the workspace on Server. I have replaced the ListBuilder with a ListPopulator and all looks promising so far - just waiting for IT to open up the email port (25).

I created a very basic workspace using a Creator > AttributeCreator > ListBuilder. Then I published it to FME Server and ran it without any issue on FME (Desktop & Server) 2019.2. Added the subsequent ListConcatenator and still no issues.

You could try placing a Recorder prior to the ListBuilder, then creating a new workspace with a Player and the ListBuilder and publishing that to FME Server to see if it's a data issue.

markdmclean
Badge

Removing attachments and versioning from the SDE Feature Class removed this error.

Reply


Community Stats

28,753
Posts
109,621
Replies
36,587
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.