Skip to main content

Hi All,

I have a workspace which works fine in desktop, and even works fine in our Pre-Production FME Server environment, but when I try to run on our Production FME Server instance I get the error message above. Any ideas? Full log below (have removed elements that showed senitive things like databases and passwords!):

 

FME 2017.1.1.1 (20171014 - Build 17652 - WIN64) 
 FME_HOME is 'C:\Program Files\FME_Server_2017.1.1.1\Server\fme\' 
 FME Engine (node locked-crc) 
 Serial Number: SAFE 
 Machine host name is: --
 START - ProcessID: 8424, peak process memory usage: 34820 kB, current process memory usage: 34820 kB 
 
 FME Configuration: Reader Keyword is `ORACLE_SPATIAL_1' 
 FME Configuration: Writer Keyword is `MULTI_WRITER' 
 FME Configuration: Writer Group Definition Keyword is `MULTI_WRITER_DEF' 
 FME Configuration: Reader type is `ORACLE_SPATIAL' 
 FME Configuration: Writer type is `MULTI_WRITER' 
 FME Configuration: No destination coordinate system set 
 FME Configuration: Current working folder is `C:\ProgramData\Safe Software\FME Server\repositories' 
 FME Configuration: Temporary folder is in the system location `C:\_FME_TEMP\fmeengines\--_Engine1' 
 FME Configuration: FME_HOME is `C:\Program Files\FME_Server_2017.1.1.1\Server\fme\' 
 FME Configuration: FME_BASE is 'no' 
 FME Configuration: FME_MF_DIR is 'C:\ProgramData\Safe Software\FME Server\repositories\Elizabeth_Line\GWP_MV_EMS_MID_creation_server/' 
 FME Configuration: FME_MF_NAME is 'GWP_MV_EMS_MID_creation_server.fmw' 
 FME Configuration: FME_PRODUCT_NAME is 'FME(R) 2017.1.1.1' 
 System Status: 10.12 GB of disk space available in the FME temporary folder (C:\_FME_TEMP\fmeengines\--_Engine1) 
 System Status: 8.00 TB of virtual memory available 
 Operating System: Microsoft Windows Server 2008 R2 Server 4.0, Enterprise Edition 64-bit Service Pack 1 (Build 7601) 
 FME Platform: WIN64 
 Locale: en_US 
 Code Page: 1252 (ANSI - Latin I) 
 FME Configuration: Process limit is 16.00 GB of physical memory 
 FME Configuration: Start freeing memory when process usage exceeds 48.00 GB of virtual memory 
 FME Configuration: Stop freeing memory when process usage is below 36.00 GB of virtual memory 
 Creating writer for format: Â 
 Warning: not all Stashed Objects that were registered were dropped before shutdown. This may cause instability

Many thanks

Dan

Apologies for formatting on log, something went wrong when posting!


Apologies for formatting on log, something went wrong when posting!

No worries about the ÂÂ's, this is a to be resolved issue since the last forum upgrade.


It is hard to say without workspace, history or more detailled log. Did it work before? What kind of format is used for the writer? Please make sure you don't use a newer Desktop then Server version. No 64/32 bit differences? Is pre production equal to production? (Because often it isn't.)


I don't believe the "not all stashed objects" warning indicates the nature of the problem, especially when the log stops abruptly like that. I think all it means is that the FME process has died and some other task is trying to communicate with it.

So... do you have any Inspector or Logger transformers in the workspace? Sometimes that is the process still trying to carry on, and it can sometimes cause the issue. Perhaps remove any that exist and re-run it.

Otherwise, I agree with Niels to be careful that the Server version is not older than the Desktop on which the workspace was built.

Besides that I would suggest to use standard debugging techniques to find out at what point the error occurs; i.e. have the workspace read data only; then read and transform; then read, transform, and write. That would help identify where the issue occurs, which is half the battle.


I don't believe the "not all stashed objects" warning indicates the nature of the problem, especially when the log stops abruptly like that. I think all it means is that the FME process has died and some other task is trying to communicate with it.

So... do you have any Inspector or Logger transformers in the workspace? Sometimes that is the process still trying to carry on, and it can sometimes cause the issue. Perhaps remove any that exist and re-run it.

Otherwise, I agree with Niels to be careful that the Server version is not older than the Desktop on which the workspace was built.

Besides that I would suggest to use standard debugging techniques to find out at what point the error occurs; i.e. have the workspace read data only; then read and transform; then read, transform, and write. That would help identify where the issue occurs, which is half the battle.

Hi Mark thanks for the response.  I can confirm both the Pre-Prod and Prod FME Server environments are both the same version as listed in the log file above (2017.1.1.1 Build 17652 64 bit) and that there are no loggers or inspectors in the workspace.  I can also confirm that the version of desktop is identical from which it is being published.

One thing I have just remembered is that after publishing the workspace to Server, FME Desktop encounters an error and needs to close (although there's no record of what happened?).  It seems to successfully publish the workspace but could the workspace be corrupt as a result of that?  Is there a dump file created somewhere if this happens?  I've had a look in Event Viewer but all I get is :

Faulting application name: fmeworkbench.exe, version: 2017.7.27.17652, time stamp: 0x59e2ab26
Faulting module name: qfmeext.dll, version: 2017.7.27.17652, time stamp: 0x59e2a8e7
Exception code: 0xc0000005
Fault offset: 0x000000000019dab3
Faulting process id: 0x22d0
Faulting application start time: 0x01d4863254103397
Faulting application path: C:\Program Files\FME_Desktop_2017.1.1.1\fmeworkbench.exe
Faulting module path: C:\Program Files\FME_Desktop_2017.1.1.1\qfmeext.dll
Report Id: c1d96bcb-f225-11e8-9025-0050569b6613

Regards

Dan


It is hard to say without workspace, history or more detailled log. Did it work before? What kind of format is used for the writer? Please make sure you don't use a newer Desktop then Server version. No 64/32 bit differences? Is pre production equal to production? (Because often it isn't.)

Sorry Niels but I think that's all I'm getting in terms of a log. Confirmation above that all versions and OS bit are identical on both servers :(


Hi Mark thanks for the response.  I can confirm both the Pre-Prod and Prod FME Server environments are both the same version as listed in the log file above (2017.1.1.1 Build 17652 64 bit) and that there are no loggers or inspectors in the workspace.  I can also confirm that the version of desktop is identical from which it is being published.

One thing I have just remembered is that after publishing the workspace to Server, FME Desktop encounters an error and needs to close (although there's no record of what happened?).  It seems to successfully publish the workspace but could the workspace be corrupt as a result of that?  Is there a dump file created somewhere if this happens?  I've had a look in Event Viewer but all I get is :

Faulting application name: fmeworkbench.exe, version: 2017.7.27.17652, time stamp: 0x59e2ab26
Faulting module name: qfmeext.dll, version: 2017.7.27.17652, time stamp: 0x59e2a8e7
Exception code: 0xc0000005
Fault offset: 0x000000000019dab3
Faulting process id: 0x22d0
Faulting application start time: 0x01d4863254103397
Faulting application path: C:\Program Files\FME_Desktop_2017.1.1.1\fmeworkbench.exe
Faulting module path: C:\Program Files\FME_Desktop_2017.1.1.1\qfmeext.dll
Report Id: c1d96bcb-f225-11e8-9025-0050569b6613

Regards

Dan

Workbench crashing just after publishing can be a pointer. Does this happen in pre-production as well? Can you reproduce it in production? Can you reproduce it in production with a new name? In a new repository? Etc.

I would start resubmitting the Workspace to Server until the crashing goes away, then check if it runs as it should.

You can change debugging levels in Server, but I would like to try other alternatives before  doing this in production. (Not sure how important your production environment is.)


Workbench crashing just after publishing can be a pointer. Does this happen in pre-production as well? Can you reproduce it in production? Can you reproduce it in production with a new name? In a new repository? Etc.

I would start resubmitting the Workspace to Server until the crashing goes away, then check if it runs as it should.

You can change debugging levels in Server, but I would like to try other alternatives before doing this in production. (Not sure how important your production environment is.)

Hi Niels, only crashes in Production, not in Pre-Prod, I will have a look into resubmitting tomorrow when I am at the office, as there may also be some permission issues on Prod and I need to talk to the admin there before I proceed any further. They can also change the debugging levels if required.


Hi Mark thanks for the response.  I can confirm both the Pre-Prod and Prod FME Server environments are both the same version as listed in the log file above (2017.1.1.1 Build 17652 64 bit) and that there are no loggers or inspectors in the workspace.  I can also confirm that the version of desktop is identical from which it is being published.

One thing I have just remembered is that after publishing the workspace to Server, FME Desktop encounters an error and needs to close (although there's no record of what happened?).  It seems to successfully publish the workspace but could the workspace be corrupt as a result of that?  Is there a dump file created somewhere if this happens?  I've had a look in Event Viewer but all I get is :

Faulting application name: fmeworkbench.exe, version: 2017.7.27.17652, time stamp: 0x59e2ab26
Faulting module name: qfmeext.dll, version: 2017.7.27.17652, time stamp: 0x59e2a8e7
Exception code: 0xc0000005
Fault offset: 0x000000000019dab3
Faulting process id: 0x22d0
Faulting application start time: 0x01d4863254103397
Faulting application path: C:\Program Files\FME_Desktop_2017.1.1.1\fmeworkbench.exe
Faulting module path: C:\Program Files\FME_Desktop_2017.1.1.1\qfmeext.dll
Report Id: c1d96bcb-f225-11e8-9025-0050569b6613

Regards

Dan

It's hard to say for sure that the crash is the cause of this problem, but I certainly think it's worth checking into first of all. The best way is to remove the existing workspace from Server and republish it. If it crashes again, let us know. Note that Workbench crashing doesn't necessarily mean that the workspace was not published correctly - these could be two separate issues. If you're still having problems running the workspace (whether the publishing wizard crashes or not) I suggest contacting our support team at safe.com/support - give them the details of what is happening and they will be able to investigate further. But, again, I would try publishing the workspace with writers disabled (for example) to see if it runs OK like that, just to see if the writing is the issue. Sometimes by disabling certain parts of a workspace, or running it on only a small sample of data, you can pin down the location of the problem, which helps to identify what the issue is.


Reply