Library 'D:\\Program Files\\FMEServer\\Server\\fme\\plugins/CSV2.dll' was found but could not be loaded. Ensure that all the dependent modules exist for this library

I'm having some issues with our FME Server which I don't understand. Every now and then, when running a nightly scheduled job, a FeatureReader fails to read a CSV file. The log returns:

Trying to find a DYNAMIC plugin for reader named `CSV2'
Library 'D:\Program Files\FMEServer\Server\fme\plugins/CSV2.dll' was found but could not be loaded. Ensure that all the dependent modules exist for this library
Module 'CSV2' is unavailable for use with this FME edition

When I resubmit the job, everything works fine... which is great for the data being processed, but harder to debug.

The issue feels somewhat familiar and I did find similar questions but not related to CSV2.dll.

I'm quite sure this is not related to desktop/server versions. Version is 2019.1.2.

As far as I am aware nothing has changed in the installation, I do know the certificates were replaced recently but I doubt this has anything to do with it.

Not sure where I should start looking, can anyone point me in the right direction?

Update: I found some jobs with the same kind of logging. These are writers instead of readers and caused the job to fail.

Trying to find a DYNAMIC plugin for writer named `ESRIJSON'
Library 'D:\Program Files\FMEServer\Server\fme\plugins/JSON.dll' was found but could not be loaded. Ensure that all the dependent modules exist for this library
Module 'ESRIJSON' is unavailable for use with this FME edition

and

Trying to find a DYNAMIC plugin for writer named `CSV'
Library 'D:\Program Files\FMEServer\Server\fme\plugins/CSV.dll' was found but could not be loaded. Ensure that all the dependent modules exist for this library
Module 'CSV' is unavailable for use with this FME edition

Page 1 / 1

Would you be in a position to disable antivirus and test again? Could be interfering?

Thanks for this suggestion, I'm gonna check with IT.

For documentation:

We got the suggestion that the cause of this error is memory related. In our case this happened during the night when several update jobs are scheduled. We have a bunch of schedules with the same start time and let the queue work through the jobs.

A possible solution was to lower the number of succesful jobs to initiate an engine restart. Also see How to Control FME Server Engine Memory Usage

To check if this might be the issue, I re-scheduled the jobs with issues an hour later and the problem went away.

This issue also might be caused by a server / desktop version mismatch, desktop being newer than server. In our case this was not a possible cause.

For documentation:

A possible solution was to lower the number of succesful jobs to initiate an engine restart. Also see How to Control FME Server Engine Memory Usage

To check if this might be the issue, I re-scheduled the jobs with issues an hour later and the problem went away.

This issue also might be caused by a server / desktop version mismatch, desktop being newer than server. In our case this was not a possible cause.

I have started seeing this pop up for users on FME Desktop. Both using FME 2021.1. The workspace was built with FME 2020.

The messages are specific to writers and sometimes fail and sometimes don't.

The workspace is pretty huge - Well over 100 transformers. I have had no reported issues on FME 2020 yet

@nielsgerrits anything similar to what you were seeing? The timing of your post seems like it could be FME version related (as in I've not seen this until now)

Edit* just saw that you're running FME 2019. I guess it's not version related

Would you be in a position to disable antivirus and test again? Could be interfering?

@hkingsbury have you seen this before?

Sporadilcally, but never repeating like in this case

I have started seeing this pop up for users on FME Desktop. Both using FME 2021.1. The workspace was built with FME 2020.

The messages are specific to writers and sometimes fail and sometimes don't.

The workspace is pretty huge - Well over 100 transformers. I have had no reported issues on FME 2020 yet

@nielsgerrits anything similar to what you were seeing? The timing of your post seems like it could be FME version related (as in I've not seen this until now)

Edit* just saw that you're running FME 2019. I guess it's not version related

Hej @virtualcitymatt I had similar errors on desktop (2016), but these where caused by something different.

This workspace was reading data from gdb and writing to gdb and dwg. Had to do with a DLL version conflict between the arcobjects and the realdwg writer. The workaround was to read an empty dwg first to force that dll being used. This workspace still runs with the workaround. Might be your issue as well? See this topic.

I have started seeing this pop up for users on FME Desktop. Both using FME 2021.1. The workspace was built with FME 2020.

The messages are specific to writers and sometimes fail and sometimes don't.

The workspace is pretty huge - Well over 100 transformers. I have had no reported issues on FME 2020 yet

@nielsgerrits anything similar to what you were seeing? The timing of your post seems like it could be FME version related (as in I've not seen this until now)

Edit* just saw that you're running FME 2019. I guess it's not version related

Ahh interesting, I have added in some 3rd partly python libs which have some binaries, perhaps this is where the problem is coming from. Thanks, will report back if anything

I have started seeing this pop up for users on FME Desktop. Both using FME 2021.1. The workspace was built with FME 2020.

The messages are specific to writers and sometimes fail and sometimes don't.

The workspace is pretty huge - Well over 100 transformers. I have had no reported issues on FME 2020 yet

@nielsgerrits anything similar to what you were seeing? The timing of your post seems like it could be FME version related (as in I've not seen this until now)

Edit* just saw that you're running FME 2019. I guess it's not version related

Did you find a cause in any 3rd party python libs?

I encounter a similar issue by upgrading FME Server 2018.1 towards 2021.0. An (extremly large workbench of 474 transformers) which was doing fine at 2018.1 returns errors as "Library 'D:\\Program Files\\FMEServer\\Server\\fme\\plugins/RasterEvaluationFactory.dll' was found but could not be loaded." It does not seem to be memory related. Removing a large part of the transformers (back towards ~80 for a sample run) the workbench runs fine.

Any thoughts?

I have started seeing this pop up for users on FME Desktop. Both using FME 2021.1. The workspace was built with FME 2020.

The messages are specific to writers and sometimes fail and sometimes don't.

The workspace is pretty huge - Well over 100 transformers. I have had no reported issues on FME 2020 yet

@nielsgerrits anything similar to what you were seeing? The timing of your post seems like it could be FME version related (as in I've not seen this until now)

Edit* just saw that you're running FME 2019. I guess it's not version related

Are you running 32 bit FME by chance? The other thing when working with FME Server is to make sure FME Desktop and FME Server versions match when publishing

I have started seeing this pop up for users on FME Desktop. Both using FME 2021.1. The workspace was built with FME 2020.

The messages are specific to writers and sometimes fail and sometimes don't.

The workspace is pretty huge - Well over 100 transformers. I have had no reported issues on FME 2020 yet

@nielsgerrits anything similar to what you were seeing? The timing of your post seems like it could be FME version related (as in I've not seen this until now)

Edit* just saw that you're running FME 2019. I guess it's not version related

Any luck tracking down your issue @bartm ? the workspace I have here is also a mega workspace. Very possible there is some kind of conflicting DLL somewhere but so far I've been unable to identify it

So I have a little update for anyone interested here. In my testing, exploration of the issue I've discovered the following for the situation I've been facing:

The issue seems to effect Windows Server installations only
The issue goes away if I disable a a fair part of the workspace (it does not seem to matter which part of the workspace)
There seems to be some kind of limit to the amount dlls which can be imported, depending on how much I disable in the workspace changes where the errors comes.
In my testing the 'limit' limit has been around 50 different dlls (concluded from looking through the various log files

I still need to do some more testing but I will be reporting my findings to Safe to see if they are able to give some more explanation. But either way it's clear that, at least on Windows Server, there may be some limit.

So I have a little update for anyone interested here. In my testing, exploration of the issue I've discovered the following for the situation I've been facing:

The issue seems to effect Windows Server installations only
The issue goes away if I disable a a fair part of the workspace (it does not seem to matter which part of the workspace)
There seems to be some kind of limit to the amount dlls which can be imported, depending on how much I disable in the workspace changes where the errors comes.
In my testing the 'limit' limit has been around 50 different dlls (concluded from looking through the various log files

Our fme server is on windows as well.

Windows Server? or Windows 10? I'd also be interested to know if those scheduled jobs you mentioned above are quite large (as in a lot of transformers). If you still have them running I'd be interested to know how many unique dlls get loaded when combing both log files.

Our fme server is on windows as well.

Windows Server

I think this workspace is not active anymore :/

Our fme server is on windows as well.

Probably for the best! I want to decommission this one I have too but sadly I have no replacement yet.

Another update - This issue seems to be gone on Windows Server 2022. If your running Windows Server 2016 or Windows Server 2019 I suggest you try Windows Server 2022.

Tested FME Server 2022.2

Reply

Community Stats

Reply

Community Stats

Sign up

An FME Account is required to contribute

Login to the community

An FME Account is required to contribute

Scanning file for viruses.

This file cannot be downloaded