How to configure FME Form to work with Anaconda

Hi @willydevera,

Use the Custom Python Interpreter setting in Workbench's Tools > FME Options > Translation > Python Interpreter. For the Preferred Python Interpreter, set the Anaconda environment's python3*.dll as the Preferred Python Interpreter and the parent directory of the .dll as the Python Home parameter.

You would also need to ensure the Python version of the Anaconda environment is supported by the FME version used.

What sort of error are you getting after configuring the above?

Hello @debbiatsafe ,

Thanks for your response.

That’s the current setting in my FME as you indicated but still running into issues.

It has no problem importing Python base libraries like os, re, sys, and collections; but then it spits out error when importing pandas and pdfplumber in my case.

Below is the error message:

Message Type: fme::internal::_v0::py::Exception
Python Exception <ModuleNotFoundError>: No module named 'pandas'
 

Thanks!

Hi ​@willydevera 

Unfortunately, I think you may be running into a bug. I was able to reproduce the error you are seeing with Anaconda3 2024.10-1 (Python 3.12.7 64-bit).

If you look at the log file, you may see a warning that looks like:

The User-specified library C:\Users\<user>\.conda\envs\<venvName>\python312.dll could not be loaded

This means that FME is not actually using the Conda virtual environment that you have specified and falls back to using the Python 3.12 interpreter shipped with FME. Our development team will investigate this issue and I will let you know once I have heard back from them.

In my testing, I have found that FME is able to load Python 3.8 or Python 3.9 Conda virtual environments. You can try using those Python versions if the libraries you use does not require a newer Python version.

Alternatively, you can install pandas and pdfplumber to a location where FME will automatically search following the instructions at https://docs.safe.com/fme/html/FME-Form-Documentation/FME-Form/Workbench/Installing-Python-Packages.htm

Thanks muchly ​@debbiatsafe  for looking into this.

Yes, that’s exactly the error prompt or warning that I saw; unfortunately I’m unable to move to lower Python version 3.8 or 3.9 as it would violate the security recommendations of our IT Department. I have likewise tried the instructions you indicated in the above link but failed. So I will retry until we hear from your development team. 

In the meantime though, I will explore using RCaller transformer instead of PythonCaller. So that means I will have port the python script into R and use the RCaller transformer. Fingers crossed.

Again, thank you!

Hi ​@debbiatsafe just to update you:

I run as Admin and changed directory to “C:\Program Files\FME\fmepython312>” and invoked the command: ‘fme.exe python -m pip install --upgrade pip’ and here’s the error message:

WARN  : The User-specified library C:\Users\wdevera\AppData\Local\anaconda3\python311.dll could not be loaded
INFORM: Using Python interpreter from `C:\Program Files\FME\fmepython312\python312.dll' with PYTHONHOME `C:\Program Files\FME\fmepython312'
INFORM: Python version 3.12 loaded successfully
Collecting pip
  Using cached pip-24.3.1-py3-none-any.whl.metadata (3.7 kB)
Using cached pip-24.3.1-py3-none-any.whl (1.8 MB)
Installing collected packages: pip
Successfully installed pip-24.3.1
ERROR: Exception:
Traceback (most recent call last):
  File "C:\Program Files\FME\python\pip\_internal\cli\base_command.py", line 180, in exc_logging_wrapper
    status = run_func(*args)
             ^^^^^^^^^^^^^^^
  File "C:\Program Files\FME\python\pip\_internal\cli\req_command.py", line 245, in wrapper
    return func(self, options, args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\FME\python\pip\_internal\commands\install.py", line 510, in run
    self._handle_target_dir(
  File "C:\Program Files\FME\python\pip\_internal\commands\install.py", line 520, in _handle_target_dir
    ensure_dir(target_dir)
  File "C:\Program Files\FME\python\pip\_internal\utils\misc.py", line 105, in ensure_dir
    os.makedirs(path)
  File "<frozen os>", line 225, in makedirs
FileNotFoundError: [WinError 2] The system cannot find the file specified: 'C:\\Users\\wdevera\\OneDrive - Pinchin Ltd\\Documents\\FME\\Plugins\\Python\\python312'

[notice] A new release of pip is available: 24.0 -> 24.3.1
[notice] To update, run: python.exe -m pip install --upgrade pip
ERROR : Error in running the Python command 'python %0'

Program Terminating

Translation FAILED.

Hence, if we are able to redirect the default FME PYTHONHOME to load the python*.dll from Anaconda, perhaps that might work…?

​@debbiatsafe 

PS: The  FME python base at “C:\Program Files\FME\fmepython312>” does not even contain the command “pip”. And so I’m unable to invoke the 1st command in your link  “fme.exe python -m pip install <package_name>” while trying to install pandas.

Hello ​@willydevera 

You’ve gone one level too deep for the python pip install command. The correct command to install pandas for example using FME’s Python:

C:\Program Files\FME\python.exe -m pip install pandas

You’ll want to clear the custom Python interpreter set in Tools > FME Options > Translation > Python Interpreter > Preferred Python Interpreter and set it to one of the FME Python options first before running the command.

We’ve also seen odd permission issues when Python libraries are installed using elevated privileges, but Workbench is not run as admin. I would recommend not using admin when installing Python libraries.

Hi ​@debbiatsafe ,

I’m unable to execute the command:

C:\Program Files\FME\python.exe -m pip install pandas

since there is no python.exe existing in the folder.  Only “python_loader.dll” exists in this folder.

And invoking python.exe launches Microsoft Store to get python; hence would you recommend doing this?

Hi ​@willydevera 

My sincerest apologies for the delayed response and for a typo in my previous response! The command should be:

C:\Program Files\fme.exe python -m pip install pandas

This should allow you to install the pandas library to %UserProfile%\Documents\FME\Plugins\Python\python312.

Hi ​@debbiatsafe ,

Thanks for reverting back to me.

I ran the command “fme.exe python -m pip install pandas”; however still run into the following error:

******************************************* 

Installing collected packages: pytz, tzdata, six, numpy, python-dateutil, pandas
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
contourpy 1.1.1 requires numpy<2.0,>=1.26.0rc1; python_version >= "3.12", but you have numpy 2.2.0 which is incompatible.
matplotlib 3.7.4 requires numpy<2,>=1.20, but you have numpy 2.2.0 which is incompatible.
Successfully installed numpy-2.2.0 pandas-2.2.3 python-dateutil-2.9.0.post0 pytz-2024.2 six-1.17.0 tzdata-2024.2
ERROR: Exception:
Traceback (most recent call last):
  File "C:\Program Files\FME\python\pip\_internal\cli\base_command.py", line 180, in exc_logging_wrapper
    status = run_func(*args)
             ^^^^^^^^^^^^^^^
  File "C:\Program Files\FME\python\pip\_internal\cli\req_command.py", line 245, in wrapper
    return func(self, options, args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\FME\python\pip\_internal\commands\install.py", line 510, in run
    self._handle_target_dir(
  File "C:\Program Files\FME\python\pip\_internal\commands\install.py", line 520, in _handle_target_dir
    ensure_dir(target_dir)
  File "C:\Program Files\FME\python\pip\_internal\utils\misc.py", line 105, in ensure_dir
    os.makedirs(path)
  File "<frozen os>", line 225, in makedirs
FileNotFoundError: [WinError 2] The system cannot find the file specified: 'C:\\Users\\wdevera\\OneDrive - Pinchin Ltd\\Documents\\FME\\Plugins\\Python\\python312'

[notice] A new release of pip is available: 24.0 -> 24.3.1
[notice] To update, run: C:\Program Files\FME\fmepython312\python.exe -m pip install --upgrade pip
ERROR : Error in running the Python command 'python %0'

Program Terminating

Translation FAILED.

****************************

Then I tried executing the command to update pip: “C:\Program Files>fme.exe python -m pip install --upgrade pip” but still run into error:

ERROR: Exception:
Traceback (most recent call last):
  File "C:\Program Files\FME\python\pip\_internal\cli\base_command.py", line 180, in exc_logging_wrapper
    status = run_func(*args)
             ^^^^^^^^^^^^^^^
  File "C:\Program Files\FME\python\pip\_internal\cli\req_command.py", line 245, in wrapper
    return func(self, options, args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\FME\python\pip\_internal\commands\install.py", line 510, in run
    self._handle_target_dir(
  File "C:\Program Files\FME\python\pip\_internal\commands\install.py", line 520, in _handle_target_dir
    ensure_dir(target_dir)
  File "C:\Program Files\FME\python\pip\_internal\utils\misc.py", line 105, in ensure_dir
    os.makedirs(path)
  File "<frozen os>", line 225, in makedirs
FileNotFoundError: [WinError 2] The system cannot find the file specified: 'C:\\Users\\wdevera\\OneDrive - Pinchin Ltd\\Documents\\FME\\Plugins\\Python\\python312'

[notice] A new release of pip is available: 24.0 -> 24.3.1
[notice] To update, run: C:\Program Files\FME\fmepython312\python.exe -m pip install --upgrade pip
ERROR : Error in running the Python command 'python %0'

Program Terminating

Translation FAILED.

*****************************************

Hence, it looks like this time & I guess, the pip command is missing in FME base:

****************************************

C:\Program Files\FME\fmepython312>python.exe -m pip install --upgrade pip
C:\Program Files\FME\fmepython312\python.exe: No module named pip