Solved

How to make "pycocotools" available in FME Desktop when using a PythonCaller?

  • 10 May 2023
  • 5 replies
  • 18 views
denisgiannelli
Badge

Hi everybody,

I have been trying to experiment the new Meta Segment Anything Model (SAM) library on FME Desktop via a PythonCaller transformer.

After lots of debugging when making SAM compatible with the IFMERaster object, I finally reached the part in which I need to "decode a mask in COCO RLE format into binary."

This is carried out via another library, named pycocotools

"from pycocotools import mask as mask_utils

mask = mask_utils.decode(annotation["segmentation"])"

Apparently there is no way to pip install such library directly and, therefore, a workaround has been proposed by philferriere in his git repository. According to them, "on Windows, you must have the Visual C++ 2015 build tools on your path".

I downloaded Visual Studio Code Build Tools 2022, but I now lack of knowledge to understand what the necessary step-by-step is to make such library available in FME when using the PythonCaller.

Could anyone give me some help? I appreciate VERY MUCH your time and effort in advance!

Greetings,

Denis Giannelli

 

 

 

icon

Best answer by kailinatsafe 16 May 2023, 17:45

View original

5 replies

kailinatsafe
Rank
Userlevel 3
Badge +13

Hello @denisgiannelli​, thanks for posting! I suspect you need to use Visual C++ Build Tools to compile pycocotools, since there aren’t any pre-built packages available? Please let me know if I am understanding this correctly, or feel free to correct me!

  1. Have you followed all the instructions here? 
  2. If so, are you able to import the newly compiled library in another environment outside of FME?

This will let us know if it’s been installed correctly. Hope this helps, Kailin

denisgiannelli
Badge

Hello @denisgiannelli​, thanks for posting! I suspect you need to use Visual C++ Build Tools to compile pycocotools, since there aren’t any pre-built packages available? Please let me know if I am understanding this correctly, or feel free to correct me!

  1. Have you followed all the instructions here? 
  2. If so, are you able to import the newly compiled library in another environment outside of FME?

This will let us know if it’s been installed correctly. Hope this helps, Kailin

Hi @kailinatsafe​ , thank you for your response ! :)

  1. Yes, I followed all the instructions present on the readme file.

When I open the Developer Command Prompt for VS 2022 (the 2015 link that philferriere does not work, there is a setup package missing or damaged), I immediately check PATH to see if the VS Build Tools are present ("On Windows, you must have the Visual C++ 2015 build tools on your path"), and they are:

PATHI then follow the pip install recommendation with

"pip install git+https://github.com/philferriere/cocoapi.git#subdirectory=PythonAPI"

PIP INSTALL GITAt this point, it is interesting to notice that, after collecting, it does not "Building wheels for collected packages: pycocotools", just as in the screenshot of philferriere. The program ends in "Preparing metadata (setup.py) ... done"

 

Alternatively, if I try to pip install it using FME's python intepreter:

" "C:\\Program Files\\FME2023\\fme.exe" python -m pip install git+https://github.com/philferriere/cocoapi.git#subdirectory=PythonAPI "

FME PYTHON PIPit complains about the lack of the module named "Cython". This is rare since Cython is available as a module inside the FME Python.

Cython 

Anyway, I am in a trial-and-test loop here...

Denis

kailinatsafe
Rank
Userlevel 3
Badge +13

Hi @kailinatsafe​ , thank you for your response ! :)

  1. Yes, I followed all the instructions present on the readme file.

When I open the Developer Command Prompt for VS 2022 (the 2015 link that philferriere does not work, there is a setup package missing or damaged), I immediately check PATH to see if the VS Build Tools are present ("On Windows, you must have the Visual C++ 2015 build tools on your path"), and they are:

PATHI then follow the pip install recommendation with

"pip install git+https://github.com/philferriere/cocoapi.git#subdirectory=PythonAPI"

PIP INSTALL GITAt this point, it is interesting to notice that, after collecting, it does not "Building wheels for collected packages: pycocotools", just as in the screenshot of philferriere. The program ends in "Preparing metadata (setup.py) ... done"

 

Alternatively, if I try to pip install it using FME's python intepreter:

" "C:\\Program Files\\FME2023\\fme.exe" python -m pip install git+https://github.com/philferriere/cocoapi.git#subdirectory=PythonAPI "

FME PYTHON PIPit complains about the lack of the module named "Cython". This is rare since Cython is available as a module inside the FME Python.

Cython 

Anyway, I am in a trial-and-test loop here...

Denis

Hey @denisgiannelli​, I did a quick test and I don't think Cython is included with FME python natively, you will need to install it. Kailin.

kailinatsafe
Rank
Userlevel 3
Badge +13

Hi @kailinatsafe​ , thank you for your response ! :)

  1. Yes, I followed all the instructions present on the readme file.

When I open the Developer Command Prompt for VS 2022 (the 2015 link that philferriere does not work, there is a setup package missing or damaged), I immediately check PATH to see if the VS Build Tools are present ("On Windows, you must have the Visual C++ 2015 build tools on your path"), and they are:

PATHI then follow the pip install recommendation with

"pip install git+https://github.com/philferriere/cocoapi.git#subdirectory=PythonAPI"

PIP INSTALL GITAt this point, it is interesting to notice that, after collecting, it does not "Building wheels for collected packages: pycocotools", just as in the screenshot of philferriere. The program ends in "Preparing metadata (setup.py) ... done"

 

Alternatively, if I try to pip install it using FME's python intepreter:

" "C:\\Program Files\\FME2023\\fme.exe" python -m pip install git+https://github.com/philferriere/cocoapi.git#subdirectory=PythonAPI "

FME PYTHON PIPit complains about the lack of the module named "Cython". This is rare since Cython is available as a module inside the FME Python.

Cython 

Anyway, I am in a trial-and-test loop here...

Denis

Hello @denisgiannelli​, this is not something we would usually do, but I ended up raising this internally and had a hand putting together the following instructions for installing pycoco tools!

 

1) Install a Python interpreter that matches the shipped FME Python version and bit architecture (64-bit Python 3.11.3 for FME23)

 

2) Open the x64 Native Tools Command Prompt for VS 2022 (64-bit to match with the FME and Python)

Step2_pycoco 

3) Make sure the PATH environment variable contains the path to your Python home and Python home\\script

Step3_pycoco 

4) Run the command from https://github.com/philferriere/cocoapi#this-clones-readme

Step4_pycoco 

5) Should be installed

Step5_pycoco 

I hope these steps work for you! Happy to help, Kailin.

denisgiannelli
Badge

Hi @kailinatsafe​ , thank you for your response ! :)

  1. Yes, I followed all the instructions present on the readme file.

When I open the Developer Command Prompt for VS 2022 (the 2015 link that philferriere does not work, there is a setup package missing or damaged), I immediately check PATH to see if the VS Build Tools are present ("On Windows, you must have the Visual C++ 2015 build tools on your path"), and they are:

PATHI then follow the pip install recommendation with

"pip install git+https://github.com/philferriere/cocoapi.git#subdirectory=PythonAPI"

PIP INSTALL GITAt this point, it is interesting to notice that, after collecting, it does not "Building wheels for collected packages: pycocotools", just as in the screenshot of philferriere. The program ends in "Preparing metadata (setup.py) ... done"

 

Alternatively, if I try to pip install it using FME's python intepreter:

" "C:\\Program Files\\FME2023\\fme.exe" python -m pip install git+https://github.com/philferriere/cocoapi.git#subdirectory=PythonAPI "

FME PYTHON PIPit complains about the lack of the module named "Cython". This is rare since Cython is available as a module inside the FME Python.

Cython 

Anyway, I am in a trial-and-test loop here...

Denis

@kailinatsafe​ , OMG thank you very much, it worked !! :D

In addition to your instructions, I also had to do the following:

  • Between your steps number 3 and 4, I had to "pip install cython" and "pip install numpy" (since the python interpreter that I had installed in step 1 was new)
  • Right before importing the recently installed pycocotools in FME's Python Caller, i had to insert a path to reference the other python interpreter (see image below)

PythonCallerIt works fine like this, so I have no problem to continue my work as an FME author.

Nevertheless, I was curious to understand why I could not import pycocotools directly in the python caller, i.e. just like I do with numpy and cython (see modules imported before the sys.path.insert).

I tried to pip install pycocotools on FME's python, just like I had done with the external python:

  • "C:\\Program Files\\FME2023\\fme.exe" python -m pip install git+https://github.com/philferriere/cocoapi.git#subdirectory=PythonAPI"
  • It complains because there would be no module Cython loaded:  ModuleNotFoundError: No module named 'Cython'
  • If I try to "C:\\Program Files\\FME2023\\fme.exe" python -m pip install cython", i get a message saying that cython already exists as a module inside the FME's python.
  • I double check this by "C:\\Program Files\\FME2023\\fme.exe" python", help, help(), modules.
  • In the list of modules loaded inside FME's python, I see both Cython (capital C) and cython (lowercase c), but of course no pycocotools.
  • In the list of modules loaded inside the external python, I see Cython, cython and then pycocotools, in this case because I managed to install it correctly

Anyway, I am just reporting all this because it might be helpful to other users. But I can already proceed with my authoring work!

Once more: thank you very, very much!

Greetings,

Denis Giannelli

Reply


Community Stats

28,670
Posts
109,366
Replies
36,531
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.