Skip to main content
  • EN

We're interested in using FME Desktop for automated ETL processes on Kubernetes cloud infrastructure. We intend to use it on "Job" type because it allows us to use our clusters for the shortest possible time, taking only the resources it needs to complete the job, and nothing else. This is why our interest lies in FME desktop for parameterized ETL jobs much more than with FME Server, which requires costly dedicated services that run unused for most of the time.

However, we have some trouble Dockerized installs of FME Desktop on newer versions of Ubuntu.

 

This is all fine:

ARG UBUNTU_VERSION=16.04
FROM ubuntu:${UBUNTU_VERSION}
# After FROM all ARGs are wiped, so we need to set again
ARG UBUNTU_VERSION=16.04

# Installing required packages
RUN apt-get update && \
    apt-get -y install \
        curl

# Frontent must be disabled during install
ARG DEBIAN_FRONTEND=noninteractive
# I'm not using the docker ADD command to be able to use cache.
ARG FME_MAJOR=2019
ARG FME_MINOR=1.2.0.19630
RUN curl --fail --location --show-error \
    https://downloads.safe.com/fme/${FME_MAJOR}/fme-desktop-${FME_MAJOR}_${FME_MAJOR}.${FME_MINOR}~ubuntu.${UBUNTU_VERSION}_amd64.deb \
    --output /tmp/fme_install.deb && \
    apt-get install -y /tmp/fme_install.deb && \
    # cleaning up afterwards to reduce image size
    rm /tmp/fme_install.deb
ENV PATH="/opt/fme-desktop-${FME_MAJOR}/bin:${PATH}"
# Test for working runtime:
RUN fme --version

 

This is all fine. However, Ubuntu 16.04 is getting a little old, so we've switched to 18.04 for quite some time. Many packages that come from Canonical with 18.04 have to be installed from other sources (such as a recent Python version), or even compiled. But, if we try to install using Ubuntu 18.04 (using a recent FME Desktop version, there is no Ubuntu Xenial version of recent FME Desktop versions):

 

# Not working for 18.04
ARG UBUNTU_VERSION=18.04
FROM ubuntu:${UBUNTU_VERSION}
# After FROM all ARGs are wiped, so we need to set again
ARG UBUNTU_VERSION=18.04

# installing required packages, note libbson and libmongoc (!)
RUN apt-get update && \
    apt-get -y install \
        curl \
        libbson-1.0-0 \
        libmongoc-1.0-0

#Frontent must be disabled during install
ARG DEBIAN_FRONTEND=noninteractive
# I'm not using the docker ADD command to be able to use cache.
ARG FME_MAJOR=2020
ARG FME_MINOR=0.1.0.20218
RUN curl --fail --location --show-error \
    https://downloads.safe.com/fme/${FME_MAJOR}/fme-desktop-${FME_MAJOR}_${FME_MAJOR}.${FME_MINOR}~ubuntu.${UBUNTU_VERSION}_amd64.deb \
    --output /tmp/fme_install.deb && \
    apt-get install -y /tmp/fme_install.deb && \
    # cleaning up afterwards to reduce image size
    rm /tmp/fme_install.deb
ENV PATH="/opt/fme-desktop-${FME_MAJOR}/bin:${PATH}"
# Check for working runtime
RUN fme --version

 

Note that we have to specify installing libbson and libmongoc. Otherwise FME Desktop will fail runtime with:

fme: error while loading shared libraries: libbson-1.0.so.0: cannot open shared object file: No such file or directory

resp.

fme: error while loading shared libraries: libmongoc-1.0.so.0: cannot open shared object file: No such file or directory

It seems like some dependencies got mixed up in different versions of FME.

However, when we do install these dependencies (as shown in the Dockerfile example), FME Desktop fails runtime with:

fme: symbol lookup error: /opt/fme-desktop-2020/bin/../fmecore/././libmongocxx.so._noabi: undefined symbol: mongoc_transaction_opts_destroy

Can you please help us out with this to figure out the Docker recipe for FME Desktop on Ubuntu 18.04? Many thanks.

 

Looks like FME 2020 Depends on a Legacy driver:

http://mongocxx.org/legacy-v1/.

Looks like FME 2020 Depends on a Legacy driver:

http://mongocxx.org/legacy-v1/.

Indeed, it would seem like you would have to compile this from source. But I don't think this is intended by the FME Desktop dev team, right?

By the way, the Debian version installs without a hitch as well:

ARG DEBIAN_VERSION=10
FROM debian:${DEBIAN_VERSION}
# After FROM all ARGs are wiped, so we need to set again
ARG DEBIAN_VERSION=10

# installing required packages
RUN apt-get update && \
    apt-get -y install \
        curl

#Frontent must be disabled during install
ARG DEBIAN_FRONTEND=noninteractive
# I'm not using the docker ADD command to be able to use cache.
ARG FME_MAJOR=2020
ARG FME_MINOR=0.1.0.20218
RUN curl --fail --location --show-error \
    https://downloads.safe.com/fme/${FME_MAJOR}/fme-desktop-${FME_MAJOR}_${FME_MAJOR}.${FME_MINOR}~debian.${DEBIAN_VERSION}_amd64.deb \
    --output /tmp/fme_install.deb && \
    apt-get install -y /tmp/fme_install.deb && \
    # cleaning up afterwards to reduce image size
    rm /tmp/fme_install.deb
# adding fme executable to path
ENV PATH="/opt/fme-desktop-${FME_MAJOR}/bin:${PATH}"

# Check and print version (doesn't do anything other than testing if the installation succeeded)
RUN fme --version

How are you licensing FME Desktop?

How are you licensing FME Desktop?

Hi @darkspatiallord I appreciate the question but it's not relevant for this issue. `fme --version` should fail in this example with a license error. Why you would need a license to check the fme version is beyond me, but in the Ubuntu 18.04 case it doesn't reach that point.

 

 

We echo simple settings to /opt/fme-desktop-${FME_MAJOR}/licenses/fme*.dat for FME Desktop to use a license server, where we use floating licenses.

Could it be because you are calling fme from the 

bin
 directory? Try changing this:
ENV PATH="/opt/fme-desktop-${FME_MAJOR}/bin:${PATH}"
# Check for working runtime
RUN fme --version

to this:

ENV PATH="/opt/fme-desktop-${FME_MAJOR}:${PATH}"
# Check for working runtime
RUN fme --version

It should then load without licensing errors.

 

 

This looks like an interesting workflow you have planned. Rather than creating your own image, you could use the FME Server engine container as your starting point and run that as a job type. If you did still want to create your own image, then you could use the FME Server container as the base image and change the command of the container to be /opt/fme-engine-2020/fme.

 

 

If we understand your requirements a little more maybe we could help you create a helm chart.

Could it be because you are calling fme from the 

bin
 directory? Try changing this:
ENV PATH="/opt/fme-desktop-${FME_MAJOR}/bin:${PATH}"
# Check for working runtime
RUN fme --version

to this:

ENV PATH="/opt/fme-desktop-${FME_MAJOR}:${PATH}"
# Check for working runtime
RUN fme --version

It should then load without licensing errors.

 

 

This looks like an interesting workflow you have planned. Rather than creating your own image, you could use the FME Server engine container as your starting point and run that as a job type. If you did still want to create your own image, then you could use the FME Server container as the base image and change the command of the container to be /opt/fme-engine-2020/fme.

 

 

If we understand your requirements a little more maybe we could help you create a helm chart.

Hey @stewartatsafe thanks that solved it! For the records & future reference: this problem was not a licensing issue - if the install reaches that stage then we're in business. But the FME Desktop install on Ubuntu 18.04 didn't get there - instead it failed on a dependency problem. 

The trouble with FME Server engine containers is that it requires a costly accompanying FME Server deployment, am I right? This consigns us to purchasing license(s) for FME Server at a much higher starting cost than of a Desktop license, and we don't intend to go there, at least not for the foreseeable future.

 

For future reference, this is the ticket:

 

ARG UBUNTU_VERSION=18.04
FROM ubuntu:${UBUNTU_VERSION}
# After FROM all ARGs are wiped, so we need to set again
ARG UBUNTU_VERSION=18.04

# installing required packages
RUN apt-get update && \
    apt-get -y install \
        curl

#Frontent must be disabled during install
ARG DEBIAN_FRONTEND=noninteractive
# I'm not using the docker ADD command to be able to use cache.
ARG FME_MAJOR=2020
ARG FME_MINOR=0.1.0.20218
RUN curl --fail --location --show-error \
    https://downloads.safe.com/fme/${FME_MAJOR}/fme-desktop-${FME_MAJOR}_${FME_MAJOR}.${FME_MINOR}~ubuntu.${UBUNTU_VERSION}_amd64.deb \
    --output /tmp/fme_install.deb && \
    apt-get install -y /tmp/fme_install.deb && \
    # cleaning up afterwards to reduce image size
    rm /tmp/fme_install.deb
ENV PATH="/opt/fme-desktop-${FME_MAJOR}:${PATH}"

Hey @stewartatsafe thanks that solved it! For the records & future reference: this problem was not a licensing issue - if the install reaches that stage then we're in business. But the FME Desktop install on Ubuntu 18.04 didn't get there - instead it failed on a dependency problem.

The trouble with FME Server engine containers is that it requires a costly accompanying FME Server deployment, am I right? This consigns us to purchasing license(s) for FME Server at a much higher starting cost than of a Desktop license, and we don't intend to go there, at least not for the foreseeable future.

 

Hi, @rein I will reach out to you via email to discuss licensing, there are options.

Reply


Community Stats

30,092
Posts
114,244
Replies
37,717
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.