Question

General way of working with a FME Server deployment on Docker

  • 16 March 2023
  • 8 replies
  • 32 views
S

I'm very new to the concept of deploying things within a Docker environment. As far as I know, FME Server works perfectly fine with this but what about the interaction with FME Server after it´s up and running. Are you as a workspace author just uploading your workspaces like you do on a normal FME Server on a regular Windows plattform? Same with FME Projects?

My second question is if there is a way to use Docker and Jenkins to install FME Projects instead of the normal way through FME Server Gui import/export.

Regards

Stefan

8 replies

virtualcitymatt
Rank
Userlevel 4
Badge +26

Uploading workspace is the same, just publish from FME Desktop - a key difference is that FME Server in docker is running on linux which means there are several limitations/differences between windows (Just make sure you have the correct host name and port number).

 

I'm not exactly sure about what you mean about using Docker and Jenkins to install an FME Project, however, a REST call can be made to import a project to FME Server. I think you first need to upload the project to a resource and then import it. This can most likely be run is a Jenkins build script

S

Uploading workspace is the same, just publish from FME Desktop - a key difference is that FME Server in docker is running on linux which means there are several limitations/differences between windows (Just make sure you have the correct host name and port number).

 

I'm not exactly sure about what you mean about using Docker and Jenkins to install an FME Project, however, a REST call can be made to import a project to FME Server. I think you first need to upload the project to a resource and then import it. This can most likely be run is a Jenkins build script

Hi, thanks. Is there any other posibilities regarding how you work with deploying new code (ws, topics, automations etc)? I'm working with FME Procjects and we try do keep a test environment that is an excact copy of prod. We create procjets in test and importing on prod. Works good apart from some issues with user access rights. Can you manage that with some tools that is present in the Docker environment? As I said, have no Docker experience and my questions might have obvious answers.

Stefan

virtualcitymatt
Rank
Userlevel 4
Badge +26

Hi, thanks. Is there any other posibilities regarding how you work with deploying new code (ws, topics, automations etc)? I'm working with FME Procjects and we try do keep a test environment that is an excact copy of prod. We create procjets in test and importing on prod. Works good apart from some issues with user access rights. Can you manage that with some tools that is present in the Docker environment? As I said, have no Docker experience and my questions might have obvious answers.

Stefan

For us we have a build pipeline where FME Server projects are the product (we deliver these to our customers). For us there aren't really many changes to the project(s) with the exception of the workspace(s) themselves so we just build the projects and make sure they can be deployed and run before releasing them. So we don't really have a typical 'test' and 'production' environment.

 

If you want a test environment to match prod then a docker FME can be a pretty good approach. This is perfect for building and testing projects and then exporting them for use in prod. Again though the difference is that docker FME server is running in linux so formats like ESRI_FILEGEODATABASE just wont work. I think you just need to think of docker as a VM (it kind of does the same thing).

 

One of the benefits of docker though is working with a fresh clean environment with very little setup overhead. If you want keep an exact copy of your prod in test then you will need to have everything in test as well which means you kind of lose some of the benefit of using docker and if you need formats which are specific to windows then it's not a great option unfortunately.

 

Perhaps docker could be a good dev environment for development and prototyping. Export your project from dev and push it to test (to actually test) and then you can push it to prod. Once the process is done you can blow away your dev docker and get a fresh clean one for the next project.

 

 

 

 

S

Hi, thanks. Is there any other posibilities regarding how you work with deploying new code (ws, topics, automations etc)? I'm working with FME Procjects and we try do keep a test environment that is an excact copy of prod. We create procjets in test and importing on prod. Works good apart from some issues with user access rights. Can you manage that with some tools that is present in the Docker environment? As I said, have no Docker experience and my questions might have obvious answers.

Stefan

Thanks, good explanation. But as I understand in the Docker scenario, you still be importing your fme project to the already up and running prod Docker FME Server? There is nothing like replacing the entire prod Docker with the test Docker including the new fme stuff you want to deploy? I suppose not, sounds like overkill to me. I'm asking on behalf of a customer. Thanks again for your help.

Stefan

virtualcitymatt
Rank
Userlevel 4
Badge +26

Hi, thanks. Is there any other posibilities regarding how you work with deploying new code (ws, topics, automations etc)? I'm working with FME Procjects and we try do keep a test environment that is an excact copy of prod. We create procjets in test and importing on prod. Works good apart from some issues with user access rights. Can you manage that with some tools that is present in the Docker environment? As I said, have no Docker experience and my questions might have obvious answers.

Stefan

Right, you could of course automate the project exports/imports with scripts etc. You can also 'replace' the whole prod with test by using the backup/restore function instead of projects. I'm not quite sure that's what you're looking for though as that would mess with users.

 

S

Hi, thanks. Is there any other posibilities regarding how you work with deploying new code (ws, topics, automations etc)? I'm working with FME Procjects and we try do keep a test environment that is an excact copy of prod. We create procjets in test and importing on prod. Works good apart from some issues with user access rights. Can you manage that with some tools that is present in the Docker environment? As I said, have no Docker experience and my questions might have obvious answers.

Stefan

It´s a customer to our company that thinks fme projects often require some manual configurations when they deploy to prod from test. They are looking for a more automatic way to deploy new fme stuff. They are using Docker and Jenkins for other things than fme so their question was if there is a more automtaic way to handle new things in fme. But as more as we chat here I think it all comes down to how things are designed. Fme projects as a deploy method is fine i think but workspace design with parameters, users and all other things needs to support moving from test to prod without additional config.

Stefan

virtualcitymatt
Rank
Userlevel 4
Badge +26

Hi, thanks. Is there any other posibilities regarding how you work with deploying new code (ws, topics, automations etc)? I'm working with FME Procjects and we try do keep a test environment that is an excact copy of prod. We create procjets in test and importing on prod. Works good apart from some issues with user access rights. Can you manage that with some tools that is present in the Docker environment? As I said, have no Docker experience and my questions might have obvious answers.

Stefan

Ahhhh right yeah, that's going to be very different for each project probably, hard to automate that. I think our customers have similar issues when we release a new version of the project. For example the project has an email subscription with a template/recommended email text, many customers change this. The next time they import the project to get the latest updates then their subscription will get overwritten. It's the same with the parameter configuration of the workspace.

Ideally you'd be able to merge 'changes'. You might be able to create an extra project with 'changes' - called something like project-x-changes. This could include just the things which have been changed, it would mean that other parts wouldn't need to be reconfigured?

S

Hi, thanks. Is there any other posibilities regarding how you work with deploying new code (ws, topics, automations etc)? I'm working with FME Procjects and we try do keep a test environment that is an excact copy of prod. We create procjets in test and importing on prod. Works good apart from some issues with user access rights. Can you manage that with some tools that is present in the Docker environment? As I said, have no Docker experience and my questions might have obvious answers.

Stefan

Sounds like Docker itself doesn't do any diffrenece here and that was my initial question really. So you helped me alot to confirm that. It's all about how we design he workspaces and configuration. I've been working with customers where we have a quite a good dev, test and prod workflow where we keep manual configuration to a minimum with Fme projects and config parameters in database tables.

/Stefan

Reply


Community Stats

28,724
Posts
109,518
Replies
36,563
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.