Question

Remote Engine

  • 21 February 2018
  • 3 replies
  • 31 views
V
Badge

Hi,

We currently have a FME Server 2017.1.1 install on our cloud (AWS), we want to install an FME Engine inside a customer network to allow access protected resources like databases, files etc.

 

Reading about we need to connect the remote engine to:

1. FME DataBase: We have our FME database on Amazon RDS, so i can give access to the engine to allow connect to the database, this is fine.

2. FME Core: To allow connections to the FME Core, FME engine needs access FME Core using the ports 7069-7072 and 7100-7150 and FME Core needs access FME Engine using the ports 7500, I can add the proper permissions on the servcurity group, so this point is ok.

3. FME Server System Share: We are using Amazon EFS for FME System Share, external access to EFS is not allowed unless we implement AWS Direct Connect.

 

We do not want to connect our customer's network to our cloud network to connect to FME Server System Share, is this really necessary? we only want to send jobs to a remote engine, basically y the job download a file form amazon S3, parse the file, call some services, and put the result on a database, the services and databases are stored on the customer's network without outside access, this is the main reason to put and engine on a separate network.

There is a way to deploy a separate engine without access to the system Share?

Regards

3 replies

G
Badge

Hi @velasquezvictor

I think this is a very good used case for an external FME Engine. However, I am afraid that there is currently no way to run an external FME engine without access to the FME Server System Share. Although you are not planning to read and write from/to that location with your workspaces, the FME Engine still needs R/W permissions for the FME Server System Share for at least the following reasons:

  • read FME Server repositories
  • read/write temporary files during FME translations

I will make sure to double check with my team to see if there is a workaround and will update here accordingly.

Update & correction:

I double checked after our call and the license for the engine is provided by the core and not via a persistent connection to the FME Server System Share. Therefore it's not part of the reasons the instance needs access to the share. Also, the FME Server log files can be written to a folder on the on-premise machine following this article.

I still don't see a workaround to use AWS EFS as an FME Server System Share for an on-premise engine, besides AWS Direct connect though.

One idea that might work without AWS Direct connect could be to have the FME Server System Share on the default EBS volume on one of the machines you installed the FME Core or the internal FME Engine on. For example, you could create a Network Share on your AWS Machine running the Engine or the Core, open the ports needed for SMB traffic (I think it's 135 through 139 and 445 TCP & UDP) and create a user on the on-premise machine that has R/W access to only critically needed folders of the FME Server System Share (e.g. Repositories & Data). This user will need to run the FME Engine service on-premise. On AWS side you would also need to limit access via the SMB ports to a single IP (of the on-premise machine). Please be aware that this traffic won't be encrypted and that this is nothing that we are testing or can guarantee any support for. Maybe it helps and gives you some other ideas moving forward.

Also, make sure to stay up to date with FME Server for Docker which might help you solutions like that in the future.

Best regards,

Gerhard

V
Badge

Hi @velasquezvictor

I think this is a very good used case for an external FME Engine. However, I am afraid that there is currently no way to run an external FME engine without access to the FME Server System Share. Although you are not planning to read and write from/to that location with your workspaces, the FME Engine still needs R/W permissions for the FME Server System Share for at least the following reasons:

  • read FME Server repositories
  • read/write temporary files during FME translations

I will make sure to double check with my team to see if there is a workaround and will update here accordingly.

Update & correction:

I double checked after our call and the license for the engine is provided by the core and not via a persistent connection to the FME Server System Share. Therefore it's not part of the reasons the instance needs access to the share. Also, the FME Server log files can be written to a folder on the on-premise machine following this article.

I still don't see a workaround to use AWS EFS as an FME Server System Share for an on-premise engine, besides AWS Direct connect though.

One idea that might work without AWS Direct connect could be to have the FME Server System Share on the default EBS volume on one of the machines you installed the FME Core or the internal FME Engine on. For example, you could create a Network Share on your AWS Machine running the Engine or the Core, open the ports needed for SMB traffic (I think it's 135 through 139 and 445 TCP & UDP) and create a user on the on-premise machine that has R/W access to only critically needed folders of the FME Server System Share (e.g. Repositories & Data). This user will need to run the FME Engine service on-premise. On AWS side you would also need to limit access via the SMB ports to a single IP (of the on-premise machine). Please be aware that this traffic won't be encrypted and that this is nothing that we are testing or can guarantee any support for. Maybe it helps and gives you some other ideas moving forward.

Also, make sure to stay up to date with FME Server for Docker which might help you solutions like that in the future.

Best regards,

Gerhard

Hi @GerhardAtSafe

 

 

Thanks for your fast answer, please keep me update when you have an answer from your team, we need to check if this is possible or not (I hope yes).

 

Regards

 

G
Badge
Hi @GerhardAtSafe

 

 

Thanks for your fast answer, please keep me update when you have an answer from your team, we need to check if this is possible or not (I hope yes).

 

Regards

 

Hi @velasquezvictor

 

Unfortunately, there is no workaround for this at the moment. All engines need to have access to the FME Server System Share.

 

For now, I would recommend suggesting your requirements as an Idea for future improvements of FME Server. Sorry for inconvenience.

 

Best regards,

 

Gerhard

 

Reply


Community Stats

28,768
Posts
109,665
Replies
36,597
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.