Skip to main content
  • EN

I am trying to connect to S3 on FME Server using the S3Connector System authentication.

 

Our FME server is using the 6th method as per https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-quickstart.html#cli-configure-quickstart-precedence :

 

Instance profile credentials – You can associate an IAM role with each of your Amazon Elastic Compute Cloud (Amazon EC2) instances. Temporary credentials for that role are then available to code running in the instance. The credentials are delivered through the Amazon EC2 metadata service. For more information, see IAM Roles for Amazon EC2 in the Amazon EC2 User Guide for Linux Instances and Using Instance Profiles in the IAM User Guide. 

I have confirmed I have access on the server to reach the S3 bucket specified directly via the CLI, but when I run through FME Server I get the following:

Attribute(encoded: UTF-8) : `fme_rejection_message' has value `S3Connector: Required Amazon S3 credentials were blank or could not be found. Please note that attribute values can not be used for credentials when browsing S3 interactively. If using system credentials, the precedence for credential sources for FME is the same as for the AWS CLI, and is explained at https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html#config-settings-and-precedence'

I have confirmed I can add some temporary credentials into ~/.aws/credentials and this works as expected. Unfortunately this won't work as permanent solution as I need to use the Instance profile credentials.

 

 

@michaelpascoe @deanhowell

Hi @luke.giles,

I just tested this on an ec2 machine running Windows Server 2019 with FME Workbench (b20596) with a very simple read-only IAM role and it works as expected.

Can you confirm that you are using to most recent S3Connector (https://hub.safe.com/publishers/safe/packages/s3connector)?

Another good test is to use the AWS CLI directly to access the bucket on the same machine. Our transformer is using the same core libraries so if it works with the AWS CLI it should also work in FME.

Just for reference, these are the permissions I used in the IAM Role:

{
    "Version": "2012-10-17",
    "Statement": Â
        {
            "Effect": "Allow",
            "Action": Â
                "s3:Get*",
                "s3:List*"
            ],
            "Resource": "*"
        }
    ]
}

I hope this helps!

Hi @luke.giles,

I just tested this on an ec2 machine running Windows Server 2019 with FME Workbench (b20596) with a very simple read-only IAM role and it works as expected.

Can you confirm that you are using to most recent S3Connector (https://hub.safe.com/publishers/safe/packages/s3connector)?

Another good test is to use the AWS CLI directly to access the bucket on the same machine. Our transformer is using the same core libraries so if it works with the AWS CLI it should also work in FME.

Just for reference, these are the permissions I used in the IAM Role:

{
    "Version": "2012-10-17",
    "Statement": Â
        {
            "Effect": "Allow",
            "Action": Â
                "s3:Get*",
                "s3:List*"
            ],
            "Resource": "*"
        }
    ]
}

I hope this helps!

Thanks @gerhardatsafe

 

 

Yes, I can definitely access the bucket and download the required files via the AWS CLI on the server and have confirmed I have S3:Get* and S3:List* in my IAM Role, still with no luck.

 

One difference is we are running FME on a linux server, not windows running FME Server 2020.0.1 (build 20218) using safe.s3connector-1.0.9.fpkg

Thanks @gerhardatsafe

 

 

Yes, I can definitely access the bucket and download the required files via the AWS CLI on the server and have confirmed I have S3:Get* and S3:List* in my IAM Role, still with no luck.

 

One difference is we are running FME on a linux server, not windows running FME Server 2020.0.1 (build 20218) using safe.s3connector-1.0.9.fpkg

I just tested this on an Ubuntu 18.04 instance with FME Server installed and it still works as expected (b20596 & safe.s3connector-1.0.10.fpkg). So I don't think it is Linux related.

 

 

The only other thing I could think of is whether the user that runs the FME Server Engine service (in case you run it as systemd service or use a different user that is not root) can also retrieve the credentials from the ec2 metadata and assume the role. Not sure if that makes a difference but you can easily check by running your CLI command with the user that is running FME Server.

 

 

If that still doesn't work I would recommend getting in touch with our technical support and file a report: https://www.safe.com/support/

Thanks

Reply


Community Stats

30,364
Posts
115,373
Replies
38,142
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.