GoogleBigQueryConnector Failed to get authorization header

Hi ​@fmejohn93 

I haven't encountered this exact error before, but I'm happy to help! 🙂 The 'Failed to get authorization header' error usually means there's an authentication issue.

To better understand what's happening, could you share:

1. What FME version you're using
2. What error you see when you test the web connection (Web Connections > Managing Web Connections)

In the meantime, here are a few quick things to try:

In FME:

• Make sure your Client ID and Secret don't have extra spaces when you copy/paste them
• Try changing the HTTP User Agent from "FME Defaults" to "System Defaults" in the web service settings

In Google Console:

• Check that you've added the right scope under the Data Access tab
  • I believe these are the minimum requirements: BigQuery Scopes
• If you made scope changes, update your Client ID and Secret in FME

Let me know what you find! If these don't work, we can dig deeper with debug translation logs either here or by submitting a support case. 

Hi ​@desiree_at_safe,

Thanks for your response. I’m using FME Workbench 2024.2.3 and using the package Google BigQuery v1.3.3.

I don’t appear able to test the web connection that I created; I had created it by hitting the Sign in with Google option in the screenshot.

I’m not sure where I would enter the client ID or Secret on FME.

​@fmejohn93 

Thank you for sharing that information and a screenshot! Did this work for you previously and is now not working?

Besides that, I do see you're using the FME package of BigQuery (safe.google-bigquery), which we do recommend! You can test this web service connection in FME by going to:
Tools > FME Options > Web Connections > Manage Web Services

Under the Manage Web Services tab, find your Google BigQuery (safe.google-bigquery) web service definition, then scroll down and click Sign in with Google to test the connection as shown here:

This should give us more clues to why authentication is failing.

Note: the Manage Web Services dialog is also where you can supply your own Client ID and Secret. This article has a bit more details on this:

•  Creating an OAuth 2.0 Web Service and Connection in FME

Let me know what you find! I'm curious about any error messages or at what stage the authentication fails (during login or after).

​@desiree_at_safe I’m able to get a successful test result on the Google BigQuery(safe.google-bigquery) Web Service from the Manage Web Services tab. I did have to change the HTTP user agent to System Default for this to work

But when I create a new web connection and assign it to that web service, I get an ‘An unknown error has occurred’ error message when trying to select a project using that web connection.

I have not had this working previously, first time trying to implement this

Thanks

​@fmejohn93 That’s so odd! The only thing I can tell is that the Account looks a bit different than what I expected. 

My web connection still had the package name (safe.google-bigquery) attached to it like I have here:

So I’d recommend double checking the Web Connection Account, to be sure it matches the one you connected to successfully from the Manage Web Services dialog.

You might need to click Add Web Connection.. to ensure you have the right one or to create a new connection.

Let me know how it goes! 

​@desiree_at_safe I tried creating a new web connection similar to the one you have above and get the same error.

I also tried creating a User Parameter for the Project ID so that I can manually enter the ID when running the workspace. Doing that gave me the following error when running the workspace:

Python Exception : HTTPSConnectionPool(host='bigquery.googleapis.com', port=443): Max retries exceeded with url: /bigquery/v2/projects/my_project/jobs?prettyPrint=false (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate in certificate chain (_ssl.c:1000)')))

Note that I replaced the actual name of my project with ‘my_project’ in the above url

Thanks

​@desiree_at_safe 

I attached the log file from the run. I don’t believe we have any restrictions on FME but I will check to make sure

Thanks

​@desiree_at_safe It was a firewall issue on our side. Thanks for your help on this

Hi ​@fmejohn93

Happy you were able to get it working! Thank you for sharing that log file and the solution. We had difficulty reproducing your issue, so the firewall cause makes perfect sense!

The SSL error you got was key, but without that experimentation you did, it's possible we wouldn't have seen it. We’re discussing if it’s possible to make the error message more telling for cases like this.

Thanks again for the detailed troubleshooting. It really helps us improve the product!