Hi @osmanbeg1980 When does the error dialogue pop-up for you? Does this occur when you're trying to select the Project, Dataset, or Table?
Our Getting Started with Google BigQuery article may also have some helpful information for troubleshooting this issue.
Hi @danminneyatsaf thank you so much for your reply. The error always occurs when trying to set the Dataset in the BigQuery Writer. I've managed to authenticate my project in FME but as soon as I try to select a dataset it brings up that error message. I've tried updating everything I could but still not getting anywhere. I definitively have Datasets within my Google BigQuery Platform. This was working back in August 2022, but all of a sudden stopped working in September 2022 when Google decided to change they authentification process on their platform. I'll have a read through your guidance and see if I can get any further. Thank you again.
Went through the "Getting Started with Google BigQuery" documentation and unfortunately there is nothing there that helps resolve my issue. I keep getting stopped when trying to add project in the BigQuery writer (apologies I said dataset before - my mistake!). The same error message keeps coming up no matter how many times I authenticate the web connection to Google BigQuery. Unsure where to go from here - I think the problem is FME as I've tested it on both my personal GCP and also my work GCP and it fails on both. Perhaps I should uninstall FME Desktop entirely and try re-installing it again? I don't know - nothing else has worked thus so far. I've been using FME to push data into GCP (BigQuery) over the last 15 months and it's only recently (since Sep 2022) that it stopped working entirely.
Hi @osmanbeg1980 apologies for the late reply as I was out of the office last week.
We recently updated the Google BigQuery package (to version 1.1.1) to correctly allow for custom proxy URLs. Do you perhaps have a proxy you're using that was not specified in a newly downloaded version of FME? If you do have a proxy enabled you'll need to specify this in the FME Options settings.
Otherwise, I suggest downloading the latest version of FME 2022.2.1.0 and testing if the Google BigQuery Writer fails in that. If the error still occurs, could you please turn on Log Debug (FME Options > Translation > Log Message Filter > enabled Log Debug) and then try re-connecting to the Project you want to write to. When the error occurs again, click on the "Show Log..." button and then copy and paste the log into a text file and share here.
Hi @danminneyatsaf apologies for the lack of response. Was pulled away into another project for some time, but have now had time to revisit this and unfortunately after updating FME Desktop to version 2022.2.2 and ensuring that the Google BigQuery package is updated to version 1.1.1 it is still failing when I try to add a Google BigQuery writer and add the project. Below is the snippet from the 'Show Log' button as you mentioned above:
Creating reader for format: Google BigQuery
Trying to find a PYTHON plugin for reader named `SAFE.GOOGLE-BIGQUERY.GOOGLEBIGQUERY'
Loaded Python module `fmepy_google_bigquery' from file `C:\Users\osman\AppData\Roaming\Safe Software\FME\Packages\22782-win64\python\safe.google-bigquery\fmepy_google_bigquery\__init__.py'
Creating reader for format: Google BigQuery
Trying to find a PYTHON plugin for reader named `SAFE.GOOGLE-BIGQUERY.GOOGLEBIGQUERY'
Loaded Python module `fmepy_google_bigquery' from file `C:\Users\osman\AppData\Roaming\Safe Software\FME\Packages\22782-win64\python\safe.google-bigquery\fmepy_google_bigquery\__init__.py'
Updating access token for 'osman Google BigQuery (safe.google-bigquery)'
Request to update access token failed. HTTP Error: HTTP/1.1 400 Bad Request
Python Exception <FMEWSException>: Type: 3, Number: 1, Message: Failed to get authorization header.
SAFE.GOOGLE-BIGQUERY.GOOGLEBIGQUERY reader: An error has occurred. Check the logfile above for details
Failed to obtain any schemas from reader 'SAFE.GOOGLE-BIGQUERY.GOOGLEBIGQUERY' from 1 datasets. This may be due to invalid datasets or format accessibility issues due to licensing, dependencies, or module loading. See logfile for more information
UniversalReader -- readSchema resulted in 0 schema features being returned
Reader Parameter(0) = ><Unused><
Reader Parameter(1) = >GBQ_CONNECTION<
Reader Parameter(1) = >osman<space>Google<space>BigQuery<space><openparen>safe.google-bigquery<closeparen><
Reader Parameter(3) = >RETRIEVE_PROJECTS<
Reader Parameter(3) = >YES<
Reader Parameter(5) = >GBQ_CREDENTIAL_SOURCE<
Reader Parameter(5) = >WEBCONN<
Reader Parameter(7) = >_MERGE_SCHEMAS<
Reader Parameter(7) = >YES<
Reader Parameter(9) = >+FME_DEBUG<
Reader Parameter(9) = >DUMP_SCHEMA<
Reader Parameter(11) = >+FME_DEBUG<
Reader Parameter(11) = >BADNEWS<
Reader Parameter(13) = >+FME_DEBUG<
Reader Parameter(13) = >HTTP_DEBUG<
Reader Parameter(15) = >+FME_DEBUG<
Reader Parameter(15) = >FTP_DEBUG<
Reader Directive(0) = >RUNTIME_MACROS<
Reader Directive(1) = >GBQ_CONNECTION,osman<space>Google<space>BigQuery<space><openparen>safe.google-bigquery<closeparen>,RETRIEVE_PROJECTS,YES,GBQ_CREDENTIAL_SOURCE,WEBCONN,_MERGE_SCHEMAS,YES<
UniversalReader -- readSchema resulted in 0 schema features being returned
Failed to obtain any schemas from reader '%0' from %1 datasets. This may be due to invalid datasets or format accessibility issues due to licensing, dependencies, or module loading. See logfile for more information
As I said previously, this worked back in Aug/Sep 2022, however since keep getting this error message, every time I add the Google BigQuery Writer to the workbench and try to add a project, and unsure what to make of it. If you need more input from me, please do not hesitate to ask.
