I’m new to FME tool and I wanted to pull data from Google BigQuery tables and convert into GeoDatabase and shapefile. I'm using FME Workbench version 2024.1.1 and I'm experiencing an issue with using Google BigQuery as a reader. But I’ve installed Google BigQuery as a transformer and attempted to pull data, but no data is being retrieved. I’ve also authenticated using a service account key file, but the data still isn’t showing up.
Page 1 / 1
This is my log when I ran the transformer:
Feature Caching is ON
The workspace may run slower because features are being output and recorded on all output ports regardless of whether or not there is a connection.
Command line to run this workspace:
"C:\Program Files\FME\fme.exe" C:\Users\sivavishnu123\Downloads\gcpbigquery_sample.fmw
--DestDataset_GEOJSON "C:\GCP\bigquery.geojson"
--DestDataset_CSV2 "C:\GCP"
--FME_LAUNCH_VIEWER_APP "YES"
Starting translation...
Current FME version: FME 2024.1.1.0 (20240729 - Build 24619 - WIN64)
System Time: 20241111103149 UTC
FME_HOME is 'C:\Program Files\FME\'
FME Form (floating)
Permanent License.
Machine host name is: vishnuiam
OS Locale Name : en_US
OS Locale Encoding : windows-1252
Process Encoding : UTF-8
FME API version: '4.0 20240212'
FME Configuration: FME_BASE is 'no'
FME Configuration: FME_MF_DIR is 'C:\Users\vishnu123\Downloads/'
FME Configuration: FME_MF_NAME is 'gcpbigquery_sample.fmw'
FME Configuration: FME_PRODUCT_NAME is 'FME(R) 2024.1.1.0'
Operating System: Microsoft Windows Server 2019 64-bit (Build 17763)
FME Platform: WIN64
System Status: 68.41 GB of disk space available in the FME temporary folder (C:\Users\vishnu123\AppData\Local\Temp\11\wbrun_1731320966892_13804\fmetmp_0)
System Status: 16.00 GB of physical memory available
System Status: 24.24 GB of virtual memory available
START - ProcessID: 11712, peak process memory usage: 41724 kB, current process memory usage: 41724 kB
FME Configuration: Command line arguments are `C:\Program Files\FME\fme.exe' `C:\Users\sivavishnu123\Downloads\wb-xlate-1731321106797_13804' `LOG_STANDARDOUT' `YES' `LOG_TIMINGS' `YES' `LogCountServerName' `58029' `LOG_STANDARDOUT' `No' `WORKBENCH_LOG_SERVER' `57691'
FME Configuration: Connection Storage: 'C:\Users\sivavishnu123\AppData\Roaming\Safe Software\FME\'
Shared folders for formats are : C:\Program Files\FME\datasources;C:\Users\sivavishnu123\Documents\FME\Formats
Shared folders for transformers are : C:\Users\sivavishnu123\AppData\Roaming\Safe Software\FME\Packages\24619-win64\transformers;C:\Program Files\FME\transformers
Shared folders for coordinate systems are : C:\Users\sivavishnu123\Documents\FME\CoordinateSystems
Shared folders for coordinate system exceptions are : C:\Users\sivavishnu123\Documents\FME\CoordinateSystemExceptions
Shared folders for coordinate system grid overrides are : C:\Users\sivavishnu123\Documents\FME\CoordinateSystemGridOverrides
Shared folders for CS-MAP transformation exceptions are : C:\Users\sivavishnu123\Documents\FME\CsmapTransformationExceptions
Shared folders for transformer categories are : C:\Users\sivavishnu123\Documents\FME\TransformerCategories
FME Configuration: Reader Keyword is `MULTI_READER'
FME Configuration: Writer Keyword is `PRIMARY_DEST'
FME Configuration: Writer Group Definition Keyword is `PRIMARY_DEST_DEF'
FME Configuration: Reader type is `MULTI_READER'
FME Configuration: Writer type is `MULTI_WRITER'
FME Configuration: No destination coordinate system set
FME Configuration: Current working folder is `C:\Users\sivavishnu123\Downloads'
FME Configuration: Temporary folder is `C:\Users\vishnu123\AppData\Local\Temp\11\wbrun_1731320966892_13804\fmetmp_0', set from environment variable `FME_TEMP'
FME Configuration: Cache folder is 'C:\Users\vishnu123\AppData\Local\Temp\11'
FME Configuration: FME_HOME is `C:\Program Files\FME\'
FME Configuration: Start freeing memory when the process exceeds 48.00 GB
FME Configuration: Stop freeing memory when the process is below 36.00 GB
Creating writer for format:
Creating reader for format:
MULTI_READER(MULTI_READER): Will fail with first member reader failure
Using Multi Reader with keyword `MULTI_READER' to read multiple datasets
Using MultiWriter with keyword `PRIMARY_DEST' to output data (ID_ATTRIBUTE is `multi_writer_id')
Writer output will be ordered by value of multi_writer_id
Emptying factory pipeline
Destination Feature Type Routing Correlator (RoutingFactory): Tested 0 input feature(s), wrote 0 output feature(s): 0 matched merge filters, 0 were routed to output, 0 could not be routed.
Final Output Nuker (TeeFactory): Cloned 0 input feature(s) into 0 output feature(s)
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Features Read Summary
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
==============================================================================
Total Features Read 0
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Features Written Summary
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
==============================================================================
Total Features Written 0
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-
-~ ~-
-~ Feature caches have been recorded at every stage of the translation. ~-
-~ To inspect the recorded features, ~-
-~ click the feature cache icons next to the ports. ~-
-~ ~-
-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-
Translation was SUCCESSFUL with 7 warning(s) (0 feature(s) output)
FME Session Duration: 0.2 seconds. (CPU: 0.0s user, 0.1s system)
END - ProcessID: 11712, peak process memory usage: 45508 kB, current process memory usage: 45508 kB
Translation was SUCCESSFUL
My issue has been resolved after submitting FME support team ticket.
For my issue I’ve to use creator transformer before GoogleBigQueryConnector to initiate transform action. GoogleBigQueryConnector itself wont start the action. Hence needed creator transformer to trigger the GBQC.