Question

AmazonAthenaConnector 1.0 - Data Type Issues

  • 2 November 2021
  • 1 reply
  • 1 view

Badge +1

We have been using AthenaConnector for awhile and have about 75 mappings developed. Last summer while using FME 2019, the AthenConnector automatically updated to version 1.0 of the connector.  A lot of mappings started bombing. Also see FME Hub Comments.

 

Here is one such error message:

2021-11-02 11:18:02|  3.6| 0.0|INFORM|AmazonAthenaConnector: Query ran successfully. Run time: 6.417 seconds. Data scanned: 640595.24 kB
 
2021-11-02 11:18:02|  3.6| 0.0|INFORM|AmazonAthenaConnector: Query results saved to: 's3:/...'
 
2021-11-02 11:18:03|  3.8| 0.2|WARN |Python Exception <ValueError>: invalid literal for int() with base 10: 'client_num'
 
2021-11-02 11:18:03|  3.8| 0.0|WARN |Traceback (most recent call last):
 
 File "plugins.py", line 447, in input
 
 File "...\AppData\Roaming\Safe Software\FME\Packages\19822-win64\python\safe.amazon-athena\fmepy_amazon_athena\connector.py", line 168, in receive_feature
 
  first_feature_row = next(result, None)
 
 File "...\AppData\Roaming\Safe Software\FME\Packages\19822-win64\python\safe.amazon-athena\fmepy_amazon_athena\api_athena.py", line 329, in run
 
  for row in self.get_query_results(query_execution_id):
 
 File "...\AppData\Roaming\Safe Software\FME\Packages\19822-win64\python\safe.amazon-athena\fmepy_amazon_athena\api_athena.py", line 222, in get_query_results
 
  for row in self._process_query_result(rows, column_info):
 
 File "...\AppData\Roaming\Safe Software\FME\Packages\19822-win64\python\safe.amazon-athena\fmepy_amazon_athena\api_athena.py", line 238, in _process_query_result
 
  row[col["Name"]] = ATHENA_TYPE_MAP.get(col["Type"], str)(data["VarCharValue"])
 
ValueError: invalid literal for int() with base 10: 'client_num'

This is a real pain, because every time we open one of these mappings it automatically updates to 1.0, then we have to reinstall the old version.

 

Any idea how to resolve this?  


1 reply

Userlevel 2
Badge +10

Hi @jbradfor​, thanks for bringing this to our attention. I will bring this issue up with my the team.

 

The change between 0.2.8 and 1.0.0 added an option to skip processing the first row of results, which usually contains column names. The default for Result has Column Names should be 'Yes' (therefore being backwards compatible), so if the connector is set to 'No'. then those column errors are being caused by treating the columns themselves as data.

 

I also have a couple questions that will help aid us in troubleshooting this issue:

  • Which version of the Connector you are currently running?
  • Which version of FME are you using?
  • Which version of Python are you using?

Reply