Has anyone figured out how to read Overture maps data using FME yet? Looks like it should accept a spatial query but maybe it requires a custom reader. I tried using the AmazonAthena connector but didn't get anywhere...
Overture documentation can be found here:
https://github.com/OvertureMaps/data/blob/main/README.md#how-to-access-overture-maps-data
Thanks in advance!
Nic
Hi @nic_ran,
The Overture map consists of Parquet files, which can be read by FME's Parquet reader directly from the S3 repository, using URLs. The trick is to find out which files to read. You can use the S3Connector transformer to get a list of the URLs, but that is only really useful if you want to read all the data, since the file names don't seem to hold any tile information.
The Athena method seems useful, since it supports spatial queries, but you would have to set up the CREATE EXTERNAL TABLE setup first. You could submit the queries through the AmazonAthenaConnector, then download the results using the S3Connector.
The Synapse method may also work. You would need to set up the Synapse Workspace as per the instructions first. FME's Synapse reader doesn't read from Queries, but you could use the Advanced - SQL parameters to create a temporary table using from a SQL query, then drop it after reading.
Hi @nic_ran,
The Overture map consists of Parquet files, which can be read by FME's Parquet reader directly from the S3 repository, using URLs. The trick is to find out which files to read. You can use the S3Connector transformer to get a list of the URLs, but that is only really useful if you want to read all the data, since the file names don't seem to hold any tile information.
The Athena method seems useful, since it supports spatial queries, but you would have to set up the CREATE EXTERNAL TABLE setup first. You could submit the queries through the AmazonAthenaConnector, then download the results using the S3Connector.
The Synapse method may also work. You would need to set up the Synapse Workspace as per the instructions first. FME's Synapse reader doesn't read from Queries, but you could use the Advanced - SQL parameters to create a temporary table using from a SQL query, then drop it after reading.
Hi
sorry but I'm not so expert I tried to use the FME's Parquet reader for Transportation
s3://overturemaps-us-west-2/release/2023-12-14-alpha.0/theme=transportation
but no result
can you elaborate a little bit the workflow ?
thx
Francesco