I see there's no Socrata Reader in FME, but is there some way that I can read data from my Socrata Open Data Portal?
Solved
Can I read data from Socrata in FME?
+20- Safer
- 3719 replies
Best answer by laurawatsafe
Yes! There are a few different ways to access data on a Socrata Open Data Portal with FME depending on how you want to access your data.
If you want to get access to all the data in a dataset at once, open the dataset in a web browser, and locate the Export option. Then under Download As, locate the format you want to download the data as (FME supports CSV, JSON, RSS, and XML), right-click and choose copy link address. Then add the appropriate writer into your workspace and set the source path to the link you copied.
Note: if the dataset you want to read from is Private, you can check the option to Use Network Authentication in the Parameters for any of those readers and enter your username/password to access the dataset.
If you want access to just a subset of the dataset (by applying a query or filter when reading), you can make use of the SODA API for the dataset to query the data through HTTPS. To find the end-point for that, open the Socrata dataset in a web browser and look for Export > SODA API. You can then interact with the API in FME using the HTTPCaller transformer.
There are also Python and R packages for interacting with the SODA API, so you could use the PythonCaller or RCaller to make queries too.
This post is closed to further activity.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.
+51- Celebrity
- 2077 replies
Hi @FMELizard,
Has a interesting question here using the transformer HTTPCaller = .Read Data Socrata
Thanks,
Danilo
+12- Safer
- 158 replies
- Best Answer
Yes! There are a few different ways to access data on a Socrata Open Data Portal with FME depending on how you want to access your data.
If you want to get access to all the data in a dataset at once, open the dataset in a web browser, and locate the Export option. Then under Download As, locate the format you want to download the data as (FME supports CSV, JSON, RSS, and XML), right-click and choose copy link address. Then add the appropriate writer into your workspace and set the source path to the link you copied.
Note: if the dataset you want to read from is Private, you can check the option to Use Network Authentication in the Parameters for any of those readers and enter your username/password to access the dataset.
If you want access to just a subset of the dataset (by applying a query or filter when reading), you can make use of the SODA API for the dataset to query the data through HTTPS. To find the end-point for that, open the Socrata dataset in a web browser and look for Export > SODA API. You can then interact with the API in FME using the HTTPCaller transformer.
There are also Python and R packages for interacting with the SODA API, so you could use the PythonCaller or RCaller to make queries too.