Skip to main content
Solved

Can I read data from Socrata in FME?


fmelizard
Contributor
Forum|alt.badge.img+17

I see there's no Socrata Reader in FME, but is there some way that I can read data from my Socrata Open Data Portal?

Best answer by laurawatsafe

Yes! There are a few different ways to access data on a Socrata Open Data Portal with FME depending on how you want to access your data.

If you want to get access to all the data in a dataset at once, open the dataset in a web browser, and locate the Export option. Then under Download As, locate the format you want to download the data as (FME supports CSV, JSON, RSS, and XML), right-click and choose copy link address. Then add the appropriate writer into your workspace and set the source path to the link you copied.

Note: if the dataset you want to read from is Private, you can check the option to Use Network Authentication in the Parameters for any of those readers and enter your username/password to access the dataset.

If you want access to just a subset of the dataset (by applying a query or filter when reading), you can make use of the SODA API for the dataset to query the data through HTTPS. To find the end-point for that, open the Socrata dataset in a web browser and look for Export > SODA API. You can then interact with the API in FME using the HTTPCaller transformer.

There are also Python and R packages for interacting with the SODA API, so you could use the PythonCaller or RCaller to make queries too.

View original
Did this help you find an answer to your question?

2 replies

danilo_fme
Evangelist
Forum|alt.badge.img+41
  • Evangelist
  • October 3, 2017

Hi @FMELizard,

Has a interesting question here using the transformer HTTPCaller = .Read Data Socrata

Thanks,

Danilo


laurawatsafe
Safer
Forum|alt.badge.img+11
  • Safer
  • Best Answer
  • October 3, 2017

Yes! There are a few different ways to access data on a Socrata Open Data Portal with FME depending on how you want to access your data.

If you want to get access to all the data in a dataset at once, open the dataset in a web browser, and locate the Export option. Then under Download As, locate the format you want to download the data as (FME supports CSV, JSON, RSS, and XML), right-click and choose copy link address. Then add the appropriate writer into your workspace and set the source path to the link you copied.

Note: if the dataset you want to read from is Private, you can check the option to Use Network Authentication in the Parameters for any of those readers and enter your username/password to access the dataset.

If you want access to just a subset of the dataset (by applying a query or filter when reading), you can make use of the SODA API for the dataset to query the data through HTTPS. To find the end-point for that, open the Socrata dataset in a web browser and look for Export > SODA API. You can then interact with the API in FME using the HTTPCaller transformer.

There are also Python and R packages for interacting with the SODA API, so you could use the PythonCaller or RCaller to make queries too.


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings