Question

FME to Socrata job not working with Socrata Covid Map

  • 1 May 2020
  • 1 reply
  • 6 views

EDIT: After doing steps below I now am able to update the LOCATION field but no new records are being being added when I run the FME job. Any ideas?

I'm attempting to automate reading an Oracle database and writing to a Socrata dataset using FME workbench 2018. Everything was working except that every time I ran the job I had to manually set the lat/long columns in Socrata to get the map working.

I googled the issue and found this documentation: https://support.socrata.com/hc/en-us/articles/207040527-Working-with-Point-Data-Using-the-FME-Socrata-Writer

Then I did the following:

I used attribute creator to create a new field called LOCATION that concatenates the latitude and longitude columns so that the values look like “POINT (-87.61694 41.70737)”. This location field is already setup in Socrata as a point geocoded column.

I have attached a screenshot of my workbench and the new Location field below.

When I run the workbench it gives me an error, and although the LOCATION field is populating and map works, the dataset sitting at 2958 rather that 2970. So it's not grabbing the new rows. Any ideas?

 

 

 

relevant log info:

 

Polling the job status: Counting records: found 1000 so far in TEMP.csv

Polling the job status: FAILURE: Processing TEMP.csv.sdiff.gz failed: Value in column "geocoded_column" uninterpretable as point in input at record 1: "POINT ( )"

Job completed with errors: FAILURE: Processing TEMP.csv.sdiff.gz failed: Value in column "geocoded_column" uninterpretable as point in input at record 1: "POINT ( )"

(jobId:f38a817f-f298-4ccd-8eb2-744c15c19897)

Socrata Writer: Failed to Execute `""C:\\FME_2018\\jre\\bin\\java" -Xmx1g -jar "C:\\FME_2018\\plugins\\socrata\\datasync.jar" -ph true -sc "C:\\Users\\StokesA\\AppData\\Local\\Temp\\FME_1588357616445_39356\\control.json" -i 4p54-9544 -c "C:\\Users\\StokesA\\AppData\\Local\\Temp\\FME_1588357616445_39356\\config.json" --userAgent "FME/2018.7.35.18592" -f "C:\\Users\\StokesA\\AppData\\Local\\Temp\\FME_1588357616445_39356\\TEMP.csv" 2>&1"'

A fatal error has occurred. Check the logfile above for details

... Last line repeated 2 times ...

Translation FAILED with 1 error(s) and 5 warning(s) (2979 feature(s) output)

FME Session Duration: 17.4 seconds. (CPU: 2.0s user, 1.8s system)

END - ProcessID: 39356, peak process memory usage: 242228 kB, current process memory usage: 233932 kB

A fatal error has occurred. Check the logfile above for details

Program Terminating

Translation FAILED.


1 reply

The null values in the lat/long columns were causing in error in the job. Once a valid lat/long was provided the job runs and the Socrata map is updated.

Reply