Hi,
I created shape files from spatial database for the use of our corporate software using fme. And I scheduled in Task manager to run this task daily to keep the files updated.
But now I needed to keep them up-to-date checking database table for every half-an-hour. I'm looking for some help to update existing shape files (similar to update detector used in case to update database table) rather than delete and create each time.
your help with is greatly appreciated.
+26
I don't really think there is a way to update shapefiles. Have you considered using a filegeodatabase instead? That supports updates and inserts and better support for a proper schema.
I don't really think there is a way to update shapefiles. Have you considered using a filegeodatabase instead? That supports updates and inserts and better support for a proper schema.
But corporate software comes with recommendation to use shape files.
+3
From my own experience, SHP files are very difficult to run Change Detection routines on due to the limitations of their 30 year+ file format and slowness to perform delta change operations (Inserts, Updates and Deletes), plus dealing with things like exploding file sizes and SHP files being very file space inefficient, and their lack of automatic data compaction routines.
It can be done, but often it is just as quick to create them from scratch depending on how large the base tables are and what quantity of data delta changes happen. Really, the SHP format is typically only suitable as a data interchange format rather than something to maintain as live data.
On a bigger picture level, if you are needing data to be refreshed every 30 minutes, then this more suggests requiring a high performance enterprise database or high performance local-file based databases. As an example, I've used locally hosted SpatiaLite databases that do frequent SQL or FME based delta change refreshes from the Enterprise Server(s) to feed our downstream GIS applications with a high performance local GIS data storage, and as it natively supports SHP export from the command line, can also fulfil this function when needed to feed any applications that can't read it (Although many mainstream applications like ArcGIS and QGIS natively can).
+3
But corporate software comes with recommendation to use shape files.
What is the corporate software and the applications that need to consume the data?
There are lots of cases where product vendors write SHP as a "recommended" format because they either don't have wider experience with or knowledge of other formats or that is just an "easy" thing to write in the documentation given the prevalence of GIS applications having native SHP read/write capability so its easier to get a basic, but less-than-ideal, software integration so they don't have to explain how to optimise that integration by using faster and more modern formats.
+26
So today I learned about how to use webservices and connect to a SharePoint online list using my organization credentials.
Regular old SAML wasn't cutting it so I had to use the WebService feature of FME.
I created a new template by replacing the [tenant] with the subdomain to which I was trying to connect. And everything went swell, I authenticated successfully and the connection saved.
Then I went and created a new workspace with a new reader for the SharePoint reader.
The good news is that the authentication went successfully.
The bad news is that when I then clicked on the ellipsis to select the list, it didn't work.
Updating access token for 'NewConnectionFromWebServiceTemplate'
Microsoft SharePoint List Reader: Client-side error: '401 Client Error: Unauthorized for url: https://subdomain.sharepoint.com/sites/subSite/_api/web/Lists?%24select=Title%2CId&%24filter=Hidden+eq+false' returned from SharePoint. Error message was: ''OK Fine I've permissions to authenticate against the site but not to view the list of lists how disappointing. I guess I'll just see what happens when I take the URL and paste it into a browser.
Wait, that works? How peculiar?
So here's the question. What am I missing that my user can list the Lists through the API, but can't retrieve the list of Lists through FME?
Hello,
this looks easy with FeatureReader and FeatureWriter instead of source and destination ... 😉
Enter your username or e-mail address. We'll send you an e-mail with instructions to reset your password.