Expose ArcGIS Online/Portal Atttachments thru FeatureReader

Yes please!

Having the ability to use the featurereader to read attachments is desperately needed. Additionally it would be helpful if the featurereader supported a where clause when attempting to read ArcGIS Online or Portal layers. What would be super helpful is if when reading the attachments, you could opt to not read the physical attachments. This applies to both AGOL and Portal readers.

Currently I have not been able to read the attachment attributes without also reading the attachment. When we have thousands of records, each with multiple attachments, we now have to read every record and all its attachments in before we can check to see if updates are needed. We want to read only the name, size, etc., and determine only the global IDs that that have rows changed in their attachment table to then use a featurereader with a where clause filtered to read only those records that have attachments that are not currently in our base dataset (or alternatively exist in our base dataset and no longer exist in the hosted feature service layer). This would dramatically increase productivity if this were possible as simply reading in the features currently takes up to an hour on some of our processes.

This would be really useful compared to a true reader! I am trying to use an attribute for the Feature Service parameter so that the feature reader wont need to be edited everytime the feature service wants to be changed.

Also saw a similar idea here: AC Idea: Include the option "Include Attachments" in the Feature Reader for AGOL (safe.com)

For anyone who is interested, I created a sample workspace for reading the attachment metadata with the httpCaller and parsing the JSON and then getting just the attachments I want. I also really like the idea of @neilmyoung​  to have an option just to read the metadata in the Reader/Feature Reader at first, as well. And then could have a reader option to just get the attachments via their url.

The Reader has an option to “Include Attachments”. Why not the Feature Reader?

According to this article, writing attachments is coming in 2024 via a packaged transformer?https://community.safe.com/data-7/how-to-write-attachments-to-arcgis-online-19577 

this link appears broken 🤕

I found that it was very slow and prone to break to read attachments + features in a reader, I switched to a feature reader and using httpcallers to get the attachment metadata + download attachments that have been changed, which has worked much smoother and faster.

Having a look at this Idea, we did release the ArcGISAttachmentConnector to allow for reading attachments. I don’t believe the request here is still valid. Could someone confirm if they still see value here?

You can read more about the ArcGISAttachmentConnector here: https://support.safe.com/hc/en-us/articles/37177466948877-How-to-Use-the-ArcGISAttachmentConnector

There is a discrepancy between the "true" reader feature type dialog and the FeatureReader dialog for ArcGIS Portal. In the true reader you can select to in the feature type dialog to have an attribute (list) with the documents, on the FeatureReader there are only the reader/format parameters not any feature type parameters. So any scenario where you have to pick a FeatureReader over a normal reader you are missing functionality, a ArcGISAttachmentConnector feels like a workaround for a problem that shouldn't exist: all options in a reader should be in a FeatureReader 😉

using the latest ArcGISAttachmentConnector requires a fair bit of change. You need to upgrade web services to utilise the oAuth and essentially upgrade v1 to v2 of the custom transformer. 

The issue remains the same, why are some of the Reader parameters like include attachments, not available on the FeatureReader for the arcgis format

 ​@j.botterill I would not invest time with using the legacy formats for my attachment workflows.  The web services template that come with ESRI ArcGIS package (EAP) and building web connections with the templates, is part of the transition from the legacy formats, web connections and transformers. Like any change it could be a pain. The EAP keeps getting better so its worth putting more efforts using the package.

With the planned deprecation of the legacy ESRI feature services formats, I see that attachment workflows are better handled with the combination of the new ESRI ArcGIS Feature Service format and ArcGISAttachmentConnector.

unfortunately, in practice many organisations are stuck using older versions of FME, compatibility with underlying RDBMS and ESRI enterprise portals are the main blockers…., so being able to use the ‘unified’ approach to ESRI ArcGIS packages are a few years away.

I totaly understand the situation. The infrastructure is definitely a big consideration. The EAP goes backward compatible up to FMEENGINE 2024.x.x b24187. From an ESRI context that would be as far back as ArcGIS Pro 3.4/Enterprise 11.4 as compatible to the FME ENGINE that supports the EAP.

This would be a good cutoff to consider for organisations that can migrate sooner than later.