Solved

Reading ArcGIS Online feature attachments

G
Badge

I have an ArcGIS Online hosted feature layer containing point features that have attachments enabled. Could someone help me with some advice for how to use the ArcGIS Online feature reader that will allow me to access the features and their associated attachments (if there are any-not all features have an attachment)? It is easy enough to get the features, but I can't see a way to access the attachments. Thank you!

icon

Best answer by mark2atsafe 20 June 2017, 23:41

View original

24 replies

ngoorman

Reading attachments from features is not currently possible using the ArcGIS Online reader (nor can you write them with the Writer). Within FME, your options are to either use the Pythoncaller or to request the attachments from the relevant features using the HTTPCaller (or equivalent transformer).

pcgveenstra
Badge +1

Reading attachments from features is not currently possible using the ArcGIS Online reader (nor can you write them with the Writer). Within FME, your options are to either use the Pythoncaller or to request the attachments from the relevant features using the HTTPCaller (or equivalent transformer).

But apparently it is not possible with the read as-is. But it surely would be nice. I am going to share two links on how to do this manually (and one via FME). I am going to try them to see how they work. From CloudPoint (Manual Method). From Lars de Vries. (via FME). This would surely make a nice enhancement to the AGOL reader.

 

 

 

 

mark2atsafe
Rank
Userlevel 4
Badge +25
But apparently it is not possible with the read as-is. But it surely would be nice. I am going to share two links on how to do this manually (and one via FME). I am going to try them to see how they work. From CloudPoint (Manual Method). From Lars de Vries. (via FME). This would surely make a nice enhancement to the AGOL reader.

 

 

 

 

This is already filed as PR#65120 with our developers and targeted for 2017.0 - so hopefully we'll see proper support before too long.

 

 

ciarab
Rank
Badge +9
@Mark2AtSafe

Would you have any update on PR#65120 , I don't think it was released in 2017.0 . Do you think its on the radar for 2017.1

 

Thanks

mark2atsafe
Rank
Userlevel 4
Badge +25
@Mark2AtSafe

Would you have any update on PR#65120 , I don't think it was released in 2017.0 . Do you think its on the radar for 2017.1

 

Thanks

Yes, that PR is targeted for 2017.1 and it has a very high priority. So - while I can't guarantee anything - it does look very likely.

 

 

ciarab
Rank
Badge +9
Yes, that PR is targeted for 2017.1 and it has a very high priority. So - while I can't guarantee anything - it does look very likely.

 

 

 

Thanks @Mark2AtSafe I will keep an eye out for it
runneals
Rank
Badge +8
Yes, that PR is targeted for 2017.1 and it has a very high priority. So - while I can't guarantee anything - it does look very likely.

 

 

@Mark2AtSafe Is there any documentation on this? I tried using it in the beta build, but couldn't seem to read any attachments that I had attached to one.

 

Thanks!
mark2atsafe
Rank
Userlevel 4
Badge +25

Looks like build 17475 is the one to use. It has the most recent fixes. See safe.com/beta for the newest betas. The functionality has been checked and verified by the quality team.

@runneals - I don't see any documentation, but according to the notes I have seen there is a new parameter on each reader feature type. It's called 'Include Attachments' and defaults to No (in case you're wondering, it's quicker and more backwards compatible to not default to yes).

So, if you set that, it should return features with the following format attributes:

New format attributes on all features:

  • arcgisonline_objectid (always)
  • arcgisonline_globalid (when globalids present)

New format attributes for attachments, present on features that have attachment(s):

  • arcgisonline_attachment{}.id
  • arcgisonline_attachment{}.globalid
  • arcgisonline_attachment{}.size
  • arcgisonline_attachment{}.name
  • arcgisonline_attachment{}.content_type
  • arcgisonline_attachment{}.data

So it looks to me like the attachment will be in the form of a data attribute, which you could then write to an output file using the AttributeFileWriter transformer.

Let me know if this doesn't help and I can investigate further,

Regards

Mark

runneals
Rank
Badge +8

Looks like build 17475 is the one to use. It has the most recent fixes. See safe.com/beta for the newest betas. The functionality has been checked and verified by the quality team.

@runneals - I don't see any documentation, but according to the notes I have seen there is a new parameter on each reader feature type. It's called 'Include Attachments' and defaults to No (in case you're wondering, it's quicker and more backwards compatible to not default to yes).

So, if you set that, it should return features with the following format attributes:

New format attributes on all features:

  • arcgisonline_objectid (always)
  • arcgisonline_globalid (when globalids present)

New format attributes for attachments, present on features that have attachment(s):

  • arcgisonline_attachment{}.id
  • arcgisonline_attachment{}.globalid
  • arcgisonline_attachment{}.size
  • arcgisonline_attachment{}.name
  • arcgisonline_attachment{}.content_type
  • arcgisonline_attachment{}.data

So it looks to me like the attachment will be in the form of a data attribute, which you could then write to an output file using the AttributeFileWriter transformer.

Let me know if this doesn't help and I can investigate further,

Regards

Mark

Thanks @Mark2AtSafe I apparently exposed the _attachment attributes, but didn't realize there was a drop down :)
geogeoff
Badge

Looks like build 17475 is the one to use. It has the most recent fixes. See safe.com/beta for the newest betas. The functionality has been checked and verified by the quality team.

@runneals - I don't see any documentation, but according to the notes I have seen there is a new parameter on each reader feature type. It's called 'Include Attachments' and defaults to No (in case you're wondering, it's quicker and more backwards compatible to not default to yes).

So, if you set that, it should return features with the following format attributes:

New format attributes on all features:

  • arcgisonline_objectid (always)
  • arcgisonline_globalid (when globalids present)

New format attributes for attachments, present on features that have attachment(s):

  • arcgisonline_attachment{}.id
  • arcgisonline_attachment{}.globalid
  • arcgisonline_attachment{}.size
  • arcgisonline_attachment{}.name
  • arcgisonline_attachment{}.content_type
  • arcgisonline_attachment{}.data

So it looks to me like the attachment will be in the form of a data attribute, which you could then write to an output file using the AttributeFileWriter transformer.

Let me know if this doesn't help and I can investigate further,

Regards

Mark

I would like to extract all the attachments from my AGOL features to a file folder on my desktop.

 

I could use a little help... I think I have all the pieces to the puzzle but.... file folder is empty....hmmm???

 

Jeff N

 

 

C
@Mark2AtSafe Is there any documentation on this? I tried using it in the beta build, but couldn't seem to read any attachments that I had attached to one.

 

Thanks!
Hi @Mark2AtSafe do you have a workbench template example showing how to extract the images from AGOL using 2017.1.1.0?
philippeb
Badge +2

I agree, a template workbench would be cool. I'm not sure how to grab the attachment.

C

Hi - I have 2017.1 beta version. I have an AGOL feature service which has attachments. I would like to set up an FME job that can extract and save these attachments - is this possible?

I have ticked on include attachments as outlined above but can you advise what I do next to save the attachments which could be documents or images.

Many thanks

Caroline

lauraatsafe
Rank
Badge +6

Hi - I have 2017.1 beta version. I have an AGOL feature service which has attachments. I would like to set up an FME job that can extract and save these attachments - is this possible?

I have ticked on include attachments as outlined above but can you advise what I do next to save the attachments which could be documents or images.

Many thanks

Caroline

Hi @cfhogan, If you attach an Inspector transformer to your AGOL reader, do you see the arcgisonline_attachment{} attributes listed when you check the Feature Information Window? If they aren't there, you may need to expose those format attributes on the Reader feature type before you can work with them. Once they are exposed, you should be able to use a transformer like the AttributeFileWriter to save the attachment files.

 

philippeb
Badge +2
Hi @cfhogan, If you attach an Inspector transformer to your AGOL reader, do you see the arcgisonline_attachment{} attributes listed when you check the Feature Information Window? If they aren't there, you may need to expose those format attributes on the Reader feature type before you can work with them. Once they are exposed, you should be able to use a transformer like the AttributeFileWriter to save the attachment files.

 

I tried the AttributeFileWriter but it only creates a file with all the text coded in it. Not the solution I'm looking for.

 

 

Someone told me about the RasterReplacer though and it works great! You need arcgisonline_attachment{}.data of course. You attach a Writer to this and that's it!

 

 

 

 

C

Hi - I have 2017.1 beta version. I have an AGOL feature service which has attachments. I would like to set up an FME job that can extract and save these attachments - is this possible?

I have ticked on include attachments as outlined above but can you advise what I do next to save the attachments which could be documents or images.

Many thanks

Caroline

Hi @LauraAtSafe

 

 

Would it be possible to get some screengrabs of the attributefilewriter . Each record could have different type of file attachment e.g. word doc and a jpg. I've tried using the list option but I can't get it to work. Thanks. Caroline.

 

C
I tried the AttributeFileWriter but it only creates a file with all the text coded in it. Not the solution I'm looking for.

 

 

Someone told me about the RasterReplacer though and it works great! You need arcgisonline_attachment{}.data of course. You attach a Writer to this and that's it!

 

 

 

 

thanks for the answer to this. had been struggling with the AttributeFileWriter

 

 

S
Badge

I have an AGOL service with attachments and want to copy that data to a SDE feature class with attachments. Getting the points across is no problem, but I'm struggling with breaking out hte attachments and sending them to the SDE attachments table and dealing with the relationship class etc. Has anyone done similar work?

G
Badge

I have an AGOL service with attachments and want to copy that data to a SDE feature class with attachments. Getting the points across is no problem, but I'm struggling with breaking out hte attachments and sending them to the SDE attachments table and dealing with the relationship class etc. Has anyone done similar work?

Have you checked out this post already?: AGOL attachments - what are you doing with the attachments after you've read them?

 

I was able to use the explanations, the additional post links, and a provided sample workspace to build my own workspace in which I read my features w/attachments from AGOL and then rebuild those attachments to point features through the relationship class in a file geodatabase. I would think doing so in SDE should be very similar. In the end, you are writing to 3 targets: The point feature class as it sounds like you are doing now, the attachment file itself into an attribute in the __ATTACH object class, and then the relationship class record in __ATTACHREL to tie the two together.

 

 

magnust66
Badge
  1. Add AGOL Reader, include the specific layer that has attachments and expose all related Attachment attributes (i.e. arcgisonline_attachment{}.content_type).
  2. Enure that Feature Types>Parameters>Format>Include Attachments is set to Yes.
  3. Add RasterReplacer transformer and set Blob Attribute: arcgisonline_attachment{0}.data
  4. Add Writer to the format you require, and set the Raster File Name: arcgisonline_attachment[0].name

Voila, you'll see your attachments in an output directory as configured.

I used FME build 2019.1

 

poalgura

Is there another transformer I need to use to expose the attachment attributes? I've set the 'Include Attachments' of my reader to 'Yes' but the attachment attributes don't show in the attributes. When I tried using the Inspector, I could see that the attributes are there..

danminneyatsaf
Rank
Userlevel 2
Badge +10

Is there another transformer I need to use to expose the attachment attributes? I've set the 'Include Attachments' of my reader to 'Yes' but the attachment attributes don't show in the attributes. When I tried using the Inspector, I could see that the attributes are there..

Hi @poalgura​,

 

After setting the Include Attachments parameter to 'Yes" you will want to expose the arcgisonline_attachment{} Format Attributes by going to the Format Attributes tab and exposing each one.

2021-10-06_9-14-08

poalgura

Hi @poalgura​,

 

After setting the Include Attachments parameter to 'Yes" you will want to expose the arcgisonline_attachment{} Format Attributes by going to the Format Attributes tab and exposing each one.

2021-10-06_9-14-08

Thanks, @danminneyatsaf​ !

 

Are you familiar with this workflow too? I tried following the steps written by @magnust66​  but i run into an issue with the RasterReplacer, "INVALID_FEATURE_CANNOT_READ_BLOB". Any idea how to resolve this one?

magnust66
Badge

To expose the attachment attributes, you can use the AttributeExposer transformer, or right-click on the AGOL Reader, and access the Format Attributes tab to also expose the hidden attributes.CaptureRightClick

Reply


Community Stats

28,670
Posts
109,365
Replies
36,531
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.