• Home
  • Forums
  • FME Form
  • Data
  • Read ArcGIS Online attachments and Maintain GlobalID's for both features and attachments
Question

Read ArcGIS Online attachments and Maintain GlobalID's for both features and attachments

jfisch25
Rank
Badge +6

I'm trying to write AGOL features and attachments coming from a feature service tied to a Survey123 survey into a SQL Server geodatabase. I've looked through most of the tutorials and questions about this, but in my use case I need to maintain Global ID's on both the features and the attachments. The feature GlobalID's are being used to name photos that get stored in S3, so I can't use the examples that use a counter to create a new relational ID. I have set the "Maintain Global ID's" parameter on the AGOL reader and have exposed the AGOL attachment attributes.

 

The attachments have a Global ID and Parent Global ID, and I figured it would be simple enough to use both of those to create the relationship class in the GDB since that's what the AGOL service is doing. When I query the service, I can see a feature's Global ID and the attachment's Global ID and its Parent Global ID (the GID of the feature it's attached to).

 

Feature popup:

AGOL_feature_GlobalID 

Query of attachment tied to the above feature:

AGOL_attachment_GlobalIDWhen I read the features in FME Desktop, the Attachment Global ID consistently comes up blank; I've tried the service I'm working with and a separate one, with no luck on either one.

FME_GlobalID_missing 

Is this a bug, or am I doing something wrong? I know both GID's are stored with the attachments, but FME isn't picking them up. Any tips? I'm using FME Desktop 2021.2.1.0 (20211203 - Build 21797 - WIN64). 

11 replies

danminneyatsaf
Rank
Userlevel 2
Badge +10

Hi @jfisch25​ 

This is something that sometimes catches me out since it's easy to miss. To read in attachments (and attachment attributes), you need to enable attachments on the Reader Feature Type. Click on the Reader in your workspace and set Include Attachments to Yes.

image 

Let me know if this helps or if you're still stuck.

gaetanpru
Rank
Badge

Hi @jfisch25​ and @danminneyatsaf​ 

 

I come back on this post because I have the same problem. I can't copy relation between entity and attachment because the globalid field is missing in the attachment table when FME reads it. When I export the geodatabase from ArcGIS Online, globalID is not missing in the data.

I try to enable attachments in my reader but I don't get the value of globalid.

 

When I try to write data in a geodatabase, translation failed with this error when entities has many attachment :

Could not register GlobalID column 'GLOBALID' in table 'TABLENAME__ATTACH'. The error number from ArcObjects is: '-2147155969'. The error message from ArcObjects is: { [ORA-01452: CREATE UNIQUE INDEX impossible ; there are duplicates(TABLENAME__ATTACH)] [TABLENAME__ATTACH]}

 

I feel that is the entity's globalid which is written in the attachment globalid, so there are duplicate ID. Despite this error, data are written in geodatabase with globalid {00000000-0000-0000-0000-000000000000}

 

Thanks for your help.

jfisch25
Rank
Badge +6

Hi @jfisch25​ and @danminneyatsaf​ 

 

I come back on this post because I have the same problem. I can't copy relation between entity and attachment because the globalid field is missing in the attachment table when FME reads it. When I export the geodatabase from ArcGIS Online, globalID is not missing in the data.

I try to enable attachments in my reader but I don't get the value of globalid.

 

When I try to write data in a geodatabase, translation failed with this error when entities has many attachment :

Could not register GlobalID column 'GLOBALID' in table 'TABLENAME__ATTACH'. The error number from ArcObjects is: '-2147155969'. The error message from ArcObjects is: { [ORA-01452: CREATE UNIQUE INDEX impossible ; there are duplicates(TABLENAME__ATTACH)] [TABLENAME__ATTACH]}

 

I feel that is the entity's globalid which is written in the attachment globalid, so there are duplicate ID. Despite this error, data are written in geodatabase with globalid {00000000-0000-0000-0000-000000000000}

 

Thanks for your help.

Hi @gaetanpru​,

Sorry to hear you're experiencing something similar! In my situation, I discovered that it was not crucial to maintain the GlobalID of the attachment features, only the GlobalID of the point features themselves, and since the attached features have that stored in the REL_GlobalID field (or parent GlobalID), I ended up randomly generating a new GlobalID for the attachment features when writing to the attachment table. I used the GUIDGenerator custom transformer and then renamed the attribute from "GFID" to "GLOBALID" and it seems to be working fine.

fme_set_attachment_globalID 

I'm still not sure why the GlobalID's for the attachment features are not showing in FME. @danminneyatsaf​, I have the Include Attachments enabled on my reader, and it's still not reading the GlobalID's for the attached features. It's not a problem for me anymore, but it does seem like it may be something beyond the scope of user error. Maybe worth exploring a little more.

fme_include_attachments

gaetanpru
Rank
Badge

Hello @jfisch25​ 

 

Thanks, it works for me too now. I also thought that the attachment's GlobalID was not important but I couldn't calculate it and I didn't know this transformer.

I'm agree with you, it's not an error from user but just this transformer that does not read this information.

 

 

danminneyatsaf
Rank
Userlevel 2
Badge +10

Hi @gaetanpru​,

Sorry to hear you're experiencing something similar! In my situation, I discovered that it was not crucial to maintain the GlobalID of the attachment features, only the GlobalID of the point features themselves, and since the attached features have that stored in the REL_GlobalID field (or parent GlobalID), I ended up randomly generating a new GlobalID for the attachment features when writing to the attachment table. I used the GUIDGenerator custom transformer and then renamed the attribute from "GFID" to "GLOBALID" and it seems to be working fine.

fme_set_attachment_globalID 

I'm still not sure why the GlobalID's for the attachment features are not showing in FME. @danminneyatsaf​, I have the Include Attachments enabled on my reader, and it's still not reading the GlobalID's for the attached features. It's not a problem for me anymore, but it does seem like it may be something beyond the scope of user error. Maybe worth exploring a little more.

fme_include_attachments

Hi @jfisch25​ thanks for bringing this to my attention. After some further testing of creating an ArcGIS Online feature, adding attachments, and querying it through the REST API, I can also see that separate global IDs exist that should be read in through the arcgisonline_attachment{}.globalid attribute. I've reported this as a bug for our developers to follow up on. For your reference, the issue number is FMEENGINE-74184. If a fix is released, we will notify you here. Sorry for the inconvenience.

 

- Dan M

jfisch25
Rank
Badge +6

Hi @gaetanpru​,

Sorry to hear you're experiencing something similar! In my situation, I discovered that it was not crucial to maintain the GlobalID of the attachment features, only the GlobalID of the point features themselves, and since the attached features have that stored in the REL_GlobalID field (or parent GlobalID), I ended up randomly generating a new GlobalID for the attachment features when writing to the attachment table. I used the GUIDGenerator custom transformer and then renamed the attribute from "GFID" to "GLOBALID" and it seems to be working fine.

fme_set_attachment_globalID 

I'm still not sure why the GlobalID's for the attachment features are not showing in FME. @danminneyatsaf​, I have the Include Attachments enabled on my reader, and it's still not reading the GlobalID's for the attached features. It's not a problem for me anymore, but it does seem like it may be something beyond the scope of user error. Maybe worth exploring a little more.

fme_include_attachments

Thanks Dan, I appreciate it!

gaetanpru
Rank
Badge

Hi @gaetanpru​,

Sorry to hear you're experiencing something similar! In my situation, I discovered that it was not crucial to maintain the GlobalID of the attachment features, only the GlobalID of the point features themselves, and since the attached features have that stored in the REL_GlobalID field (or parent GlobalID), I ended up randomly generating a new GlobalID for the attachment features when writing to the attachment table. I used the GUIDGenerator custom transformer and then renamed the attribute from "GFID" to "GLOBALID" and it seems to be working fine.

fme_set_attachment_globalID 

I'm still not sure why the GlobalID's for the attachment features are not showing in FME. @danminneyatsaf​, I have the Include Attachments enabled on my reader, and it's still not reading the GlobalID's for the attached features. It's not a problem for me anymore, but it does seem like it may be something beyond the scope of user error. Maybe worth exploring a little more.

fme_include_attachments

Hello,

Thanks for the report.

I don't know if it's related but when I run the script with all tables, the globalid is not kept in relationship's table, I have an empty field. To make it work , I have to run the script with only main tables one time and run a second time with only the relationship's tables.

B

Hello @jfisch25​ 

 

Thanks, it works for me too now. I also thought that the attachment's GlobalID was not important but I couldn't calculate it and I didn't know this transformer.

I'm agree with you, it's not an error from user but just this transformer that does not read this information.

 

 

Hi @gaetanpru​ , which FME version are you using? I am still having this issue with FME 2021.2.

gaetanpru
Rank
Badge

Hi @gaetanpru​ , which FME version are you using? I am still having this issue with FME 2021.2.

Hi,

I'm using FME 2020.2

danminneyatsaf
Rank
Userlevel 2
Badge +10

Hi @gaetanpru​ , which FME version are you using? I am still having this issue with FME 2021.2.

Hi @brandonguo​, have you tried the workaround @jfisch25​ detailed above on using the GUIDGenerator?

The issue is still being resolved by our developers, so in the meantime the only way to workaround this is by adding those "fake" GlobalIDs using the GUIDGenerator.

B

Hi @gaetanpru​ , which FME version are you using? I am still having this issue with FME 2021.2.

Thank you. Yes, the GUIDGenerator works perfectly.

Reply


Community Stats

28,707
Posts
109,478
Replies
36,552
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.