Why is it now so slow to read an AGOL service with attachments set to 'Yes'

Hi @bbunch​ 

Are you using the very latest 2022.0.1 version?

Let me test this locally and I will let you know.

Hello @bbunch​ , if you're using the latest 2022.0.1 and still experiencing this issue. Would you be able to share log files from both translations (Include Attachments = YES and Include Attachments = NO).

Hi, Thanks for the replies. The installed version is Build 22343, 2022.0.0.2

I will update to the 2022.1 Beta and test.

👎 I have run 2022.1 with no attachments and that works as expected when attachments are set to 'no'. I have attached the log. The attachment 'yes' process took over 4 hours to complete

Thanks

posted a file. This is with no attachment

This is with attachments yes after 4hrs plus.

Yes

Hello @bbunch​ , I've created a case to investigate this further. C677160. Best, Kailin.

You might find that it is actually doing stuff in the background even when the only log message is "Updating access token". It's the same for my workspace at the moment which is updating the access token for a Google Drive web connection, but when I look at the features they are slowly moving through.

For my workflow reading AGOL attachments, I found that I needed to drop the number "Features Per Request" on the reader all the way down to 20 otherwise the requests could timeout. From your log it looks like it's requesting too many in one go, so you could try reducing that parameter? You'll get more log messages that way and can see where it's stalling. I have an attachment on virtually every feature so that works out as ~20 image attachments per request.

I also found issues with the data interop (Esri) 2022 version (since the log4j patch) that killed many of my workflows that worked flawlessly before (it kept failing to save the .ffs cache files), and had to roll back to 2021 to fix it.

Hello @James Singleton​ , thanks for sharing this information! After reading your comment, this has me curious, its very odd a significant amount of your workspaces were killed. Would you be able to share any further information about this, such as:

• What version of data interop you were using before you upgraded to 2022? What version did you upgrade from?
• Did all workspaces fail with the same error? (were all workspaces affected by the same issue?)
• Are you able to provide any logfiles from these dead workspaces?
• What version of FME 2021 did you roll back to?

Sorry for the laundry-list of questions! Just trying to get a clearer picture. Best, Kailin.

Hi @kailinatsafe​ ,

I made a post about this before so you should be able to access the logs I submitted at the time and the relevant information on that post.

Yes, I had multiple workspaces failing with the same error and at different points, it was very unusual.

I'm back on FME(R) 2021.0.0.0 (20210305 - Build 21302 - WIN64) with no problems so I haven't tried to pursue the issue any further. If I remember correctly I noticed 2021 workspaces performed more quickly when reverting back (related to the original post in this question).

Hi @kailinatsafe​ On a related item, can you advise if the "include attachments" toggle can be added to the AGOL and ESRI Portal reader settings when connecting using the "Feature Reader" transformer, as found in the 'native' readers? I have a workflow that attempts to use the Feature Reader to connect to Portal and expose the additional arcgisonline_attachment{} list elements, but nothing comes back - even after using the attributeExposer post read. I am currently running 2021.0. Thanks very much.

This is a limitation with the Feature Reader which I would like to see implemented too. I've just posted it as an idea (Edit: just found it posted as an idea already here).

I'm still having this issue in fme 2023.1 Is there a workaround suggested?

Hello @jakethepainter​, unfortunately we did not arrive at a resolution. @James Singleton​ suggestions above are likely the best steps to start with. Does the translation finish or does it hang indefinitely when FME tries to update the token? Also, would you be able to share a logfile? Best, Kailin.

Hi @kailinatsafe​ I've attached one of the shorter logs where the timeout was within 10 min but didn't manage to read any batches of records. I've had other times where the same workspace has run for an hour+ and managed to read some records but I can't get the workspace to finish in 2023 whereas it only takes a minute (literally) in 2019.

See also Why is reading ArcGIS Online attachment failing in 2023 when it works great in 2019? (safe.com)

Hello @jakethepainter​, I noticed you shared a job log, is the same behavior seen on FME Form? or is reading attachments more performant on Form? I might suggest you submit a support case so we can investigate further.

Was a resolution ever found to this? 

Since we upgraded to 2022.2 this has been an issue.  We had a background process which would export survey123 tables and photo attachments field teams had completed that week.  A process that used to take an hour now gets nowhere near completing if left running all day and eventually times out.  Unfortunately quite a significant backlog has built up now, so not entirely sure how to proceed.  Thanks.

I have experienced problems with reading features with attachments from AGOL - with the connection sometimes being dropped while reading, due to timeouts.

A workaround that I have found works out very well - just use the REST API and HttpCallers. For my case we are reading out the features first without attachments, and then using the REST API to get the attachments.

Using HttpCaller has multiple benefits - one can easily run parallell processing with multiple threads, and retry on failure.

Hi, ​@sam_tw and ​@nordpil.

Thank you for your concern and for your patience. We encourage you to submit a support ticket with us and we can take a look there. The workaround we find we can put in this post.

Best,