HTTPCaller truncating edit request with ESRI's REST API

I can think of the following advice

1. modulo counter to create some grouping and then a test filter to duplicated http callers, effectively distributing your http requests?

    

2. under the httpcaller advanced > turn down the concurrent request from 25 to say 10
3. embed the HTTPcaller into a custom transformer with a looping arrangement introducing  https://support.safe.com/hc/en-us/articles/25407586812429-Getting-Started-with-API-Pagination make sure the httpcaller only has 1 set for the concurrent requests

​@j.botterill, I’m only making one HTTP call. There’s not a huge amount of JSON as you can see in the screenshot and using a web browser to make the call via REST is successful. I’m wondering why FME is truncating the HTTP call and if I can prevent it from happening. Any idea how to do so?

The applyEdits endpoint only supports POST operations. You appear to be using GET
https://developers.arcgis.com/rest/services-reference/enterprise/apply-edits-feature-service/

GET requests (which use the query string) make a URL longer. Servers have a limit of how long this incoming url can be. It’s very possible that this url is exceeding that.

You would need to use a POST request to apply your edits. This sends a separate body along with the request which can be much larger.
https://www.w3schools.com/tags/ref_httpmethods.asp

Additionally, have you considered using Writers instead of the HTTPCaller?

I am using POST in my HTTP request (see attached screenshot). Is there something indicating the HTTPCaller is using GET instead?

With the exception of a Portal Feature Service Writer inside a FeatureWriter transformer, we are using HTTPCallers to perform a variety of tasks - purge locks, apply edits, start reading, start editing, stop editing, stop reading, reconciling (and posting), validating UN network topology, and updating the UN network. It all works nicely with one or two records so we were planning to go that route if possible. As noted above, I can paste the JSON in a web browser using ESRI’s REST endpoint but it fails with FME HTTPCaller if there are more than a few records in the JSON payload. I figure it should work in both. Hoping I’m missing something small.

Where are you placing the JSON in the HTTPCaller? Under query string parameters or body?

I place the JSON under query string parameters. That’s how I pass in the parameters for all of the HTTP calls using ESRI’s REST API. (Apologies for the delayed response. Been on vacation.) 

You’ve run into a limitation: most systems don’t allow URLs longer than 2048 characters, which means your JSON may be getting cut off.
Have a look at the ESRI documentation on how to send the JSON in the request body rather than in the query string.

Yea, that will be the problem. Place it in the body.

GET is designed to retrieve information. Parameters (like search terms, bounding boxes, or output formats) are passed in the query string of the URL. GET requests should not have a body.

POST is designed to send data to a service for processing. Because the data can be large or complex, it’s usually passed in the body of the request. Unlike GET, POST can include both a body and query string parameters if needed.

I went with ​@bruceharold’s suggestion and switched to writing via a Portal Feature Service rather than use applyEdits with an HTTPCaller. (Some context: I’m writing to the Associations layer in ESRI’s Utility Network, which is a hidden service in the ESRI ecosystem. But not for FME! I can see it in my layer list for the service so it’s easy to read and write to. Thanks Safe!)