Issue writing to ArcGIS Online - error code 1000

@runneals​ has updated me that this is an ESRI issue that he's pursuing.

Update.... esri is aware and actively looking into it, although they don't have a timeframe...

I believe the following errors are all related. The first code block shows errors when trying to query/read REST services.

<- HTTP body response ->
{"error":{"code":400,"message":"The type initializer for '\u003cModule\u003e' threw an exception.","details":[]}}
<- FME logs ->
The error code from the server was '1000' and the message was: 'The type initializer for '' threw an exception.'
 The error code from the server was '1000' and the message was: 'The type initializer for '<Module>' threw an exception.'
 
<- HTTP body response ->
{"error":{"code":400,"message":"Cannot perform query. Invalid query parameters.","details":["Unable to perform query. Please check your parameters."]}}
<- FME logs ->
Python Exception <HTTPError>: 404 Client Error: Not Found for url: {SERVICEURL}/FeatureServer/0/query?f=json&token={TOKEN}

This second code block below shows the only error that I was able to figure out when applyEdits/writing (insert/update) to a service on ArcGIS Online.

The error code from the server was '1000' and the message was: 'The type initializer for '<Module>' threw an exception.'

As of 2021/01/25 @ 11pm UTC, esri support said they have a lead and that it's impacting at least hive0 which are organizations whose services are from the services.arcgis.com url.

I have had success in completing the workspace that was bringing up this issue, so I can overwrite feature service data successfully, but not sure if it means it is solved for the wider world.

Resolved!

Cause: An instance went bad.

Solution: Work with esri to have them restart it.

If you ever encounter any of these weird issues, attempt to mass produce the issue by using a simple powershell script to request the resource, then use a windows task scheduler to hit it every minute. Run fiddler to capture network logs and look to see if the X-ArcGIS-Instance parameter in the header is the same for all the ones having issues.

For the windows task scheduler, set the program to: Powershell.exe and the Add arguments to: -ExecutionPolicy Bypass LOCATION\\TO\\PowershellScript.ps1

I had a 502 and 504 error separately yesterday, after that resolution was reported by @runneals​, but as often occurs with ESRI errors there was little detail. I haven't got Fiddler up and running yet but if it happens again I may well do so. Process ran ok on schedule last night but will update this if it does fail again.