I have set up a http caller with query string parameters. It would appear that the server throws a 500 error when its finished returning the records. Does not appear to affect results but would like to get rid of error.
Page 1 / 1
Hello @nick824 , would you be able to share a screenshot of how your HTTPCaller is currently set up? Also, it might help to know the full error message! If you're able to share. What version of FME are you using currently? Best, Kailin
Hi kailin,
Thanks for getting back to me. Please see attached. I'm on 2020.1
Hi @nick824
It's hard to tell without knowing which API service you are using, but is it always the last record that's giving the 500 error? I'm wondering if there might be a slight logic issue here? Are you always retrieving 1 too many pages?
Hi @siennaatsafe ,
There are occasions when the error does not occur but this is rare. I have 3 loops similar to the one I sent details of that work off the same id token in succession. The error seems to occur at the end of the process of each loop, so I get 3 identical errors, one for each loop. The additional problem is that I don't know whether all data is returned because of the error. I have just run it again and the 'has_next_page' was true for all returned results so I guess that is not what is causing the process to end. The only other criteria is the query parameters. One thing I have noticed is that the incremental offset starts at 25 and increases in chunks of 25 as its supposed to. The offset of 0 results are however returned at the end.
Hi @nick824 , have you tried using a decelerator? Sometimes looping can overwhelm a web service because we are hitting it too quickly and with too many calls. In newer versions of FME we can rate limit our requests to help prevent this. http://docs.safe.com/fme/html/FME_Desktop_Documentation/FME_Transformers/Transformers/httpcaller.htm
Hi @siennaatsafe (Safer) ,ok I'll try that - thanks. Is there a best place to put the decelerator?
@nick824 as a test, I'd put it right before the HTTPCaller so that any input to the HTTPCaller will hit the Decelerator first.
ok thanks @siennaatsafe (Safer). I'll give it a go.
I tried the Decelerator using various timings/settings but unfortunately I'm still getting the error. Any other thoughts?
thanks
I tried the Decelerator using various timings/settings but unfortunately I'm still getting the error. Any other thoughts?
thanks
Is it possible to upgrade your FME Desktop? If so you could try the Automatic Retries in the HTTPCaller see Retry Failed Requests here: https://docs.safe.com/fme/html/FME_Desktop_Documentation/FME_Transformers/Transformers/httpcaller.htm
This was added in 2021.2 build 21701+
Hi siennaatsafe, That looks promising. I'll try and persuade the powers that be!