I've got a HTTPCaller in a loop that retrieves data page by page (I use offset and limit). I have a tester that tests whether 'has_last_page' is true. This all seems to works ok but the translation does not stop - it just keeps repeating the last URL even though all records have been returned.
Solved
Hi all, I am fairly new to FME but have a problem with an httpcaller in a loop. I cannot seem to stop the translation when all records returned. Any help would be appreciated.
Best answer by nick824
Hi all, I tried cloner option and worked very well if I can get records per page etc. When the only data available is 'has_next_page' I eventually found that by moving the JSON extractor and JSON fragmenter stages out of the loop and placing them just before the writer it works like a treat.
This post is closed to further activity.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.
Maybe the sample workspace posted by @bruceharold here can give you some clues: https://community.safe.com/s/question/0D54Q000080hPDg/resolving-pagination-with-httpcallerloop
+47- Celebrity
As an alternative, if you know how many records per page and how many total records there are you can use a cloner to create the required number of features and calculate their offset. This let's you avoid loops, however, if the total number of records isn't present then this isn't an option.
Thanks for the advice. Unfortunately I don't know the number of total records. I just have 'has_next_page' and 'total_rows'. I know that there are 25 rows/page but the 'total_rows' only reports a maximum of 501 so does not necessarily reflect actual.
Hi all, I tried cloner option and worked very well if I can get records per page etc. When the only data available is 'has_next_page' I eventually found that by moving the JSON extractor and JSON fragmenter stages out of the loop and placing them just before the writer it works like a treat.
+47- Celebrity
Hi all, I tried cloner option and worked very well if I can get records per page etc. When the only data available is 'has_next_page' I eventually found that by moving the JSON extractor and JSON fragmenter stages out of the loop and placing them just before the writer it works like a treat.
Nice, good result!
Login to the community
No account yet? Create an account
An FME Account is required to contribute
LoginEnter your E-mail address. We'll send you an e-mail with instructions to reset your password.