Hi all, this is my first run with both FME and HTTPS API endpoints. The data I'm requesting comes back to me as paginated JSON in sets of 25 and the API uses offset=[Integer] to allow me to cycle through the data.
This means that when I want to pull almost any data in my day-to-day I need to hit the same endpoint multiple times with offsets 0, 25, 50 etc... until I've got all the data. The JSON comes back with a "has_next_page" boolean so I was hoping to use that to do the following:
1: establish a variable called 'offset' = 0 and an attribute called 'endpoint' = =UrlString] which references 'offset' at the end, run the HTTPCaller
2: when the JSON comes back, take the "has_next_page" value and turn it into an attribute
3: run a test to determine if it's true
4: if true, offset+25, run HTTPCaller again
5: repeat until 'has_next_page' returns false
what's the best way to go about this?