Skip to main content
Question

Resolving pagination with Httpcaller/loop


Hello,

I am currently working with some scooter mds data and I am trying to find references to help with pagination. I would like to be able to call page one and use a json fragmenter to process and then go onto page 2, etc. Has anyone worked with something like this before or has some ideas? I'd love some input.

The API has links in the json that have "first", "next", "last" that reference the corresponding page number.

My thought was to keep referencing the "next" page url for processing until the last page: It looks a little like this:

next: http://(scooter provider mds)trips?end_time=(date/time)&page=2&start_time=(date/time)

http://(scooter provider mds)trips?end_time=(date/time)&page=3&start_time=(date/time)

7 replies

bruceharold
Contributor
Forum|alt.badge.img+17
  • Contributor
  • April 3, 2020

Here is a workspace to give you the pattern:

githubreporter.fmw


  • Author
  • April 3, 2020
bruceharold wrote:

Here is a workspace to give you the pattern:

githubreporter.fmw

@bruceharold I am running into the issue of the string searcher not matching with any values. Do you know why this would be occurring?


bruceharold
Contributor
Forum|alt.badge.img+17
  • Contributor
  • April 3, 2020
jlars23 wrote:

@bruceharold I am running into the issue of the string searcher not matching with any values. Do you know why this would be occurring?

You'll need to get the regular expression working, the regex editor should help there. Maybe a case issue with next versus Next or something?


  • Author
  • April 3, 2020
bruceharold wrote:

You'll need to get the regular expression working, the regex editor should help there. Maybe a case issue with next versus Next or something?

Thank you for the information! I actually ended up using a json extractor to call the "next" page.


flavien
Contributor
Forum|alt.badge.img
  • Contributor
  • September 23, 2020
bruceharold wrote:

Here is a workspace to give you the pattern:

githubreporter.fmw

Do you mind sharing your example workspace again? It seems the link is not working anymore..


bruceharold
Contributor
Forum|alt.badge.img+17
  • Contributor
  • September 23, 2020
flavien wrote:

Do you mind sharing your example workspace again? It seems the link is not working anymore..

Here you go.


flavien
Contributor
Forum|alt.badge.img
  • Contributor
  • September 23, 2020
flavien wrote:

Do you mind sharing your example workspace again? It seems the link is not working anymore..

Thank you!


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings