Skip to main content
Solved

How to send an unknow number of REST API request with HTTP Caller in a Loop.

  • May 24, 2022
  • 3 replies
  • 287 views

Hi,

 

I would like to use FME in order to retrieve some data by sending HTTP request to a REST API.

 

A GET request can only retrieve 1000 entries per call and follow the FIFO method (First In, First Out). So I need to GET 1000 entries, process the data and then DELETE 1000 entries and start the cycle again until there is no data left.

 

I setup a loop in FME:

CaptureFME1I've read on this forum that a blocking transformer can not be used in a loop without beeing exported, so I've exported the custom transformer GET_API that contains (.FMX):

CaptureFME2 

So each loop should produce 1000 features.

Outside the loop it works well when I only send 1 request. But inside the loop I get this error message:

 

HTTPCaller is blocking and cannot be used in a loop

 

I can not use a CLONER or something like that since I don't know how many cycle is needed.

 

Do you have another solution ? Is that even possible ?

 

 

Best answer by hkingsbury

in newer versions, the HTTPCaller defaults to running multiple (25) conncurrent connections at once (and is therefore blocking). Change the concurrency to 1 and it will revert to a non blocking transformer

image 

This post is closed to further activity.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.

3 replies

hkingsbury
Celebrity
Forum|alt.badge.img+63
  • Celebrity
  • 1625 replies
  • Best Answer
  • May 24, 2022

in newer versions, the HTTPCaller defaults to running multiple (25) conncurrent connections at once (and is therefore blocking). Change the concurrency to 1 and it will revert to a non blocking transformer

image 


  • Author
  • 5 replies
  • May 30, 2022

Perfect !


vhruska
Contributor
Forum|alt.badge.img+6
  • Contributor
  • 12 replies
  • August 17, 2023

in newer versions, the HTTPCaller defaults to running multiple (25) conncurrent connections at once (and is therefore blocking). Change the concurrency to 1 and it will revert to a non blocking transformer

image 

Thank you, Sir!