Skip to main content
Solved

How to send an unknow number of REST API request with HTTP Caller in a Loop.


Hi,

 

I would like to use FME in order to retrieve some data by sending HTTP request to a REST API.

 

A GET request can only retrieve 1000 entries per call and follow the FIFO method (First In, First Out). So I need to GET 1000 entries, process the data and then DELETE 1000 entries and start the cycle again until there is no data left.

 

I setup a loop in FME:

CaptureFME1I've read on this forum that a blocking transformer can not be used in a loop without beeing exported, so I've exported the custom transformer GET_API that contains (.FMX):

CaptureFME2 

So each loop should produce 1000 features.

Outside the loop it works well when I only send 1 request. But inside the loop I get this error message:

 

HTTPCaller is blocking and cannot be used in a loop

 

I can not use a CLONER or something like that since I don't know how many cycle is needed.

 

Do you have another solution ? Is that even possible ?

 

 

Best answer by hkingsbury

in newer versions, the HTTPCaller defaults to running multiple (25) conncurrent connections at once (and is therefore blocking). Change the concurrency to 1 and it will revert to a non blocking transformer

image 

View original
Did this help you find an answer to your question?

3 replies

hkingsbury
Celebrity
Forum|alt.badge.img+54
  • Celebrity
  • Best Answer
  • May 24, 2022

in newer versions, the HTTPCaller defaults to running multiple (25) conncurrent connections at once (and is therefore blocking). Change the concurrency to 1 and it will revert to a non blocking transformer

image 


  • Author
  • May 30, 2022

Perfect !


vhruska
Contributor
Forum|alt.badge.img+6
  • Contributor
  • August 17, 2023
hkingsbury wrote:

in newer versions, the HTTPCaller defaults to running multiple (25) conncurrent connections at once (and is therefore blocking). Change the concurrency to 1 and it will revert to a non blocking transformer

image 

Thank you, Sir!


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings