Skip to main content
Solved

Custom Transformer - API Request with http caller in a loop, does not exist on time


icemanserna
Participant
Forum|alt.badge.img+4

  Im fairly new to FME, I have a programming background, and I was hoping to bring those skill here, but i can see is not fully the same. So I apologize before hand 😂

 

I have a custom transformer that does a API calls and loops until it finds “last-batch” set to true from the Header area. 

For Reference Here is the custom transformer:

The first call uses uses the InitialHTTPCaller, which uses the the request url and access token from the main workbench. Here is a screen cap:

 

The _response_header list will have the variables I need to determine if the its the last-batch = true and the next batch url. 

The JSONFragmenter, is exposing the values for _response_header{5}.value,_response_header{6}.value.

I have a tester that checks if _response_header(5) is true to go the pass else it will fail and will go to Output and call upon the Loop to the NextPage. 

 

When the NextHTTPCaller gets called it will be using the User Parameter named NextUrl of which im setting it with the @Value(_response_header{6}.value). This is the actual full url to call the next batch.

The looping portion seems to work properly, because i get all the loops going. i added some loggers to see if the that flag last-batch actually comes back at least once the True value and it does. 

 

My issue is that i dont understand why if its true it should go the output and call it good, but its not. I have also tried to put a tester before the NextHTTPCaller to check if the $(NextUrl) is true, but it does not seem to be set to true.  I have also tried the Variable Setter and it JUST does not set it properly. Honestly I believe im not setting it properly.

Best answer by icemanserna

This way seems to work now.

I changed it a little bit, because i was reading in this community threads and some one mentioned that the check if had to be done before the JSON transformer. I figure why not let me give it a shot. I can say that on the testing workbenched i was able to pull the full amount of records that showed on the actual call.. Im sorry i cant remember where or it was that gave me the idea.

But this shows that this forums work :). thank you to everyone. Im currently running the hopefully the good run. 

 

View original
Did this help you find an answer to your question?

2 replies

icemanserna
Participant
Forum|alt.badge.img+4
  • Author
  • Participant
  • March 18, 2025

Here is proof that the testfilter or the regular tester is finding when the last-batch is True or False. 

In essence when its true it suppose to exit, but its not, and it will time out for sure right now.

The logger clearly shows that value is True and its suppose to load it. the issue here is that its not exiting out.


icemanserna
Participant
Forum|alt.badge.img+4
  • Author
  • Participant
  • Best Answer
  • March 19, 2025

This way seems to work now.

I changed it a little bit, because i was reading in this community threads and some one mentioned that the check if had to be done before the JSON transformer. I figure why not let me give it a shot. I can say that on the testing workbenched i was able to pull the full amount of records that showed on the actual call.. Im sorry i cant remember where or it was that gave me the idea.

But this shows that this forums work :). thank you to everyone. Im currently running the hopefully the good run. 

 


Reply


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings