Skip to main content
Question

how to run multiple http caller at a time

  • November 3, 2017
  • 5 replies
  • 155 views

f.kemminje
Contributor
Forum|alt.badge.img+11

Hi Team,

Each feature should go to httpcaller(service request) and comes out with response body. in my case httpcall takes 30 secs for each feature. But I can use 10 httpcaller at a time. Now my query is when my first httpcaller runs, the next feature should go to second httpcaller and third one to thirs httpcaler and so on....

used modulo counter , so it resets at 10, and used attrubute filter to separate port outs. But http caller waits to each feature. I mean second httpcaller waits for the first one to complete.

how can I achieve this? for me, I can run the http service parallel y 10 times(10 Licenses I have).

Now i need fme to run the http parallally

 

Please help

 

5 replies

f.kemminje
Contributor
Forum|alt.badge.img+11
  • Author
  • Contributor
  • November 3, 2017
please refer my fmw capture.jpgscreen shot

 

 


nathanatsafe
Safer
Forum|alt.badge.img+7

Hi @fkemminje, thanks for the question!

You might want to wait for more knowledgeable answers, but I'd suggest taking a look at these posts in the meantime:

 

https://knowledge.safe.com/questions/41114/does-httpcaller-support-multiple-simultaneous-requ.html

 

https://knowledge.safe.com/content/idea/34725/httpcaller-to-support-parallel-processing.html

 

https://knowledge.safe.com/questions/51270/loop-on-httpcaller.html

It sounds like what you're trying to do is not fully supported at the moment, but there are a few possible workarounds, either using a JavaScriptCaller and node.js multiprocessing or embedding HTTPCallers within a custom transformer.

Hope this sparks some ideas!

Nathan


f.kemminje
Contributor
Forum|alt.badge.img+11
  • Author
  • Contributor
  • November 4, 2017

@NathanAtSafe

embedding HTTPCallers within a custom transformer.

I tried your 2nd option ie embedding HTTPCallers within a custom transformer. I kept modulo counter in main page and attribute filter and httpcallers are inside custom transformer window. But I still see httpcallers are not running simultaneously.

 

i Have attached my working fmw with this msg, pls check where am i going wrong

 

Regards

Farook

hub-trace-working.fmw


david_r
Celebrity
  • November 6, 2017

How about using a master workspace with several WorkspaceRunners in parallel?

It's not something I'd usually recommend because of the overhead of starting an instance of fme.exe, but if each http call takes 30-ish seconds it could make sense.


nathanatsafe
Safer
Forum|alt.badge.img+7
f.kemminje wrote:

@NathanAtSafe

embedding HTTPCallers within a custom transformer.

I tried your 2nd option ie embedding HTTPCallers within a custom transformer. I kept modulo counter in main page and attribute filter and httpcallers are inside custom transformer window. But I still see httpcallers are not running simultaneously.

 

i Have attached my working fmw with this msg, pls check where am i going wrong

 

Regards

Farook

hub-trace-working.fmw

Hi @fkemminje

 

Thanks for following up! I think the idea with the custom transformer is to loop through the HTTPCaller - essentially tricking it into running multiple times within one transformer. Please consult the following documentation if you're not sure how to configure looping custom transformers:

https://s3.amazonaws.com/gitbook/Desktop-Advanced-2017/DesktopAdvanced3CustomTransformers/3.Exercise6.html

https://docs.safe.com/fme/html/FME_Desktop_Documentation/FME_Workbench/Workbench/transformers_custom_looping.htm

 

Perhaps the most straightforward is @david_r's idea of running multiple workspaces, each with its own HTTPCaller. You can find the WorkspaceRunner documentation here.

 

Hope this helps!

 

All best,

Nathan


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings