Skip to main content

Hi there,

We have an excel file with an easy table. Just 2 columns: Position and Id. The file has about 100 lines.

We want to read the file and process each line. We have tried with a custom transformer but we are not able to process the lines individually. If we read the Excelfile we get the 100 lines and all lines will be process at the same time. If we convert the lines into a List, we get the error "Transformer/Factory test_loop_2_ListBuilder is blocking and cannot be used in a loop, but was found between 'Loop to test_loop_2_Looper1697190238 Input Splitter' and 'test_loop_2_Looper1697190238 Input Splitter'".

 

Has anyone an idea to make it?

I have found this link (https://docs.safe.com/fme/html/FME-Form-Documentation/FME-Form/Workbench/transformers_custom_looping.htm) but it helps me only with the loop. Not how I can process the ids separatelly.

 

Many thanks in advance!

When you assign unique ids to the lines, you can configure the custom transformer to do a group by and process the lines individually. I would not use a loop for this.


I'm wondering what process would make you use a loop for processing this simple dataset. FME will already process each line (each feature) apart.


I'm wondering what process would make you use a loop for processing this simple dataset. FME will already process each line (each feature) apart.

@dataman​ is probably not aware of the default Group By functionality in FME. Or is missing it in certain transformers. A custom transformer can than be a possible solution. I remember a specific case where I needed it in a FeatureReader.


Thanks for the answers!

Yes, I am a freshman in FME and I have no experience using Group By... I do have one column with unique IDs. Where can I configure the custom transformer to do a group by? I do not see any option to do that (I mean clicking on the green Input transformers). Or should I check the "Group by" check box of the normal transformers within the custom transformer?


Thanks for the answers!

Yes, I am a freshman in FME and I have no experience using Group By... I do have one column with unique IDs. Where can I configure the custom transformer to do a group by? I do not see any option to do that (I mean clicking on the green Input transformers). Or should I check the "Group by" check box of the normal transformers within the custom transformer?

Most transformers have a Group Processing option which will isolate features which have a common ID. See the Clipper for example, if you check Group Processing and select "GroupId" it will only compare clippers and clippees which have the same GroupId.

 

In the custom transformer, in the Navigator window, Transformer Parameters, Group By, right click and choose Create User Parameter. Click OK. In the main workspace, you can now choose an attribute to run the custom transformer grouped by.

 

If needed, you can also create user parameters for Group By Mode and Parallel Processing.


Most transformers have a Group Processing option which will isolate features which have a common ID. See the Clipper for example, if you check Group Processing and select "GroupId" it will only compare clippers and clippees which have the same GroupId.

 

In the custom transformer, in the Navigator window, Transformer Parameters, Group By, right click and choose Create User Parameter. Click OK. In the main workspace, you can now choose an attribute to run the custom transformer grouped by.

 

If needed, you can also create user parameters for Group By Mode and Parallel Processing.

I have to use the Values of one column for the URL of a HTTPCaller. I have checked the parameters of the HTTPCaller and there I have no GroupBy Option.

Can I read the table and after that just put a HTTPCaller or would it be too much for the performance? The Workbench must run about 100 URLs.

 

Maybe I explain what we want to do in order to a better understanding:

  1. We read an Excel file with 2 Columns: Location | ID
  2. for each ID we have to get the values from a CSV File (HTTPCaller)
  3. Process the data: if we have no data of this ID or the values are always the same -> this device has a problem and we write the ID (and Location if possible) in a output Excel file.

 

Because of this iteration (100 times) I had a loop in mind. But if there is a better option, it is wellcome :-)


Most transformers have a Group Processing option which will isolate features which have a common ID. See the Clipper for example, if you check Group Processing and select "GroupId" it will only compare clippers and clippees which have the same GroupId.

 

In the custom transformer, in the Navigator window, Transformer Parameters, Group By, right click and choose Create User Parameter. Click OK. In the main workspace, you can now choose an attribute to run the custom transformer grouped by.

 

If needed, you can also create user parameters for Group By Mode and Parallel Processing.

Can I read the table and after that just put a HTTPCaller or would it be too much for the performance?

 

Yes, this is how FME works. Just concat the url from the attributes in the HTTPCaller. Not hard on performance.


Reply