Skip to main content
Question

Simple Join failing


Forum|alt.badge.img+1

Hi folks,

I got the attached error, very basic process, I'm joining a table loaded from a CSV with another CSV. It should be easy and my transformers are correct.

https://imgur.com/P3jkoQM

5 replies

danilo_fme
Evangelist
Forum|alt.badge.img+45
  • Evangelist
  • October 9, 2017

Hi @osarhan,

Are you try to merger between two CSV files?

If yes, try with the transformer FeatureMerger.

Thanks,

Danilo


Forum|alt.badge.img+1
  • Author
  • October 10, 2017
danilo_fme wrote:

Hi @osarhan,

Are you try to merger between two CSV files?

If yes, try with the transformer FeatureMerger.

Thanks,

Danilo

I was originally doing this, but both files are extremely big so it it hits memory constraints :(

danilo_fme
Evangelist
Forum|alt.badge.img+45
  • Evangelist
  • October 10, 2017
osarhan wrote:
I was originally doing this, but both files are extremely big so it it hits memory constraints :(
Perfect. Could you share the print inside your transformer Paramenters Joiner and share it?

 

 

Thanks,

 

Danilo

 

 


david_r
Celebrity
  • October 10, 2017

I don't think the Joiner is particularly well suited to join two huge CSV files, personally.

I would use the FeatureMerger, but it's important to make sure that you use two separate readers, one for the supplier CSV and one for the requestor CSV.

In the Navigator, make sure the suppliers CSV comes at the very top, this means that dataset will be read before the requestors.

You can then tell the FeatureMerger that the suppliers come first, which will "unblock" then transformer making it a lot more efficient memory-wise.


david_r
Celebrity
  • October 10, 2017

Another alternative could be to use the InlineQuerier, but it's slightly more complicated. Let us know if the "non-blocking" FeatureMerger doesn't work, and we'll help you if needed.


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings