Hi folks,
I got the attached error, very basic process, I'm joining a table loaded from a CSV with another CSV. It should be easy and my transformers are correct.
This post is closed to further activity.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.
+51- Celebrity
- 2077 replies
Hi @osarhan,
Are you try to merger between two CSV files?
If yes, try with the transformer FeatureMerger.
Thanks,
Danilo
+1Hi @osarhan,
Are you try to merger between two CSV files?
If yes, try with the transformer FeatureMerger.
Thanks,
Danilo
+51- Celebrity
- 2077 replies
I was originally doing this, but both files are extremely big so it it hits memory constraints :(
Thanks,
Danilo
- 8394 replies
I don't think the Joiner is particularly well suited to join two huge CSV files, personally.
I would use the FeatureMerger, but it's important to make sure that you use two separate readers, one for the supplier CSV and one for the requestor CSV.
In the Navigator, make sure the suppliers CSV comes at the very top, this means that dataset will be read before the requestors.
You can then tell the FeatureMerger that the suppliers come first, which will "unblock" then transformer making it a lot more efficient memory-wise.
- 8394 replies
Another alternative could be to use the InlineQuerier, but it's slightly more complicated. Let us know if the "non-blocking" FeatureMerger doesn't work, and we'll help you if needed.