Skip to main content
Question

Reading CSV file via directory path and FeatureReader - Get Duplicate records.


Hello everyone,

 

I used "featureReader" to read all CSV files from a specific directory via DirectoryPath and it works fine with me but I am surprised a little bit why I got duplicate data.

 

For example, if I have 3 files, it counts the read 9. What is the best way to ready all CSV files fast and quickly?

 

Do you advise to enable the cache files?

 

5 replies

hkingsbury
Celebrity
Forum|alt.badge.img+54
  • Celebrity
  • May 9, 2021

How many features do you have going into the featureReader? If you have three features going in, you are asking FME to perform that read three times, hence the duplicates


amdubois
  • December 8, 2021

Hi,

I am having the same issue. I have a directory reader to extract gdb from multiple folders. Then I feed the result to a FeatureReader but everything is duplicated. In my example, I have 2 features going from the reader to Feature reader which is correct in my case since there are two gdbs. I don't understand why FME would be executing the Feature Reader twice? Isn't it just reading each gdb ONCE? Otherwise, how are we supposed to program it not to double output data?

first reader outputs 2 gdbfeatureReader outputs doubled 

Kind regards,

Anne-Marie

 

 


sigbono
Contributor
Forum|alt.badge.img+11
  • Contributor
  • December 19, 2022
amdubois wrote:

Hi,

I am having the same issue. I have a directory reader to extract gdb from multiple folders. Then I feed the result to a FeatureReader but everything is duplicated. In my example, I have 2 features going from the reader to Feature reader which is correct in my case since there are two gdbs. I don't understand why FME would be executing the Feature Reader twice? Isn't it just reading each gdb ONCE? Otherwise, how are we supposed to program it not to double output data?

first reader outputs 2 gdbfeatureReader outputs doubled 

Kind regards,

Anne-Marie

 

 

Same issue with FeatureReader here. Any solution to this? Thanks.


dustin
Influencer
Forum|alt.badge.img+31
  • Influencer
  • December 19, 2022
sigbono wrote:

Same issue with FeatureReader here. Any solution to this? Thanks.

@sigbono​ Can you provide some specifics of what you are seeing? It seems the previous issues were due to multiple features being fed to the FeatureReader.


sigbono
Contributor
Forum|alt.badge.img+11
  • Contributor
  • March 22, 2023
sigbono wrote:

Same issue with FeatureReader here. Any solution to this? Thanks.

@dustin​ This might be the case. I just added a Tester to filter out the specific path and the issue was solved.


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings