Skip to main content
  • EN

I'm using FME Workbench 2018.1 FME(R) 2018.1.0.2 (20180903 - Build 18547 - WIN64)

Sometimes the directory and file pathnames reader does not stop once it has read all files on my external hard drive. It reads the same files multiple times. The only way I've found to fix this is to delete the reader and recreate it, however even then it's pretty inconsistent with whether it stops or not after reading all the files.

I have tried setting max number of features to read to 5 in order to test if it stops, and it doesn't stop after 5 features.

Has anyone else had this happen? Is there a solution?

After a little further testing, it appears that I start running into problems when I use * as a path filter.

Definitely should not be doing that. Are you able to send over a log file / workspace when it is doing the wrong thing? I wonder if it has to do with using an wildcards (*) in the dataset parameter? that could cause this behaviour potentially -- in modern FMEs we've tried to make that harder to do.

I did more testingto narrow down the actual circumstances that this occurs in.

  • The external hard drive is ExFat RAID 5.
  • The Path Filter is *.

The problem doesnot occur if the external hard drive is NTFS RAID 5. It does not occur if theexternal hard drive is ExFat without RAID 5. It does not occur if there is nopath filter.

I have uploaded a zip file containing four logs, 78610.zip.

  • EXFAT_PathFilter.log was run on an exFAT formatted external hard drive without RAID 5 and used a path filter. No issue.
  • EXFAT_RAID5_NoPathFilter.log was run on an exFAT formatted external hard drive with RAID 5 but did not use a path filter. No issue.
  • EXFAT_RAID5_PathFilter.log was run on an exFAT formatted external hard drive with RAID 5 and used a path filter. I stopped it after it had read about 400,000 files. There are only 110k-140k files on the drive.
  • NTFS_RAID5_PathFilter.log was run on an NTFS formatted external hard drive with RAID 5 and used a path filter. No issue.

I did more testingto narrow down the actual circumstances that this occurs in.

  • The external hard drive is ExFat RAID 5.
  • The Path Filter is *.

The problem doesnot occur if the external hard drive is NTFS RAID 5. It does not occur if theexternal hard drive is ExFat without RAID 5. It does not occur if there is nopath filter.

I have uploaded a zip file containing four logs, 78610.zip.

  • EXFAT_PathFilter.log was run on an exFAT formatted external hard drive without RAID 5 and used a path filter. No issue.
  • EXFAT_RAID5_NoPathFilter.log was run on an exFAT formatted external hard drive with RAID 5 but did not use a path filter. No issue.
  • EXFAT_RAID5_PathFilter.log was run on an exFAT formatted external hard drive with RAID 5 and used a path filter. I stopped it after it had read about 400,000 files. There are only 110k-140k files on the drive.
  • NTFS_RAID5_PathFilter.log was run on an NTFS formatted external hard drive with RAID 5 and used a path filter. No issue.

Hi @deb, Thanks a lot for doing such thorough testing to narrow the scenario down. much appreciated! I've reported the issue to our Developement.

 

Due to the unique requirement to reproduce this issue, it might take some time for our team to get to the bottom of the root cause. But I will surely let you know when there is any update on this.

 

In the mean time, is there anyway you can complete the workflow without reading from this particular external hard drive, or somehow avoid using path filter?

 

I did more testingto narrow down the actual circumstances that this occurs in.

  • The external hard drive is ExFat RAID 5.
  • The Path Filter is *.

The problem doesnot occur if the external hard drive is NTFS RAID 5. It does not occur if theexternal hard drive is ExFat without RAID 5. It does not occur if there is nopath filter.

I have uploaded a zip file containing four logs, 78610.zip.

  • EXFAT_PathFilter.log was run on an exFAT formatted external hard drive without RAID 5 and used a path filter. No issue.
  • EXFAT_RAID5_NoPathFilter.log was run on an exFAT formatted external hard drive with RAID 5 but did not use a path filter. No issue.
  • EXFAT_RAID5_PathFilter.log was run on an exFAT formatted external hard drive with RAID 5 and used a path filter. I stopped it after it had read about 400,000 files. There are only 110k-140k files on the drive.
  • NTFS_RAID5_PathFilter.log was run on an NTFS formatted external hard drive with RAID 5 and used a path filter. No issue.

I am avoiding the path filter as a work-around.

 

 

I did more testingto narrow down the actual circumstances that this occurs in.

  • The external hard drive is ExFat RAID 5.
  • The Path Filter is *.

The problem doesnot occur if the external hard drive is NTFS RAID 5. It does not occur if theexternal hard drive is ExFat without RAID 5. It does not occur if there is nopath filter.

I have uploaded a zip file containing four logs, 78610.zip.

  • EXFAT_PathFilter.log was run on an exFAT formatted external hard drive without RAID 5 and used a path filter. No issue.
  • EXFAT_RAID5_NoPathFilter.log was run on an exFAT formatted external hard drive with RAID 5 but did not use a path filter. No issue.
  • EXFAT_RAID5_PathFilter.log was run on an exFAT formatted external hard drive with RAID 5 and used a path filter. I stopped it after it had read about 400,000 files. There are only 110k-140k files on the drive.
  • NTFS_RAID5_PathFilter.log was run on an NTFS formatted external hard drive with RAID 5 and used a path filter. No issue.

Hi again @deb, Would you mind sharing the workspace that exhibit the problem with us? Our developers are investigating this issue. Having your workspace will help a lot! Thank you!

 

Hi again @deb, Would you mind sharing the workspace that exhibit the problem with us? Our developers are investigating this issue. Having your workspace will help a lot! Thank you!

 

path2none.fmw

 

This is the workspace I created to test the issue.

 

 

Reply


Community Stats

30,108
Posts
114,295
Replies
37,743
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.