Skip to main content
  • EN

Hi,

As the title says, FME is running much slower on my machine vs a colleague running the exact same process with a similar spec machine. The process took them 2hrs 48 minutes, while I had to kill the process after 4 days of running.

 

We tried changing the fme temp directory off my C drive and onto another and I noticed something in the log file. At the start of the process the log file states that the temp file is:

2020-05-12 14:33:56| 0.2| 0.0|INFORM|System Status: 2.78 TB of disk space available in the FME temporary folder (U:\\osihfile04\\Land_Cover_Store\\FME_TEMP)

However later on at the end of the log file it states:

|1405.6| 0.0|STATS |Stored 51098 feature(s) to FME feature store file `C:\\Users\\jonesD\\AppData\\Local\\Temp\\wb-inspectors-vsYHBZ\\inspector.ffs'

2020-05-12 14:57:24|1405.6| 0.0|INFORM|Saving spatial index into file 'C:\\Users\\jonesD\\AppData\\Local\\Temp\\wb-inspectors-vsYHBZ\\inspector.fsi'

2020-05-12 14:57:24|1405.7| 0.0|INFORM|Finished saving spatial index into file 'C:\\Users\\jonesD\\AppData\\Local\\Temp\\wb-inspectors-vsYHBZ\\inspector.fsi'

As you can see at the start of the process it is saying it is writing the temp files to U drive while at the end it seems to be writing the files to the C drive. Would this make any difference?

 

Thanks for any help.

B

You can set the path for the temp directory to a local directory.

https://knowledge.safe.com/articles/176/fme-temp-environment-variable.html

 

Also, setting the log file to run locally can increase speed dramatically. I'm assuming U: is a mapped drive on the network and will slow things down.

 

You can set the path for the temp directory to a local directory.

https://knowledge.safe.com/articles/176/fme-temp-environment-variable.html

 

Also, setting the log file to run locally can increase speed dramatically. I'm assuming U: is a mapped drive on the network and will slow things down.

 

Yes we had it mapped to my G drive which is a local ssd. But it was slow here. This drive had about 500gb of free space. After this IT wanted to map it to a drive with more space, so they mapped it to U drive, which has 2.5tb of space. It is a network drive, but this is an experiment and not a long term solution. I'm more interested in the fact that at the start of the log file we are sending temp files to U and at the end we are sending them to C. Is this normal does it matter?

Thanks

B

Yes we had it mapped to my G drive which is a local ssd. But it was slow here. This drive had about 500gb of free space. After this IT wanted to map it to a drive with more space, so they mapped it to U drive, which has 2.5tb of space. It is a network drive, but this is an experiment and not a long term solution. I'm more interested in the fact that at the start of the log file we are sending temp files to U and at the end we are sending them to C. Is this normal does it matter?

Thanks

B

It's not supposed to reference drives that aren't in the process or environment variable.

Verify your environment variable is correct by opening a command prompt and typing;

echo %FME_TEMP%

 

If that is correct, open the workbench with notepad and see if the u: drive is referenced. That will give you a better idea of what it's doing. Always use notepad btw. You can modify it and it won't corrupt the workbench.

 

 

Hi @bjudes, why use inspector if speed is critical?

Update, I rolled back my version of FME from 2019.1 to 2018.1 and the workbench has now run with the same speed as my colleagues was running. Down from 4 days (and running) to circa 3 hours.

However, going back in versions appears to have thrown up another problem. I am trying to run about 5500 features through the AreaOnAreaOverlayer and it is giving me 80000+ features on the output. This was never a problem with the previous version. Does anyone have any idea why this would happen?

Thanks for any help you can give.

B

Hi @bjudes, why use inspector if speed is critical?

That is a fair observation, I didn't think of it. Removed them. Thanks

Update, I rolled back my version of FME from 2019.1 to 2018.1 and the workbench has now run with the same speed as my colleagues was running. Down from 4 days (and running) to circa 3 hours.

However, going back in versions appears to have thrown up another problem. I am trying to run about 5500 features through the AreaOnAreaOverlayer and it is giving me 80000+ features on the output. This was never a problem with the previous version. Does anyone have any idea why this would happen?

Thanks for any help you can give.

B

Is the tolerance set to the same value?

Is the tolerance set to the same value?

Hi @ebygomm, thanks for the reply. Yes the tolerance is set to Automatic on both. I've attached a log file if that is any help.

Thanks

B

 

FME_LOG_FILE.txt

Reply


Community Stats

30,353
Posts
115,356
Replies
38,088
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.