Skip to main content
Solved

Make FME use more CPU

  • September 14, 2015
  • 2 replies
  • 201 views

Hi,

 

 

I was wondering if there is a way to make FME use more CPU? I have workflow with GML Reader and Multiple SQL Server Non-Spatial Writers. Multiple writers are there because I need to insert into identity columns in four different tables parts of transformed GML data.

 

Here FME is reading GML and writing it into SQL Server by using 10-13% of CPU. How can I make it use e.g. 70-80%? Memory occupied bu FME is also 500-700MB.

 

 

How does this work?

Best answer by david_r

Hi

 

 

FME will use as much resources as possible and necessary. For most scenarios, this means using one CPU core at up to 100%. So if you have a CPU with 4 cores, the task manager will report the total charge at 25% in this case. Developing an application capable of spreading its load over several cores can be very complicated and costly from a development point of view, all depending on how easy it is to break one task into small sub-tasks (which is necessary for spreading an application over several CPU cores). This is not a limitation of FME, but rather a "feature" of CPU architecture.

 

 

You can try to play around with the parallel processing (https://knowledge.safe.com/articles/Samples_and_Demos/Parallel-Processing) options in certain transformers, but frankly, my experiences have been less than amazing. You might have more luck though, you'll just have to try. Also, this option will do nothing for your readers/writers, as it only applies to transformers with Group By settings.

 

 

Also consider the possibility that the database can be slow to send data to FME, in which case FME can do very little about it.

 

 

David

 

 
View original
Did this help you find an answer to your question?
This post is closed to further activity.
It may be a question with a best answer, an implemented idea, or just a post needing no comment.
If you have a follow-up or related question, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.

2 replies

david_r
Celebrity
  • Best Answer
  • September 14, 2015
Hi

 

 

FME will use as much resources as possible and necessary. For most scenarios, this means using one CPU core at up to 100%. So if you have a CPU with 4 cores, the task manager will report the total charge at 25% in this case. Developing an application capable of spreading its load over several cores can be very complicated and costly from a development point of view, all depending on how easy it is to break one task into small sub-tasks (which is necessary for spreading an application over several CPU cores). This is not a limitation of FME, but rather a "feature" of CPU architecture.

 

 

You can try to play around with the parallel processing (https://knowledge.safe.com/articles/Samples_and_Demos/Parallel-Processing) options in certain transformers, but frankly, my experiences have been less than amazing. You might have more luck though, you'll just have to try. Also, this option will do nothing for your readers/writers, as it only applies to transformers with Group By settings.

 

 

Also consider the possibility that the database can be slow to send data to FME, in which case FME can do very little about it.

 

 

David

 

 

  • Author
  • September 14, 2015
Thanks David! As I supposed... Big undertaking for me to change complete logic at this moment. But, thanks.

Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings