Question

How do edge servers enhance data processing efficiency in FME workflows?

  • 9 January 2024
  • 2 replies
  • 7 views

Hello FME Community,

 

I hope this post finds you well. I've been exploring ways to enhance data processing efficiency in FME workflows, and I've come across the concept of edge servers. I'm intrigued by their potential benefits, but I would love to hear from those with hands-on experience.

 

Could someone kindly share insights on how edge servers contribute to optimizing data processing efficiency in FME workflows?

 

I appreciate any real-world experiences, tips, or resources you can share on this topic. Your insights will undoubtedly help me and others in the community gain a deeper understanding of the role of edge servers in FME workflows.

 

Thank you in advance for your valuable contributions!


2 replies

Userlevel 4
Badge +27

I'm sure others will have input, here a couple of scenarios where I see it as being beneficial

 

1.

Say you're an organisation that has most of their data and processing resources internally. However, you have some very large datasets that for various reasons, can't be stored on your internal infrastructure. Instead it is stored in a cloud hosted environment (AWS/Azure etc).

 

Due to constraints you are not geographically located near the data and/or access to these resources is poor and performance is slow. If you need to use that data it will be very slow accessing the data.

 

If you place an FME Flow instance (or an Engine) on a virtual machine in the same geographic location access to the data will be very very quick, and all you'll need to be doing is sending the request and requesting the results of the process from this remote location

 

2.

A similar example to the above, you're a multinational Engineering firm with offices in Sydney, New York and London. Each location stores it data for its area (Sydney stores Australian data, NY stores USA data etc).

 

However, the way the company works is you might be based in Sydney but working on a USA project. Accessing the data from the NY office is possible, but slow(er). So if you have an FME Flow instance in NY, then the processing times will be much faster

 

Userlevel 4
Badge +27

I'm sure others will have input, here a couple of scenarios where I see it as being beneficial

 

1.

Say you're an organisation that has most of their data and processing resources internally. However, you have some very large datasets that for various reasons, can't be stored on your internal infrastructure. Instead it is stored in a cloud hosted environment (AWS/Azure etc).

 

Due to constraints you are not geographically located near the data and/or access to these resources is poor and performance is slow. If you need to use that data it will be very slow accessing the data.

 

If you place an FME Flow instance (or an Engine) on a virtual machine in the same geographic location access to the data will be very very quick, and all you'll need to be doing is sending the request and requesting the results of the process from this remote location

 

2.

A similar example to the above, you're a multinational Engineering firm with offices in Sydney, New York and London. Each location stores it data for its area (Sydney stores Australian data, NY stores USA data etc).

 

However, the way the company works is you might be based in Sydney but working on a USA project. Accessing the data from the NY office is possible, but slow(er). So if you have an FME Flow instance in NY, then the processing times will be much faster

 

@mark2atsafe​  this might be a good idea for a 'lighter' less tech focused blog?

Reply