Skip to main content
Question

Reading from MS Access is SUPER SLOW


Forum|alt.badge.img

Hello: I'm experiencing very slow read speeds from one (1) Access database table that only has approx. 114K records in it. This is also a local copy of the database (.mdb) file (did so to rule out network latency). Is there some trick to increase the read speed for Access?

12 replies

david_r
Celebrity
  • August 23, 2018

Is there a WHERE-clause defined in the reader? If yes, are the referenced fields indexed?


Forum|alt.badge.img
  • Author
  • August 23, 2018

No WHERE clause at all. It's just a simple read on all records in the table.


david_r
Celebrity
  • August 24, 2018

Have you tried compacting the mdb first?


virtualcitymatt
Celebrity
Forum|alt.badge.img+35

How slow are we talking here - any resorsouce warnings in your log file?


fmelizard
Safer
Forum|alt.badge.img+18
  • Safer
  • August 27, 2018

Curious what platform you're on, and if you tried the JDBC MS Access reader as well. That one has greater performance in our tests. Wondering if there may be something special about your dataset too -- if you're willing to share it, please do send in to support@safe.com or post it here.


Forum|alt.badge.img
  • Author
  • August 30, 2018
fmelizard wrote:

Curious what platform you're on, and if you tried the JDBC MS Access reader as well. That one has greater performance in our tests. Wondering if there may be something special about your dataset too -- if you're willing to share it, please do send in to support@safe.com or post it here.

The data is rather sensitive but I could try the JDBC MS Access reader to see if that improves things.

 

 


Forum|alt.badge.img
  • Author
  • August 30, 2018
david_r wrote:

Have you tried compacting the mdb first?

Thanks! I'm not the owner of the data but I can contact the developer to give this a try.

 

 


  • July 16, 2019

I'm experiencing the same problem. Reading an .mdb database of 200 MB with FME workbench is taking me close to 2 hours to read with the MSACCESS_JDBC reader. I've added the following environment variables in >tools> Edit Header

FME_JVM_MIN_HPEAP_SIZE: 1024M

 

FME_JVM_MAX_HEAP_SIZE: 16384M

And the improvement is quite minimum (~10% improvement).

 

Any suggestions?

 

In case it helps, mdb_process_test.txtI attach the log file where I am reading only a few MB of the database, and it still takes almost 2 minutes:

 

Thanks!!

 


david_r
Celebrity
  • July 16, 2019
linares2 wrote:

I'm experiencing the same problem. Reading an .mdb database of 200 MB with FME workbench is taking me close to 2 hours to read with the MSACCESS_JDBC reader. I've added the following environment variables in >tools> Edit Header

FME_JVM_MIN_HPEAP_SIZE: 1024M

 

FME_JVM_MAX_HEAP_SIZE: 16384M

And the improvement is quite minimum (~10% improvement).

 

Any suggestions?

 

In case it helps, mdb_process_test.txtI attach the log file where I am reading only a few MB of the database, and it still takes almost 2 minutes:

 

Thanks!!

 

Is the input file on a local drive or on a network share? Are you running a fairly recent version of FME?


Forum|alt.badge.img+1
  • February 11, 2020

I'm experiencing the same problem with a 173MB Access database. I've copied the Access database locally and compacted it for only a small performance gain.


Forum|alt.badge.img+1
  • February 13, 2020
deanrother wrote:

I'm experiencing the same problem with a 173MB Access database. I've copied the Access database locally and compacted it for only a small performance gain.

Used ADO drive instead of JDBC driver an much much faster!


Forum|alt.badge.img
  • Author
  • February 14, 2020
deanrother wrote:

Used ADO drive instead of JDBC driver an much much faster!

My apologies for not responding in time. That is good to know. One of the issues I had at the time (I later discovered) was network latency issues. At the time to remedy this, I simply copied the access database locally to perform my processing. So even with the JDBC driver, I experienced more efficient processing. I may try to run this process from the network location with the ADO driver to see if that makes a difference. Thanks!


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings