Question

MongoDB GridFS - Huge Document Sizes

  • 27 September 2018
  • 2 replies
  • 5 views

Does anyone know if FME has compatibility with MongoDB's GridFS to store documents that are larger than 16MB? I've written an embedded JSON but the average document size is 49MB.

Will I need to load it into mongo outside of FME or is there capability for this currently?


2 replies

Badge +7

Hi @majobajob,

Thanks for the question. Currently we don't support GridFS files, so if you are dealing with documents above 16MB you will have to load them outside of FME. This is a known issue and we are tracking user demand in advance of adding support for GridFS. Would you consider adding an idea on our Ideas forum and tell us a little bit about your use case for MongoDB and large documents? Any comments you might have will help us if and when it comes time to add this feature.

 

 

Thanks,

Nathan

Hi @majobajob,

Thanks for the question. Currently we don't support GridFS files, so if you are dealing with documents above 16MB you will have to load them outside of FME. This is a known issue and we are tracking user demand in advance of adding support for GridFS. Would you consider adding an idea on our Ideas forum and tell us a little bit about your use case for MongoDB and large documents? Any comments you might have will help us if and when it comes time to add this feature.

 

 

Thanks,

Nathan

Hi Nathan,

 

 

Thanks for the answer and apologies for the delay in response. I wanted to load a subset of my data into MongoDB using GridFS to see if it worked before i came back to you. Turns out there isn't much of a use case in this situation as my data was a large embedded set of documents and GridFS wouldn't allow me to query these efficiently.

 

 

I am currently looking into a solution using bucketing instead but I may also need to load large images into the database that could require GridFS - so i'll definitely remember the ideas forum if it turns out it could still be useful later on.

 

 

Reply