Skip to main content

Hello,

I am evaluating FME for a project but have hit a hurdle trying to write to a MongoDB database. I have put together a very simple workspace that reads data from an Excel file and dumps it into MongoDB. When I run it, I get the following error:

MONGODB writer: Existing feature type 'rawData' does not contain an attribute matching the schema attribute 'customerId'. Attributes named 'customerId' on incoming features will not be written to the existing feature type

I get the same error for each attribute that I have defined. The writer succeeds in creating the collection and populating it with documents, but all the documents only have the '_id' field set.

On the writer, I've set Attribute Definition to Automatic and also given the collection name under the General tab. Otherwise I've left the defaults.

Am I missing something?

,

Hello,

I am evaluating FME for a project and put together a simple workspace that pulls data from an Excel spreadsheet and dumps them into a MongoDB collection. When I run it, I get errors like this:

MONGODB writer: Existing feature type 'rawData' does not contain an attribute matching the schema attribute 'customerId'. Attributes named 'customerId' on incoming features will not be written to the existing feature type

I get an error like this for every single attribute. The resulting MongoDB collection contains a bunch of documents (one for each row in the Excel sheet) whose only defined field is _id. Under User Attributes on the writer, I've set Attribute Definition to Automatic. I've also set the Collection name under the General tab to the already existing empty collection. Everything else is Default. Is there something I'm missing?

In Feature Type Properties > General > Table Handling: set the value to Drop and Create

The MongoDB writer is using the existing documents in the collection as a template for what attributes should exist. Since there are no documents, it doesn't think there are any attributes.

However, if you "Drop and Create" the collection then the MongoDB writer will create your attributes since it's not trying to match any existing documents.


Ok I'll try that. I'll admit it sounds odd to me. The whole point of MongoDB is that you are not forced to require every single document to have the exact same attributes. If in the future, I add a new column into my Excel spreadsheet and want to pull that into my database while preserving the existing contents, shouldn't I be able to do that?

Thanks!


Ok I'll try that. I'll admit it sounds odd to me. The whole point of MongoDB is that you are not forced to require every single document to have the exact same attributes. If in the future, I add a new column into my Excel spreadsheet and want to pull that into my database while preserving the existing contents, shouldn't I be able to do that?

Thanks!

I do agree. And we can consider changing the behaviour, though in our case we're going through another library to get to Mongo. You can kick our writer in raw-JSON mode, in which case you can prep the JSON ahead of time to send in, and then anything can be put in. Having said that it does seem like we should look to loosen this restriction off. Thanks for the feedback, I'll share it with the team.

 

 


I do agree. And we can consider changing the behaviour, though in our case we're going through another library to get to Mongo. You can kick our writer in raw-JSON mode, in which case you can prep the JSON ahead of time to send in, and then anything can be put in. Having said that it does seem like we should look to loosen this restriction off. Thanks for the feedback, I'll share it with the team.

 

 

How can I follow up on whether / when this behavior was changed? it is crucial for my use case.

 

 


I do agree. And we can consider changing the behaviour, though in our case we're going through another library to get to Mongo. You can kick our writer in raw-JSON mode, in which case you can prep the JSON ahead of time to send in, and then anything can be put in. Having said that it does seem like we should look to loosen this restriction off. Thanks for the feedback, I'll share it with the team.

 

 

would selecting 'dynamic attributes' on the writer force it to take all attributes?

Hi @b4. Development created a workaround workspace. It mangles together a JSON attribute using all of the FME feature attributes, and then that attribute is specified on the MongoDB writer.

 


Hi @NatalieAtSafe,

 

I assume that this is the best known solution? Are there plans to modify this any time soon?

 

 

The environment is beyond my level of understanding, so I am opting to find another solution.


Hi @NatalieAtSafe,

 

I assume that this is the best known solution? Are there plans to modify this any time soon?

 

 

The environment is beyond my level of understanding, so I am opting to find another solution.

Hi @b4. There may be other workarounds, yet they may be similar in their complexity.

Reply