Question

I'm facing an error of features will be logged and skipped.. please help me asap as I'm enable to write my entire data on postgres.

  • 4 September 2022
  • 9 replies
  • 14 views

Hi,

I'm having this error when I try to write the values in postgres through fme tool. The attribute has 6 possible values (0-6). what is the best way of solving this problems.

Value of attribute 'n_lane' could not be converted to type 'int2'. Feature will be logged and skipped.

And then at last the tool throws an error of insufficient memory available.

I mean I really don't understand how to solve this issue..

Please someone help me out with this.

I'm attaching a photo so that you can have a clear vision of this error.​


9 replies

Userlevel 6
Badge +32

Skipped feature: It looks like the attribute "n_lane" with the value 101036196 is being written to a postgres column of the type int2 which is probably a smallint with a 2 byte storage size. The range of a smallint type is -32768 to +32767 so 101036196 does not fit so the feature got skipped. See the postgres documentation for more information on numeric types.

 

Insufficient Memory: You process needs more more RAM then you have available. As suggested in the log you can check the article Adjusting Memory Resources.

Personally, having 32GB RAM in my workstation, if I have memory related issues my solution is almost always to redesign the workspace. But without knowing more about your process it is hard to say what your solution should be.

Badge +20

On top of what @nielsgerrits​ wrote, Insufficient Memory sometimes means that FME also ran out of disk space to write the cache (when the RAM filled up).

Skipped feature: It looks like the attribute "n_lane" with the value 101036196 is being written to a postgres column of the type int2 which is probably a smallint with a 2 byte storage size. The range of a smallint type is -32768 to +32767 so 101036196 does not fit so the feature got skipped. See the postgres documentation for more information on numeric types.

 

Insufficient Memory: You process needs more more RAM then you have available. As suggested in the log you can check the article Adjusting Memory Resources.

Personally, having 32GB RAM in my workstation, if I have memory related issues my solution is almost always to redesign the workspace. But without knowing more about your process it is hard to say what your solution should be.

Redesigning the workspace as in I have to write each and everything again? ?Bcoz my tool has been updated last week..and I have 16gb ram in my system with 64 bit OS..​can you further help me in this.

Userlevel 6
Badge +32

Redesigning the workspace as in I have to write each and everything again? ?Bcoz my tool has been updated last week..and I have 16gb ram in my system with 64 bit OS..​can you further help me in this.

Did you run it with Feature Caching on or off?

Redesigning the workspace as in I have to write each and everything again? ?Bcoz my tool has been updated last week..and I have 16gb ram in my system with 64 bit OS..​can you further help me in this.

The version of Fme that I'm using doesn't have this option of feature caching..so what should I do further?​

Userlevel 6
Badge +32

Redesigning the workspace as in I have to write each and everything again? ?Bcoz my tool has been updated last week..and I have 16gb ram in my system with 64 bit OS..​can you further help me in this.

Check how much free diskspace you have, as @caracadrian​ pointed out.

Userlevel 5
Badge +25

Redesigning the workspace as in I have to write each and everything again? ?Bcoz my tool has been updated last week..and I have 16gb ram in my system with 64 bit OS..​can you further help me in this.

You don't necessarily have to rewrite the whole workspace, but it wouldn't hurt to take a closer look at it. From your screenshots it looks like you're trying to merge two datasets with a FeatureMerger, if both sets are large this can be a memory hog.

 

Also, which version of FME are you using? Later versions tend to be better when it comes to performance / memory usage so you may want to consider upgrading (and then updating transformers in your workspace to make use of the new functionality)

Redesigning the workspace as in I have to write each and everything again? ?Bcoz my tool has been updated last week..and I have 16gb ram in my system with 64 bit OS..​can you further help me in this.

I'm using fme 2015..on 16gb ram ..I clear all the cache and temporary written in postgres and arccatalog..but still getting that error of memory usage.Clearly, can't understand what to do now.​

Userlevel 5
Badge +25

Redesigning the workspace as in I have to write each and everything again? ?Bcoz my tool has been updated last week..and I have 16gb ram in my system with 64 bit OS..​can you further help me in this.

There's a few things you can do:

  • If possible, upgrade to a newer version of FME. FME 2015 is technically not supported anymore as it is 7 years old. If you upgrade FME and upgrade your transformers (which may be a tedious task) you should see a performance increase.
  • Take a look at your process and see if that could be done smarter. If you can post some screenshots here we might be able to offer you advice.
  • Check, as has been pointed out by @caracadrian​ and @nielsgerrits​ , how much free disk space you have on the drive where your temp files are being written. If that's too low, see if you can increase that.

Reply