Solved

Is there a way to limit the logging of a long feature attribute value?

  • 12 October 2017
  • 4 replies
  • 3 views

Badge

Hi! I have a HTTPCaller transformer in workspace that downloads 80 MB of data in the same feature attribute before failing. Once it fails, FME desktop attempted to write a feature in the log dispaly and display. The data it tried to display is so big such that the FME desktop crashed. Is there a way to still let it write, but truncate the size of each attribute?

Thank you!

icon

Best answer by david_r 12 October 2017, 09:51

View original

4 replies

Userlevel 4
Badge +30

Hi @sui_huang,

Could you share your log file to analyze better?

Thanks,

Danilo

Userlevel 4

Try the following code in a PythonCaller, modify MAX_LENGTH as needed:

import fmeobjects

MAX_LENGTH = 10

def AttributeLengthTrimmer(feature):
    for attr in feature.getAllAttributeNames():
        value = feature.getAttribute(attr)
        if value:
            feature.setAttribute(attr, str(value)[:MAX_LENGTH])
Badge

Hi @sui_huang,

Could you share your log file to analyze better?

Thanks,

Danilo

Hi @danilo_inovacao

 

unfortunately I cannot upload the log file because the information inside maybe confidential.

 

 

The lines before the logging of the huge feature in log file is as following, and they are triggered by the Terminator transformer.

 

 

2017-10-12 11:30:59| 13.8| 4.6|FATAL |The below feature caused the translation to be terminated

 

2017-10-12 11:30:59| 13.8| 0.0|STATS |Storing feature(s) to FME feature store file `C:\\Desktop\\Report_log.ffs'

 

2017-10-12 11:30:59| 13.8| 0.0|INFORM|+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

 

2017-10-12 11:30:59| 13.8| 0.0|INFORM|Feature Type: `AttributeCreator_OUTPUT'

 

2017-10-12 11:30:59| 13.8| 0.0|INFORM|Attribute(encoded: utf-16) :.....

 

 

Badge

Try the following code in a PythonCaller, modify MAX_LENGTH as needed:

import fmeobjects

MAX_LENGTH = 10

def AttributeLengthTrimmer(feature):
    for attr in feature.getAllAttributeNames():
        value = feature.getAttribute(attr)
        if value:
            feature.setAttribute(attr, str(value)[:MAX_LENGTH])
Hi @david_r Instead of using Python script, I tried put an AttributeCreator in between and use "Left" string function inside to truncate the attribute. It works. It is the same idea as you proposed but more specific to my situation because I know which attribute is long.

 

Thank you.

 

Reply