Skip to main content

Hi! I have a HTTPCaller transformer in workspace that downloads 80 MB of data in the same feature attribute before failing. Once it fails, FME desktop attempted to write a feature in the log dispaly and display. The data it tried to display is so big such that the FME desktop crashed. Is there a way to still let it write, but truncate the size of each attribute?

Thank you!

Hi @sui_huang,

Could you share your log file to analyze better?

Thanks,

Danilo


Try the following code in a PythonCaller, modify MAX_LENGTH as needed:

import fmeobjects

MAX_LENGTH = 10

def AttributeLengthTrimmer(feature):
    for attr in feature.getAllAttributeNames():
        value = feature.getAttribute(attr)
        if value:
            feature.setAttribute(attr, str(value) :MAX_LENGTH])

Hi @sui_huang,

Could you share your log file to analyze better?

Thanks,

Danilo

Hi @danilo_inovacao

 

unfortunately I cannot upload the log file because the information inside maybe confidential.

 

 

The lines before the logging of the huge feature in log file is as following, and they are triggered by the Terminator transformer.

 

 

2017-10-12 11:30:59| 13.8| 4.6|FATAL |The below feature caused the translation to be terminated

 

2017-10-12 11:30:59| 13.8| 0.0|STATS |Storing feature(s) to FME feature store file `C:\\Desktop\\Report_log.ffs'

 

2017-10-12 11:30:59| 13.8| 0.0|INFORM|+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

 

2017-10-12 11:30:59| 13.8| 0.0|INFORM|Feature Type: `AttributeCreator_OUTPUT'

 

2017-10-12 11:30:59| 13.8| 0.0|INFORM|Attribute(encoded: utf-16) :.....

 

 


Try the following code in a PythonCaller, modify MAX_LENGTH as needed:

import fmeobjects

MAX_LENGTH = 10

def AttributeLengthTrimmer(feature):
    for attr in feature.getAllAttributeNames():
        value = feature.getAttribute(attr)
        if value:
            feature.setAttribute(attr, str(value) :MAX_LENGTH])
Hi @david_r Instead of using Python script, I tried put an AttributeCreator in between and use "Left" string function inside to truncate the attribute. It works. It is the same idea as you proposed but more specific to my situation because I know which attribute is long.

 

Thank you.

 


Reply