Question

PythonCaller: Use logMessageString Problems with Encoding

  • 18 September 2013
  • 7 replies
  • 5 views

Hello!

 

 

I'm using the PythonCaller to write lines into the log message.

 

I use logMessage String and with data that has mutated vowels I get following error message:

 

 

 

Python Exception <UnicodeEncodeError>: 'ascii' codec can't encode character u'\\xfc' in position 67: ordinal not in range(128)

 

 

 

What can I do that I can write also data with ä,ü,ö to the log message?

 

 

Thanks for your help!

 

 

Isabell

7 replies

Userlevel 2
Badge +17

Hi Isabell,

 

 

  I also often encounter such a situation. My environment is Japanese locale.

My workaround is to test whether the character string is a unicode instance or not, and to encode the string by the default encoding if it's unicode.

# Example
import fmeobjects
import locale
class FeatureProcessor(object):
    def __init__(self):
        loc = locale.getdefaultlocale()
        self.enc = loc[1] # save the defualt encoding
        self.logger = fmeobjects.FMELogFile()

    def input(self, feature):
        s = feature.getAttribute('attr')
        if isinstance(s, unicode):
            self.logger.logMessageString(s.encode(self.enc))
        else:
            self.logger.logMessageString(str(s))

    def close(self):
        pass

I'm not sure whether this workaround works fine in your locale.

Hope more suitable solution will be provided.

Reference link: Unicode HOWTO http://docs.python.org/2/howto/unicode.html

 

 

Takashi
Userlevel 4
Hi,

 

 

it's a bit tricky, because text attributes in FME are usual encoded as Unicode, where as the logMessageString() function seems to expect Latin1 encoding, for some unknown reason.

 

 

But try this:

 

 

log_text = feature.getAttribute("my_text").encode("latin1")

 

FMELogFile().logMessageString(log_text, fmeobjects.FME_WARN)

 

 

The important part is in bold. Please let us know if this works for you.

 

 

David
Userlevel 4
@Takashi: for what it's worth, your suggestion works find with my Swiss locale settings.

 

 

I like your more general approach. Thanks for sharing!

 

 

David
Hello!

 

 

I tried Takashi's suggestion and it works fine!!!

 

 

Thank you both very much for your help!

 

 

Regards

 

 

Isabell

 

Badge +5

Start your python script with the coding and for each variable you can add (@var).encode('utf-8')

# -*- coding: utf-8 -*-
Userlevel 2
Badge +17

Hi there, @isabell, @david_r, @julien,

I found this description in the FME 2016.1.1.0 "What's New" list.

-----------------------------
FME 2016.1.1.0 16593 20160620
-----------------------------
Python: Modified logMessage() and getMessage() to allow mixed unicode and str values in the message parameters. Fixed encoding of source code passed from PythonFactory to Python interpreter. (PR#70683)
Confirmation. This script within a PythonCaller (FME 2016.1.1 build 16609) was executed without error :-)
import fmeobjects
class FeatureProcessor(object):
    def __init__(self):
        self.logger = fmeobjects.FMELogFile()
        
    def input(self, feature):
        msg = feature.getAttribute("attr")
        if isinstance(msg, unicode): # check if the value is a unicode instance
            self.logger.logMessageString("the value is a unicode instance")
        self.logger.logMessageString(msg)
Userlevel 4

Hi there, @isabell, @david_r, @julien,

I found this description in the FME 2016.1.1.0 "What's New" list.

-----------------------------
FME 2016.1.1.0 16593 20160620
-----------------------------
Python: Modified logMessage() and getMessage() to allow mixed unicode and str values in the message parameters. Fixed encoding of source code passed from PythonFactory to Python interpreter. (PR#70683)
Confirmation. This script within a PythonCaller (FME 2016.1.1 build 16609) was executed without error :-)
import fmeobjects
class FeatureProcessor(object):
    def __init__(self):
        self.logger = fmeobjects.FMELogFile()
        
    def input(self, feature):
        msg = feature.getAttribute("attr")
        if isinstance(msg, unicode): # check if the value is a unicode instance
            self.logger.logMessageString("the value is a unicode instance")
        self.logger.logMessageString(msg)
Hi Takashi, thanks! Yes, I saw that too, excellent news indeed.

 

 

Reply