Skip to main content
Question

FuzzyStringComparer - many versions - many issues


sigtill
Supporter
Forum|alt.badge.img+24

Just to let anyone using the FuzzyStringComparer some hands up with FME2017 - its a real struggle to get it working as it has changed a lot and there are many versions of it. If it does not work - try to right click on it - edit and copy all the transformers to the main workspace.

11 replies

itay
Supporter
Forum|alt.badge.img+17
  • Supporter
  • April 24, 2017
Hands or heads? :)

 

 


Forum|alt.badge.img+1
I'm struggling with this right now! Any ideas to fix? I get this error:

 

 

Python Exception <AttributeError>: 'NoneType' object has no attribute 'lower'

 

Error encountered while calling function `FuzzyStringCompare'

 

f_267(PythonFactory): PythonFactory failed to process feature

 

 

 


takashi
Evangelist
  • April 27, 2017
miikamakela wrote:
I'm struggling with this right now! Any ideas to fix? I get this error:

 

 

Python Exception <AttributeError>: 'NoneType' object has no attribute 'lower'

 

Error encountered while calling function `FuzzyStringCompare'

 

f_267(PythonFactory): PythonFactory failed to process feature

 

 

 

Hi @miikamakela, the error occurs when the script tried calling the method called 'lower' on None object. The method should be called on a string object.

 

I looked at the script in the FuzzyStringComparer and found the error could occur if one of string attributes to be compared was missing actually. Make sure that both string attributes have string value.

Forum|alt.badge.img+1
takashi wrote:
Hi @miikamakela, the error occurs when the script tried calling the method called 'lower' on None object. The method should be called on a string object.

 

I looked at the script in the FuzzyStringComparer and found the error could occur if one of string attributes to be compared was missing actually. Make sure that both string attributes have string value.
Thanks @takashi I will try to fix this! I indeed can have null values in the other field - previously the tool has managed them.

 


Forum|alt.badge.img+1
takashi wrote:
Hi @miikamakela, the error occurs when the script tried calling the method called 'lower' on None object. The method should be called on a string object.

 

I looked at the script in the FuzzyStringComparer and found the error could occur if one of string attributes to be compared was missing actually. Make sure that both string attributes have string value.
actually, I still did not get this to work by fixing the strings, but the original suggestion by @sigtill to copy transformers to the main workbench and map the attributes works.

 

 


Forum|alt.badge.img+1
miikamakela wrote:
actually, I still did not get this to work by fixing the strings, but the original suggestion by @sigtill to copy transformers to the main workbench and map the attributes works.

 

 

Hi guys, I'm having issues with this again.

 

Now I got this error message:

 

 

Python Exception <UnicodeEncodeError>: 'ascii' codec can't encode characters in position 21-25: ordinal not in range(128)

 

Error encountered while calling function `FuzzyStringCompare'

 

f_274(PythonFactory): PythonFactory failed to process feature

 

A fatal error has occurred. Check the logfile above for details

 

 

The string contains text in English|Hebrew|Arabic each separated with a | pipe. This seems to be the issue.

 

 

Before, with just "normal" characters this python script has worked nicely:

 

 

import difflib

 

def FuzzyStringCompare(feature):

 

string1 = str(feature.getAttribute('FuzzyStringCompare.string1')or '').lower()

 

string2 = str(feature.getAttribute('FuzzyStringCompare.string2')or '').lower()

 

ratio = difflib.SequenceMatcher(None,string1,string2).ratio()

 

feature.setAttribute('FuzzyStringCompare.ratio',ratio)

 

 

Any suggestions how to change it to accommodate the more complex strings?

 

 


davideagle
Contributor
Forum|alt.badge.img+21
  • Contributor
  • July 7, 2017

Maybe @DaveAtSafe can help by bundling up some of the versions into a Versioned custom transformer. I'm pretty sure it was one of his originally. He certainly helped me out a while back when one of the two Fuzzy transformers disappeared altogether... It certainly looks like its seen a few edits lately.


daveatsafe
Safer
Forum|alt.badge.img+19
  • Safer
  • July 10, 2017

@miikamakela,

I recently updated the transformer to produce a comparison string as well as a ratio. It appears that this code is failing when one of the attributes is missing, and also with the Arabic and Hebrew characters. Would you mind sending me your sample strings, and I will see if I can fix the problem for you.


Forum|alt.badge.img+1
daveatsafe wrote:

@miikamakela,

I recently updated the transformer to produce a comparison string as well as a ratio. It appears that this code is failing when one of the attributes is missing, and also with the Arabic and Hebrew characters. Would you mind sending me your sample strings, and I will see if I can fix the problem for you.

Hi, my colleague adapted the python code so it works now - at least for our use case. He added UTF-8 encoding. See script below.

 

 

import difflib

 

def FuzzyStringCompare(feature):

 

string1 = (feature.getAttribute('FuzzyStringCompare.string1')or '').encode('utf-8').lower() string2 = (feature.getAttribute('FuzzyStringCompare.string2')or '').encode('utf-8').lower()

 

ratio = difflib.SequenceMatcher(None,string1,string2).ratio() feature.setAttribute('FuzzyStringCompare.ratio',ratio)

 

 


daveatsafe
Safer
Forum|alt.badge.img+19
  • Safer
  • July 27, 2017

Hi @sigtill, @

miikamakela,

I have updated the FuzzyStringComparer transformer to handle unicode strings and to fail gracefully when the compare strings are not present on the feature.

Please give it a try and let me know if you have any other problems with it.


Forum|alt.badge.img+1
daveatsafe wrote:

Hi @sigtill, @

miikamakela,

I have updated the FuzzyStringComparer transformer to handle unicode strings and to fail gracefully when the compare strings are not present on the feature.

Please give it a try and let me know if you have any other problems with it.

Thanks @DaveAtSafe. I will give it a go!

 

 


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings