The CRCCalculator transformer will create an MD5 hash
Hello,
If you want to stick with python, this PythonCaller with following code is working fine:
import fme
import fmeobjects
import hashlib, os
def getHash(feature):
mystring = feature.getAttribute('concatenated')
# Assumes the default UTF-8
hash_object = hashlib.md5(mystring.encode())
print(hash_object.hexdigest())
Regards,
Larry
Hi
You could look at the CRCCalculator, but if you really require MD5 I think Python is the way to go. Here's a working example that also handles the case where mystring returns None (missing attribute):
import fmeobjects
import hashlib, os
class getHash(object):
def input(self,feature):
mystring = feature.getAttribute('concatenated')
if mystring:
hash_object = hashlib.md5(unicode(mystring))
hash = hash_object.hexdigest()
else:
hash = ''
feature.setAttribute('md5', hash)
self.pyoutput(feature)
Remember to keep in mind that with any hashing algorithm collisions can occur, see here for a discussion around the CRCCalculator in particular.
David
The CRCCalculator transformer will create an MD5 hash
Is this only starting FME 2016? I couldn't find anything that resembled MD5 using FME 2015.1.1.
Wow, everyone got right onto this one! Mark is (of course!) correct but has an FME 2016 dependency.
Since I went round on it while everyone else was also doing so I relearned hashlib in Python 3 - below 
.
# hashlib_example.py - Python 3
# -*- coding: utf-8 -*-
import hashlib
name_str = 'Praha-východ'
as_bytes = name_str.encode('utf-8')
my_hash = hashlib.md5()
my_hash.update(as_bytes)
my_hash_hex_str = my_hash.hexdigest(
I am running FME Server 2020.2 and have an automation setup for email polling (IMAP) that triggers a workspace when new emails arrive. For whatever reason when a new email arrives, FME Server passes an empty email.body key and stores the text in the email body as an attachment.
This inbox is a shared inbox in Office365. Any ideas as to why it would be parsing the email body as an attachment?
Here is an example of the FME_TOPIC_MESSAGE:
{
"email.to": "REDACTED",
"email.subject": "OPUS solution : JTT_PECOS_1.20o OP1606798966776",
"email.attachment": "C:\\ProgramData\\Safe Software\\FME Server\\resources\\system\\temp\\emailattachments\\20201130230450-OPUS_Result_Parsing-OPUS_solution__JTT_PECOS_1.20o_OP1606798966776\\attachment_0",
"email.received": "Mon Nov 30 23:04:50 CST 2020",
"source": "imap",
"time": "2020-11-30T23:05:49-06:00",
"email.from": "opus@ngs.noaa.gov",
"email.sent": "Mon Nov 30 23:04:45 CST 2020",
"email.attachments": "C:\\ProgramData\\Safe Software\\FME Server\\resources\\system\\temp\\emailattachments\\20201130230450-OPUS_Result_Parsing-OPUS_solution__JTT_PECOS_1.20o_OP1606798966776"
}
The email body is read as an attachment labeled "attachment_0" in the email attachments folder. I am successfully parsing this attachment, but wondered why it would be reading the body as an attachment rather than in the email.body key?
Great, it works.
I tried last week the CRCCalculator but I got 2 same codes back but have different strings into the attributes.
Now with the python it works perfect.
Note: in switzerland and other german speaking countries we have some special characters (ä, ö, ü). The python hash create can't work with this. First I have to replace them with a stringreplacer (ae, oe, ue) and then it works perfect.
Thanks very much for your support and the answers.
Hi
For info, the Python hashlib works with all kinds of international characters, you just have to be careful about the encoding (as with anything in Python 2.x). To wit:
>>> import hashlib
>>> text = 'Mein kleiner grüner Kaktus steht draußen am Balkon'
>>> my_hash = hashlib.md5(text)
>>> my_hash.hexdigest()
'd726676d1532ae8328c066dae8be8960'
Maybe try casting your string to Unicode before sending it off to hashlib, as in my example code above.
David
what do I need to fix to get it to write to a field called md. when I run it I get the following error
Python Exception <TypeError>: Could not convert attribute value to a supported attribute type.
Error encountered while calling function `getHash'
PythonFactory failed to process feature
import fme
import fmeobjects
import hashlib, os
def getHash(feature):
mystring = feature.getAttribute('path_windows')
# Assumes the default UTF-8
hash_object = hashlib.md5(mystring.encode())
print(hash_object.hexdigest())
feature.setAttribute("md", hash_object)
