Does anyone know if its possible to encrypt and de-crpyt strings in FME. So if I needed to encrypt an email address to save it to a database and decrpyt it to send email out at a later date. Is that possible?
Thanks
Sure, the AttributeCompressor with AES-256 encryption and a good password should do the trick.
+9- Contributor
david_r wrote:
Sure, the AttributeCompressor with AES-256 encryption and a good password should do the trick.
+4- Contributor
I wanted to do the same thing (I am currently using FME desktop 2018.1.0.3) but have had nothing but problems with the AttributeCompressor / decompressor with AES-256 encryption and a good password.
It compresses it fine but then will not decompress it. the AttributeDecompressor just fails.
This is very frustrating.
bilal wrote:
I wanted to do the same thing (I am currently using FME desktop 2018.1.0.3) but have had nothing but problems with the AttributeCompressor / decompressor with AES-256 encryption and a good password.
It compresses it fine but then will not decompress it. the AttributeDecompressor just fails.
This is very frustrating.
I haven't got enough details to know if this is the case here, but a common issue is that the AttributeCompressor does not return a string, but a binary value (which is shown as a string of hexadecimal values in the FME inspector).
If you write the binary value to e.g. a database string field and then read it back, the contents will probably not be the same anymore due to automatic conversion mechanisms in FME, leading the AttributeDecompressor to fail.
You should use the BinaryEncoder / BinaryDecoder to work around this issue.
+4- Contributor
david_r wrote:
I haven't got enough details to know if this is the case here, but a common issue is that the AttributeCompressor does not return a string, but a binary value (which is shown as a string of hexadecimal values in the FME inspector).
If you write the binary value to e.g. a database string field and then read it back, the contents will probably not be the same anymore due to automatic conversion mechanisms in FME, leading the AttributeDecompressor to fail.
You should use the BinaryEncoder / BinaryDecoder to work around this issue.
Hi David,
Thank you for the speedy reply. I had a look at the BinaryEncoder / BinaryDecoder however I am reluctant to use those as a pseudo encryption because they do not provide any security since they are so easy to reverse. Unless I am misunderstanding your suggestion and need to use the BinaryEncoder / BinaryDecoder in conjunction with the AttributeCompressor / decompressor.
My main use is to add very strong encryption to some database fields in a table that contains personal information, then decode it via FME as it is loaded into a cloud based asset management system.
+54
david_r wrote:
I haven't got enough details to know if this is the case here, but a common issue is that the AttributeCompressor does not return a string, but a binary value (which is shown as a string of hexadecimal values in the FME inspector).
If you write the binary value to e.g. a database string field and then read it back, the contents will probably not be the same anymore due to automatic conversion mechanisms in FME, leading the AttributeDecompressor to fail.
You should use the BinaryEncoder / BinaryDecoder to work around this issue.
Thanks, learned something new today!
@mark2atsafe Adding the BinaryEncoder / BinaryDecoder part to the AttributeCompressor documentation might help others.
bilal wrote:
Hi David,
Thank you for the speedy reply. I had a look at the BinaryEncoder / BinaryDecoder however I am reluctant to use those as a pseudo encryption because they do not provide any security since they are so easy to reverse. Unless I am misunderstanding your suggestion and need to use the BinaryEncoder / BinaryDecoder in conjunction with the AttributeCompressor / decompressor.
My main use is to add very strong encryption to some database fields in a table that contains personal information, then decode it via FME as it is loaded into a cloud based asset management system.
The BinaryEncoder/Decoder are not there to provide security, only to transform between strings and binary data for storage. The security aspect is provided by the AttributeCompressor/Decompressor. So yes, you need to use them together.
Data flow:
... -> AttributeCompressor -> BinaryEncoder -> WRITE TO DATABASE
READ FROM DATABASE -> BinaryDecoder -> AttributeDecompressor -> ...You may be able to avoid using the BinaryEncoder/Decoder if you do not store the encrypted data as a string, but as a BLOB in the database.
+50- Celebrity
bilal wrote:
I wanted to do the same thing (I am currently using FME desktop 2018.1.0.3) but have had nothing but problems with the AttributeCompressor / decompressor with AES-256 encryption and a good password.
It compresses it fine but then will not decompress it. the AttributeDecompressor just fails.
This is very frustrating.
I've had a similar issue with writing encrypted values to a text file and reading them back, specifying the encoding to be UTF-8 (on the writer) seemed to have fixed it for me.
david_r wrote:
The BinaryEncoder/Decoder are not there to provide security, only to transform between strings and binary data for storage. The security aspect is provided by the AttributeCompressor/Decompressor. So yes, you need to use them together.
Data flow:
... -> AttributeCompressor -> BinaryEncoder -> WRITE TO DATABASE
READ FROM DATABASE -> BinaryDecoder -> AttributeDecompressor -> ...You may be able to avoid using the BinaryEncoder/Decoder if you do not store the encrypted data as a string, but as a BLOB in the database.
Untested, but I suspect you could also use the TextEncoder/Decoder (rather than the BinaryEncoder/Decoder) and use Base64 rather than HEX for storage, which should be slightly more efficient, space wise.
+44- Safer
nielsgerrits wrote:
Thanks, learned something new today!
@mark2atsafe Adding the BinaryEncoder / BinaryDecoder part to the AttributeCompressor documentation might help others.
I did not know that either. I will suggest to the docs team that we add that. Thanks.
+44- Safer
david_r wrote:
I haven't got enough details to know if this is the case here, but a common issue is that the AttributeCompressor does not return a string, but a binary value (which is shown as a string of hexadecimal values in the FME inspector).
If you write the binary value to e.g. a database string field and then read it back, the contents will probably not be the same anymore due to automatic conversion mechanisms in FME, leading the AttributeDecompressor to fail.
You should use the BinaryEncoder / BinaryDecoder to work around this issue.
I wonder if we give a warning in the log file when that happens? I'm checking with the developers. If we don't warn then we should do.
mark2atsafe wrote:
I wonder if we give a warning in the log file when that happens? I'm checking with the developers. If we don't warn then we should do.
Interestingly, there seems to be some cases where FME is implicitely handling the conversion between binary and text without having to use any additional transformers, see the comment from @redgeographics below.
+44- Safer
david_r wrote:
Interestingly, there seems to be some cases where FME is implicitely handling the conversion between binary and text without having to use any additional transformers, see the comment from @redgeographics below.
Yes, sadly I think this is going to be on a writer-by-writer basis. One writer might handle the conversion, especially where the data type isn't really specific (like text) but other writers might fail. One developer view is that the BinaryEncoder is a better choice than the TextEncoder, but just because we aren't sure if the TextEncoder has any unknown limitations.
+44- Safer
bilal wrote:
Hi David,
Thank you for the speedy reply. I had a look at the BinaryEncoder / BinaryDecoder however I am reluctant to use those as a pseudo encryption because they do not provide any security since they are so easy to reverse. Unless I am misunderstanding your suggestion and need to use the BinaryEncoder / BinaryDecoder in conjunction with the AttributeCompressor / decompressor.
My main use is to add very strong encryption to some database fields in a table that contains personal information, then decode it via FME as it is loaded into a cloud based asset management system.
What database are you using? I'd be interested to know if there was a warning message when the data was written.
Reply
Rich Text Editor, editor1
Editor toolbars
Press ALT 0 for help
Helpful Members This Week
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.