Does anyone know if its possible to encrypt and de-crpyt strings in FME. So if I needed to encrypt an email address to save it to a database and decrpyt it to send email out at a later date. Is that possible?
Thanks
This post is closed to further activity.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.
+9- Contributor
- 167 replies
Â
Would the attribute Compresser and Decompressor be the way to do it?
- 8394 replies
Sure, the AttributeCompressor with AES-256 encryption and a good password should do the trick.
+9- Contributor
- 167 replies
Sure, the AttributeCompressor with AES-256 encryption and a good password should do the trick.
Â
+4- Contributor
- 14 replies
I wanted to do the same thing (I am currently using FME desktop 2018.1.0.3) but have had nothing but problems with the AttributeCompressor / decompressor with AES-256 encryption and a good password.
It compresses it fine but then will not decompress it. the AttributeDecompressor just fails.
This is very frustrating.
- 8394 replies
I wanted to do the same thing (I am currently using FME desktop 2018.1.0.3) but have had nothing but problems with the AttributeCompressor / decompressor with AES-256 encryption and a good password.
It compresses it fine but then will not decompress it. the AttributeDecompressor just fails.
This is very frustrating.
I haven't got enough details to know if this is the case here, but a common issue is that the AttributeCompressor does not return a string, but a binary value (which is shown as a string of hexadecimal values in the FME inspector).
If you write the binary value to e.g. a database string field and then read it back, the contents will probably not be the same anymore due to automatic conversion mechanisms in FME, leading the AttributeDecompressor to fail.
You should use the BinaryEncoder / BinaryDecoder to work around this issue.
+4- Contributor
- 14 replies
I haven't got enough details to know if this is the case here, but a common issue is that the AttributeCompressor does not return a string, but a binary value (which is shown as a string of hexadecimal values in the FME inspector).
If you write the binary value to e.g. a database string field and then read it back, the contents will probably not be the same anymore due to automatic conversion mechanisms in FME, leading the AttributeDecompressor to fail.
You should use the BinaryEncoder / BinaryDecoder to work around this issue.
Hi David,
Thank you for the speedy reply. I had a look at the BinaryEncoder / BinaryDecoder however I am reluctant to use those as a pseudo encryption because they do not provide any security since they are so easy to reverse. Unless I am misunderstanding your suggestion and need to use the BinaryEncoder / BinaryDecoder in conjunction with the AttributeCompressor / decompressor.
My main use is to add very strong encryption to some database fields in a table that contains personal information, then decode it via FME as it is loaded into a cloud based asset management system.
+60- 2940 replies
I haven't got enough details to know if this is the case here, but a common issue is that the AttributeCompressor does not return a string, but a binary value (which is shown as a string of hexadecimal values in the FME inspector).
If you write the binary value to e.g. a database string field and then read it back, the contents will probably not be the same anymore due to automatic conversion mechanisms in FME, leading the AttributeDecompressor to fail.
You should use the BinaryEncoder / BinaryDecoder to work around this issue.
Thanks, learned something new today!
@mark2atsafe Adding the BinaryEncoder / BinaryDecoder part to the AttributeCompressor documentation might help others.
- 8394 replies
Hi David,
Thank you for the speedy reply. I had a look at the BinaryEncoder / BinaryDecoder however I am reluctant to use those as a pseudo encryption because they do not provide any security since they are so easy to reverse. Unless I am misunderstanding your suggestion and need to use the BinaryEncoder / BinaryDecoder in conjunction with the AttributeCompressor / decompressor.
 My main use is to add very strong encryption to some database fields in a table that contains personal information, then decode it via FME as it is loaded into a cloud based asset management system.
The BinaryEncoder/Decoder are not there to provide security, only to transform between strings and binary data for storage. The security aspect is provided by the AttributeCompressor/Decompressor. So yes, you need to use them together.
Data flow:
... -> AttributeCompressor -> BinaryEncoder -> WRITE TO DATABASE
READ FROM DATABASE -> BinaryDecoder -> AttributeDecompressor -> ...You may be able to avoid using the BinaryEncoder/Decoder if you do not store the encrypted data as a string, but as a BLOB in the database.
+60- Celebrity
- 3702 replies
I wanted to do the same thing (I am currently using FME desktop 2018.1.0.3) but have had nothing but problems with the AttributeCompressor / decompressor with AES-256 encryption and a good password.
It compresses it fine but then will not decompress it. the AttributeDecompressor just fails.
This is very frustrating.
I've had a similar issue with writing encrypted values to a text file and reading them back, specifying the encoding to be UTF-8 (on the writer) seemed to have fixed it for me.
- 8394 replies
The BinaryEncoder/Decoder are not there to provide security, only to transform between strings and binary data for storage. The security aspect is provided by the AttributeCompressor/Decompressor. So yes, you need to use them together.
Data flow:
... -> AttributeCompressor -> BinaryEncoder -> WRITE TO DATABASE
READ FROM DATABASE -> BinaryDecoder -> AttributeDecompressor -> ...You may be able to avoid using the BinaryEncoder/Decoder if you do not store the encrypted data as a string, but as a BLOB in the database.
Untested, but I suspect you could also use the TextEncoder/Decoder (rather than the BinaryEncoder/Decoder) and use Base64 rather than HEX for storage, which should be slightly more efficient, space wise.
+56- Safer
- 2554 replies
Thanks, learned something new today!
@mark2atsafe Adding the BinaryEncoder / BinaryDecoder part to the AttributeCompressor documentation might help others.
I did not know that either. I will suggest to the docs team that we add that. Thanks.
+56- Safer
- 2554 replies
I haven't got enough details to know if this is the case here, but a common issue is that the AttributeCompressor does not return a string, but a binary value (which is shown as a string of hexadecimal values in the FME inspector).
If you write the binary value to e.g. a database string field and then read it back, the contents will probably not be the same anymore due to automatic conversion mechanisms in FME, leading the AttributeDecompressor to fail.
You should use the BinaryEncoder / BinaryDecoder to work around this issue.
I wonder if we give a warning in the log file when that happens? I'm checking with the developers. If we don't warn then we should do.
- 8394 replies
I wonder if we give a warning in the log file when that happens? I'm checking with the developers. If we don't warn then we should do.
Interestingly, there seems to be some cases where FME is implicitely handling the conversion between binary and text without having to use any additional transformers, see the comment from @redgeographics below.
+56- Safer
- 2554 replies
Interestingly, there seems to be some cases where FME is implicitely handling the conversion between binary and text without having to use any additional transformers, see the comment from @redgeographics below.
Yes, sadly I think this is going to be on a writer-by-writer basis. One writer might handle the conversion, especially where the data type isn't really specific (like text) but other writers might fail. One developer view is that the BinaryEncoder is a better choice than the TextEncoder, but just because we aren't sure if the TextEncoder has any unknown limitations.
+56- Safer
- 2554 replies
Hi David,
Thank you for the speedy reply. I had a look at the BinaryEncoder / BinaryDecoder however I am reluctant to use those as a pseudo encryption because they do not provide any security since they are so easy to reverse. Unless I am misunderstanding your suggestion and need to use the BinaryEncoder / BinaryDecoder in conjunction with the AttributeCompressor / decompressor.
My main use is to add very strong encryption to some database fields in a table that contains personal information, then decode it via FME as it is loaded into a cloud based asset management system.
What database are you using? I'd be interested to know if there was a warning message when the data was written.