This might not be the best way to do it, but I use a work around to get the attachments if there are more than one in the email when using the Automations in FME Server.
First I get the attachment ID from email.attachment in $(FME_TOPIC_MESSAGE) JSON using regex in the string searcher.
(?<=emailattachments\\\\)(.*)(?=\\\\)
Then I use a directory and file pathnames feature reader to return the attachment files for that specific ID.
C:/ProgramData/Safe Software/FME Server/resources/system/temp/emailattachments/@Value(_attachment_id)
Now you will have all of the attachment file paths.
This might not be the best way to do it, but I use a work around to get the attachments if there are more than one in the email when using the Automations in FME Server.
First I get the attachment ID from email.attachment in $(FME_TOPIC_MESSAGE) JSON using regex in the string searcher.
(?<=emailattachments\\\\)(.*)(?=\\\\)
Then I use a directory and file pathnames feature reader to return the attachment files for that specific ID.
C:/ProgramData/Safe Software/FME Server/resources/system/temp/emailattachments/@Value(_attachment_id)
Now you will have all of the attachment file paths.
Thanks for your answer. Instead of your regex , I've used a FilenamePartExtractor, but neither solution solves my problem.
I think that the only way to get rid of it would be to change the name of the subfolder where FME download the attachment, but I'don't how we can configure it or if it's possible.
The content of the topic should also contain the path to the attachment as a string in the json.
{ "email_publisher_subject": "FW: what??", "email_publisher_sent": "Tue Oct 22 14:22:20 UTC 2019", "email_publisher_to": "test@<somehost>", "email_publisher_from": "someone@someone.com", "email_publisher_content_type{1}": "text/plain; charset=\"utf-8\"", "email_publisher_content_type{0}": "text/html; charset=\"utf-8\"", "ws_topic": "GETEMAIL", "email_publisher_received": "Tue Oct 22 14:22:21 UTC 2019", "email_publisher_attachment{0}": "/data/fmeserver/resources/data/emailattachemtns/20191022142221-test@<somehost>-FW_what/attachemt.txt",
Note the subject has a ':' and two '?', however, the Attachement path points correctly to the file.
Do you see the same behaviour when monitoring your topic? or is it mire like you have a 'ä' or 'ö' in the email and this is what is causing the problem?
Note: I am running on a Linux FME Server so if you are on windows perhaps it's different
In the notification, the encoding seems correct :
{
"imap_publisher_received": "Wed Oct 23 16:28:20 CEST 2019",
"imap_publisher_sent": "Wed Oct 23 16:28:19 CEST 2019",
"imap_publisher_attachment{0}": "C:\\ProgramData\\Safe Software\\FME Server\\resources\\temp\\NEPTUNE\\mailBierne\\20191023162820-Neptune_IMAP-Résultats_partiel_R2019.10.004.14\\XXXX.xls",
"ws_topic": "NEPTUNE",
"imap_publisher_subject": "Résultats partiel R2019.10.004.14",
"imap_publisher_from": "*****@*****.***",
"imap_publisher_content{0}": "<html><head></head><body><br></body></html>",
"imap_publisher_content{1}": "\r\n",
"imap_publisher_to": "****@*****.***",
"imap_publisher_content_type{0}": "text/html; charset=utf-8",
"fns_type": "imap_publisher",
"imap_publisher_content_type{1}": "text/plain; charset=utf-8"
}
In FME Server, I see the directory created in the Resources correctly :
20191023162820-Neptune_IMAP-Résultats_partiel_R2019.10.004.14
But I still have the error when my workspace runs :
Excel Reader: Failed to open the dataset 'C:\ProgramData\Safe Software\FME Server\resources\temp\NEPTUNE\mailBierne\20191023162122-Neptune_IMAP-Résultats_partiel_R2019.10.004.14\XXXX.xls'. Error message was 'can't open file for reading'
The problem seems to occur between the Email Publisher and the FME Server Core.
In the imap.log, the encoding is correct :
Wed-23-Oct-2019 04:28:44.472 PM INFORM Neptune IMAP 809008 : neptune-bierne : Processed email with subject "Résultats partiel R2019.10.004.14" with 1 attachment(s). Sending event...
in the fmeserver.log, the encoding is wrong :
Wed-23-Oct-2019 04:28:45.628 PM INFORM Engine-*********_Engine1@1933 401868 : Job 1934: Sending job to FME Engine **********@1933 on host **********. Request is >"NEPTUNE/******.fmw" -XML_PARAMETER <?xml version="1.1" encoding="UTF-8"?><TRANSFORMATION_REQUEST><PUBLISHED_PARAMETER name="*******">******</PUBLISHED_PARAMETER><PUBLISHED_PARAMETER name="SourceDataset_XLSXR">""C:\ProgramData\Safe Software\FME Server\resources\temp\NEPTUNE\mailBierne\20191023162820-Neptune_IMAP-Résultats_partiel_R2019.10.004.14\XXXX.xls""</PUBLISHED_PARAMETER></TRANSFORMATION_REQUEST>
In the notification, the encoding seems correct :
{
"imap_publisher_received": "Wed Oct 23 16:28:20 CEST 2019",
"imap_publisher_sent": "Wed Oct 23 16:28:19 CEST 2019",
"imap_publisher_attachment{0}": "C:\\ProgramData\\Safe Software\\FME Server\\resources\\temp\\NEPTUNE\\mailBierne\\20191023162820-Neptune_IMAP-Résultats_partiel_R2019.10.004.14\\XXXX.xls",
"ws_topic": "NEPTUNE",
"imap_publisher_subject": "Résultats partiel R2019.10.004.14",
"imap_publisher_from": "*****@*****.***",
"imap_publisher_content{0}": "<html><head></head><body><br></body></html>",
"imap_publisher_content{1}": "\r\n",
"imap_publisher_to": "****@*****.***",
"imap_publisher_content_type{0}": "text/html; charset=utf-8",
"fns_type": "imap_publisher",
"imap_publisher_content_type{1}": "text/plain; charset=utf-8"
}
In FME Server, I see the directory created in the Resources correctly :
20191023162820-Neptune_IMAP-Résultats_partiel_R2019.10.004.14
But I still have the error when my workspace runs :
Excel Reader: Failed to open the dataset 'C:\ProgramData\Safe Software\FME Server\resources\temp\NEPTUNE\mailBierne\20191023162122-Neptune_IMAP-Résultats_partiel_R2019.10.004.14\XXXX.xls'. Error message was 'can't open file for reading'
The problem seems to occur between the Email Publisher and the FME Server Core.
In the imap.log, the encoding is correct :
Wed-23-Oct-2019 04:28:44.472 PM INFORM Neptune IMAP 809008 : neptune-bierne : Processed email with subject "Résultats partiel R2019.10.004.14" with 1 attachment(s). Sending event...
in the fmeserver.log, the encoding is wrong :
Wed-23-Oct-2019 04:28:45.628 PM INFORM Engine-*********_Engine1@1933 401868 : Job 1934: Sending job to FME Engine **********@1933 on host **********. Request is >"NEPTUNE/******.fmw" -XML_PARAMETER <?xml version="1.1" encoding="UTF-8"?><TRANSFORMATION_REQUEST><PUBLISHED_PARAMETER name="*******">******</PUBLISHED_PARAMETER><PUBLISHED_PARAMETER name="SourceDataset_XLSXR">""C:\ProgramData\Safe Software\FME Server\resources\temp\NEPTUNE\mailBierne\20191023162820-Neptune_IMAP-Résultats_partiel_R2019.10.004.14\XXXX.xls""</PUBLISHED_PARAMETER></TRANSFORMATION_REQUEST>
Right, OK so yeah this looks to be encoding specific like you say. I would report this to Safe via tech support.
An interesting test would be to create a folder/file with this character in the Data folder and then set up a simple workspace to test to see if the FME Server will read the file with the Excel Reader.
That would also help Safe track down the issue.
One thing which you could try is changing the system encoding on the FME Server machine to one that supports this character. It's not a great solution but it might be all you can do until the problem is fixed.
perhaps @lenaatsafe can help with this?
Right, OK so yeah this looks to be encoding specific like you say. I would report this to Safe via tech support.
An interesting test would be to create a folder/file with this character in the Data folder and then set up a simple workspace to test to see if the FME Server will read the file with the Excel Reader.
That would also help Safe track down the issue.
One thing which you could try is changing the system encoding on the FME Server machine to one that supports this character. It's not a great solution but it might be all you can do until the problem is fixed.
perhaps @lenaatsafe can help with this?
Thanks for your answer, but i'm not sure to understand well your suggestion. I'v tried to change the encoding of my reader but it doesn't change anything.
How can I change the system encoding on the FME Server machine?
Thanks for your answer, but i'm not sure to understand well your suggestion. I'v tried to change the encoding of my reader but it doesn't change anything.
How can I change the system encoding on the FME Server machine?
My suggestion was to put an excel file in the data folder which contained the é character in the file name (or folder name) - then create simple workspace which reads the file. This would help to understand which part of FME server is not recognising the é character properly.
To change the system encoding on the FME Server will be in the Control Panel > Region > Administrative tab > Change System Locale.
Just keep in mind that this could have other effects and you should know how to switch back. I would only do it as a test to see if it helps with this problem
