Skip to main content
Question

FMEServer 2019 - Email publication - unable to read attachment

K
Forum|alt.badge.img

I'm using FME server to execute a workspace when an email arrives. I have configured a notification and it works fine, my workspace is launched.

 

My workspace fails sometimes when I try to read an attachment in my workspace. The error message indicates that the file can't be read. This problem occurs because FME server downloads the file in a temporary folder that has the same name as the email subject. If this subject contains special characters (such as ?), the workspace can't access to that directory, due to encoding problems.

Is there a way to configure the directory in which the attachement are downloaded? I've tried to use the "Download Attachments To" option while creating may Email Notification, but Fme Server stills create subfolder.

 

I've also tried to use Automations, but it doesn't work because I may receive several attachments in the same email, and automation is unable to process multiple attachments.

 

How can I solve my problem?

7 replies

warrendev
Enthusiast
Forum|alt.badge.img+23
  • warrendevEnthusiast
  • Enthusiast
  • October 18, 2019

This might not be the best way to do it, but I use a work around to get the attachments if there are more than one in the email when using the Automations in FME Server.

 

 

First I get the attachment ID from email.attachment in $(FME_TOPIC_MESSAGE) JSON using regex in the string searcher.

(?<=emailattachments\\\\)(.*)(?=\\\\)

 

 

Then I use a directory and file pathnames feature reader to return the attachment files for that specific ID.

 

C:/ProgramData/Safe Software/FME Server/resources/system/temp/emailattachments/@Value(_attachment_id)

 

 

Now you will have all of the attachment file paths.

 

 

K
Forum|alt.badge.img
  • kukur
  • Author
  • October 22, 2019
warrendev wrote:

This might not be the best way to do it, but I use a work around to get the attachments if there are more than one in the email when using the Automations in FME Server.

 

 

First I get the attachment ID from email.attachment in $(FME_TOPIC_MESSAGE) JSON using regex in the string searcher.

(?<=emailattachments\\\\)(.*)(?=\\\\)

 

 

Then I use a directory and file pathnames feature reader to return the attachment files for that specific ID.

 

C:/ProgramData/Safe Software/FME Server/resources/system/temp/emailattachments/@Value(_attachment_id)

 

 

Now you will have all of the attachment file paths.

 

 

Thanks for your answer. Instead of your regex , I've used a FilenamePartExtractor, but neither solution solves my problem.

I think that the only way to get rid of it would be to change the name of the subfolder where FME download the attachment, but I'don't how we can configure it or if it's possible.

 

virtualcitymatt
Celebrity
Forum|alt.badge.img+35

The content of the topic should also contain the path to the attachment as a string in the json.

{ "email_publisher_subject": "FW: what??", "email_publisher_sent": "Tue Oct 22 14:22:20 UTC 2019", "email_publisher_to": "test@<somehost>", "email_publisher_from": "someone@someone.com", "email_publisher_content_type{1}": "text/plain; charset=\"utf-8\"", "email_publisher_content_type{0}": "text/html; charset=\"utf-8\"", "ws_topic": "GETEMAIL", "email_publisher_received": "Tue Oct 22 14:22:21 UTC 2019", "email_publisher_attachment{0}": "/data/fmeserver/resources/data/emailattachemtns/20191022142221-test@<somehost>-FW_what/attachemt.txt",

 

Note the subject has a ':' and two '?', however, the Attachement path points correctly to the file. 

 

 

Do you see the same behaviour when monitoring your topic? or is it mire like you have a 'ä'  or 'ö' in the email and this is what is causing the problem?

 

 

 

Note: I am running on a Linux FME Server so if you are on windows perhaps it's different
K
Forum|alt.badge.img
  • kukur
  • Author
  • October 23, 2019

In the notification, the encoding seems correct :

{
    "imap_publisher_received": "Wed Oct 23 16:28:20 CEST 2019",
    "imap_publisher_sent": "Wed Oct 23 16:28:19 CEST 2019",
    "imap_publisher_attachment{0}": "C:\\ProgramData\\Safe Software\\FME Server\\resources\\temp\\NEPTUNE\\mailBierne\\20191023162820-Neptune_IMAP-Résultats_partiel_R2019.10.004.14\\XXXX.xls",
    "ws_topic": "NEPTUNE",
    "imap_publisher_subject": "Résultats partiel R2019.10.004.14",
    "imap_publisher_from": "*****@*****.***",
    "imap_publisher_content{0}": "<html><head></head><body><br></body></html>",
    "imap_publisher_content{1}": "\r\n",
    "imap_publisher_to": "****@*****.***",
    "imap_publisher_content_type{0}": "text/html; charset=utf-8",
    "fns_type": "imap_publisher",
    "imap_publisher_content_type{1}": "text/plain; charset=utf-8"
}

 

In FME Server, I see the directory created in the Resources correctly :

20191023162820-Neptune_IMAP-Résultats_partiel_R2019.10.004.14

 

But I still have the error when my workspace runs :

Excel Reader: Failed to open the dataset 'C:\ProgramData\Safe Software\FME Server\resources\temp\NEPTUNE\mailBierne\20191023162122-Neptune_IMAP-Résultats_partiel_R2019.10.004.14\XXXX.xls'. Error message was 'can't open file for reading'

 

The problem seems to occur between the Email Publisher and the FME Server Core.

 

In the imap.log, the encoding is correct :

Wed-23-Oct-2019 04:28:44.472 PM   INFORM   Neptune IMAP   809008 : neptune-bierne : Processed email with subject "Résultats partiel R2019.10.004.14" with 1 attachment(s). Sending event...

 

in the fmeserver.log, the encoding is wrong :

Wed-23-Oct-2019 04:28:45.628 PM   INFORM   Engine-*********_Engine1@1933   401868 : Job 1934: Sending job to FME Engine **********@1933 on host **********. Request is >"NEPTUNE/******.fmw" -XML_PARAMETER <?xml version="1.1" encoding="UTF-8"?><TRANSFORMATION_REQUEST><PUBLISHED_PARAMETER name="*******">******</PUBLISHED_PARAMETER><PUBLISHED_PARAMETER name="SourceDataset_XLSXR">""C:\ProgramData\Safe Software\FME Server\resources\temp\NEPTUNE\mailBierne\20191023162820-Neptune_IMAP-Résultats_partiel_R2019.10.004.14\XXXX.xls""</PUBLISHED_PARAMETER></TRANSFORMATION_REQUEST>
virtualcitymatt
Celebrity
Forum|alt.badge.img+35
kukur wrote:

In the notification, the encoding seems correct :

{
    "imap_publisher_received": "Wed Oct 23 16:28:20 CEST 2019",
    "imap_publisher_sent": "Wed Oct 23 16:28:19 CEST 2019",
    "imap_publisher_attachment{0}": "C:\\ProgramData\\Safe Software\\FME Server\\resources\\temp\\NEPTUNE\\mailBierne\\20191023162820-Neptune_IMAP-Résultats_partiel_R2019.10.004.14\\XXXX.xls",
    "ws_topic": "NEPTUNE",
    "imap_publisher_subject": "Résultats partiel R2019.10.004.14",
    "imap_publisher_from": "*****@*****.***",
    "imap_publisher_content{0}": "<html><head></head><body><br></body></html>",
    "imap_publisher_content{1}": "\r\n",
    "imap_publisher_to": "****@*****.***",
    "imap_publisher_content_type{0}": "text/html; charset=utf-8",
    "fns_type": "imap_publisher",
    "imap_publisher_content_type{1}": "text/plain; charset=utf-8"
}

 

In FME Server, I see the directory created in the Resources correctly :

20191023162820-Neptune_IMAP-Résultats_partiel_R2019.10.004.14

 

But I still have the error when my workspace runs :

Excel Reader: Failed to open the dataset 'C:\ProgramData\Safe Software\FME Server\resources\temp\NEPTUNE\mailBierne\20191023162122-Neptune_IMAP-Résultats_partiel_R2019.10.004.14\XXXX.xls'. Error message was 'can't open file for reading'

 

The problem seems to occur between the Email Publisher and the FME Server Core.

 

In the imap.log, the encoding is correct :

Wed-23-Oct-2019 04:28:44.472 PM   INFORM   Neptune IMAP   809008 : neptune-bierne : Processed email with subject "Résultats partiel R2019.10.004.14" with 1 attachment(s). Sending event...

 

in the fmeserver.log, the encoding is wrong :

Wed-23-Oct-2019 04:28:45.628 PM   INFORM   Engine-*********_Engine1@1933   401868 : Job 1934: Sending job to FME Engine **********@1933 on host **********. Request is >"NEPTUNE/******.fmw" -XML_PARAMETER <?xml version="1.1" encoding="UTF-8"?><TRANSFORMATION_REQUEST><PUBLISHED_PARAMETER name="*******">******</PUBLISHED_PARAMETER><PUBLISHED_PARAMETER name="SourceDataset_XLSXR">""C:\ProgramData\Safe Software\FME Server\resources\temp\NEPTUNE\mailBierne\20191023162820-Neptune_IMAP-Résultats_partiel_R2019.10.004.14\XXXX.xls""</PUBLISHED_PARAMETER></TRANSFORMATION_REQUEST>

Right, OK so yeah this looks to be encoding specific like you say. I would report this to Safe via tech support.

 

An interesting test would be to create a folder/file with this character in the Data folder and then set up a simple workspace to test to see if the FME Server will read the file with the Excel Reader.

 

That would also help Safe track down the issue.

 

 

One thing which you could try is changing the system encoding on the FME Server machine to one that supports this character. It's not a great solution but it might be all you can do until the problem is fixed.

 

perhaps @lenaatsafe can help with this?
K
Forum|alt.badge.img
  • kukur
  • Author
  • November 6, 2019
virtualcitymatt wrote:

Right, OK so yeah this looks to be encoding specific like you say. I would report this to Safe via tech support.

 

An interesting test would be to create a folder/file with this character in the Data folder and then set up a simple workspace to test to see if the FME Server will read the file with the Excel Reader.

 

That would also help Safe track down the issue.

 

 

One thing which you could try is changing the system encoding on the FME Server machine to one that supports this character. It's not a great solution but it might be all you can do until the problem is fixed.

 

perhaps @lenaatsafe can help with this?

Thanks for your answer, but i'm not sure to understand well your suggestion. I'v tried to change the encoding of my reader but it doesn't change anything.

How can I change the system encoding on the FME Server machine?

virtualcitymatt
Celebrity
Forum|alt.badge.img+35
kukur wrote:

Thanks for your answer, but i'm not sure to understand well your suggestion. I'v tried to change the encoding of my reader but it doesn't change anything.

How can I change the system encoding on the FME Server machine?

My suggestion was to put an excel file in the data folder which contained the é character in the file name (or folder name) - then create simple workspace which reads the file. This would help to understand which part of FME server is not recognising the é character properly.

 

 

To change the system encoding on the FME Server will be in the Control Panel > Region > Administrative tab > Change System Locale.

 

 

Just keep in mind that this could have other effects and you should know how to switch back. I would only do it as a test to see if it helps with this problem

 

 

Reply


Helpful Members This Week

Community Stats

31,200
Posts
118,266
Replies
38,807
Members

Latest FME

Terms & ConditionsCookie settings

Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.