Skip to main content
Solved

JSON from email publication invalid

djmcdermott
Contributor
Forum|alt.badge.img+6

Hello,

I have set up a IMAP publisher to poll one of our inboxes. The idea is for it to trigger a workbench to download and process data when our supplier informs us of an update. When I send a test email which is either blank for in plain text it works, but the supplier emails are HTML and the JSON in the Topic is invalid. The part of the JSON that is invalid is in the imap_publisher_content{0} section. This is the HTML message which I think it treats and JSON so fails when it’s expecting a : or }. Please can you help?

 

Thanks,

David

Best answer by markw

djmcdermott wrote:

Hi @Mark Warren​ I have attached the JSON that is dropped in the Topic Monitoring. I have removed sensitive data. I think what’s happening is FME is escaping HTML characters but not CSS. For instance in this element:

<span style=\\"font-size:12.0pt;font-family:"Source Sans Pro Light",sans-serif\\">

It escapes the “ but then see’s the ; and thinks its part of the JSON. 

 

Thanks

Thanks for sharing the data! I've found that the errors are happening at the double-quotes (") –specifically the font-family style attributes (i.e: "Arial", "Segoe UI", "Source Sans Pro Light")–since JSON key values need to have "s escaped or else it'll be considered invalid. I used some StringReplacers to remove the double quotes from those 3 font families, and FME was able to parse the JSON after, like this:

Screen Shot 2022-02-03 at 12.07.22 PM 

I'm looking into this with your partner 1Spatial so we'll investigate further and share more info through the support case.

 

UPDATE: Using the Email Trigger in Automations and the Email Body output key avoids the issue.

View original
Did this help you find an answer to your question?

3 replies

markw
Forum|alt.badge.img
  • markw
  • February 2, 2022

Hi @djmcdermott​ , could you please share a snippet or screenshot of the errors you're getting in the job? I'm wondering if the issue is occurring at a JSON-parsing transformer and something is slightly off with a JSON query. We should be able to handle various content types in the imap_publisher_content{0} section like plain text and HTML since it's still "wrapped" in the JSON payload.

djmcdermott
Contributor
Forum|alt.badge.img+6
  • djmcdermottContributor
  • Author
  • Contributor
  • February 3, 2022
markw wrote:

Hi @djmcdermott​ , could you please share a snippet or screenshot of the errors you're getting in the job? I'm wondering if the issue is occurring at a JSON-parsing transformer and something is slightly off with a JSON query. We should be able to handle various content types in the imap_publisher_content{0} section like plain text and HTML since it's still "wrapped" in the JSON payload.

Hi @Mark Warren​ I have attached the JSON that is dropped in the Topic Monitoring. I have removed sensitive data. I think what’s happening is FME is escaping HTML characters but not CSS. For instance in this element:

<span style=\\"font-size:12.0pt;font-family:"Source Sans Pro Light",sans-serif\\">

It escapes the “ but then see’s the ; and thinks its part of the JSON. 

 

Thanks

1 Attachments
markw
Forum|alt.badge.img
  • markw
  • Best Answer
  • February 3, 2022
djmcdermott wrote:

Hi @Mark Warren​ I have attached the JSON that is dropped in the Topic Monitoring. I have removed sensitive data. I think what’s happening is FME is escaping HTML characters but not CSS. For instance in this element:

<span style=\\"font-size:12.0pt;font-family:"Source Sans Pro Light",sans-serif\\">

It escapes the “ but then see’s the ; and thinks its part of the JSON. 

 

Thanks

Thanks for sharing the data! I've found that the errors are happening at the double-quotes (") –specifically the font-family style attributes (i.e: "Arial", "Segoe UI", "Source Sans Pro Light")–since JSON key values need to have "s escaped or else it'll be considered invalid. I used some StringReplacers to remove the double quotes from those 3 font families, and FME was able to parse the JSON after, like this:

Screen Shot 2022-02-03 at 12.07.22 PM 

I'm looking into this with your partner 1Spatial so we'll investigate further and share more info through the support case.

 

UPDATE: Using the Email Trigger in Automations and the Email Body output key avoids the issue.

Reply


Helpful Members This Week

Community Stats

31,399
Posts
119,072
Replies
39,018
Members

Latest FME

Terms & ConditionsCookie settings

Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.