Skip to main content
  • EN

Hi,

 

I'm currently working on a workbench that will be invoked with the fmedatastreaming web service in FME Server. The web service is called with URL parameters, like so:

https://my-fme-server/fmedatastreaming/Test_Public/VindfangDEV.fmw?session=5218605431&contact_name=Innmåler&contact_org=Virksomhet&contact_phone=12345678&metadata_soksys=123&files=I+-+Trekkerør+Utført.gmi&debug=true&contact_mobile=12345678

If I inspect the parameter values fetched from ParameterFetcher with a Logger transformer inside my workbench, I get encoding errors:

 

0 

However, in both Request Data and earlier in the log file, these values are not only properly URL encoded in the request URL, but also correctly encoded and displayed:

 

1And:

2 

The encoding problem appears to only occour in the FME engine executing the workbench, but not in the web service nor the admin web interface.

 

Manually encoding the problematic characters prior to calling the fmedatastreaming web service (...&contact_name=Innm%C3%A5ler&&files=I+-+Trekker%C3%B8r+Utf%C3%B8rt.gmi&...) does not make any difference.

 

But if I simply click the "Resubmit Job" button from FME Server's "Jobs / Completed" interface, ParameterFetcher will encode the characters corretly and my Logger transformer will show the parameters with correct encoding!

 

5 

So the encoding problem only occours when the workbench is invoked through the web services. The same problem (and solution with resubmitting the job) occours when I try the fmejobsubmitter web service.

 

Because the Request Data tab and the log file shows correctly encoded characters, I don't think the problem lies with Tomcat. Otherwise, I would expect the encoding errors to appear in both FME Server's web admin interface and in the log file too. Just for completeness, here's how character encoding in my fmedatastreaming Tomcat webapp is setup (default from FME Server installation):

 

3 

PS. I also tried to encode the string with the TextEncoder transformer to unicode code points, but then my Logger just say that the problematic characters are \uFFFD, a.k.a "replacement for an incoming character whose value is unknown or unrepresentable in Unicode".

 

Both FME Server and FME Workbench used to author the workbench are version:

  • FME Server 2021.0.2
  • Build 21321 - win64

 

I'm not sure how I can proceed to debug this further. Any advice or ideas would be greatly appreciated!

I just did a basic test on our linux server (FME 2020) and it worked fine.

 

It does seem like the engine is using the system encoding for these instead of UTF-8. What happens if you put in %F8 (the Windows-1252 encoding of ø).

 

Also what are the parameter "types" used here.

 

 

 

 

I just did a basic test on our linux server (FME 2020) and it worked fine.

 

It does seem like the engine is using the system encoding for these instead of UTF-8. What happens if you put in %F8 (the Windows-1252 encoding of ø).

 

Also what are the parameter "types" used here.

 

 

 

 

Thank you! Based on your reply, I managed to google my way to this article: https://community.safe.com/s/article/UTF-8-Names-FAQ-Encoding-in-FME

 

If I understand it correctly, my character encoding issue should resolve itself once I upgrade to FME Server 2022.0+. For now, all calls to web services my FME Server originates from web pages/apps I author, so I can manually escape and unescape these characters until then.

 

I tried using %F8, but then the engine interprets it as \\u001a. The parameters types are single line text.

Just for fun, I also tried to manually change FME_NAMES_ENCODING to "utf-8" and re-uploading my workspace, but it had no effect.

Thank you! Based on your reply, I managed to google my way to this article: https://community.safe.com/s/article/UTF-8-Names-FAQ-Encoding-in-FME

 

If I understand it correctly, my character encoding issue should resolve itself once I upgrade to FME Server 2022.0+. For now, all calls to web services my FME Server originates from web pages/apps I author, so I can manually escape and unescape these characters until then.

 

I tried using %F8, but then the engine interprets it as \\u001a. The parameters types are single line text.

Just for fun, I also tried to manually change FME_NAMES_ENCODING to "utf-8" and re-uploading my workspace, but it had no effect.

One thing could be to try and change from single line to multiline text to see if it has an effect, however, it probably wont. Certainly seems like a bug to me though. But yeah, hopefully upgrading to FME 2022 will solve the problem

Reply


Community Stats

30,149
Posts
114,437
Replies
37,792
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.