Error pivot table with utf-8 attribute

What problem did you get?

Hi @hoa_le

Thanks you workspace.

I checked that you used the transformer PythonCaller and you checked if the input information is Unicode.

In my opinion the problem is your transformer checked the information unicode and the result was inputed in the Log file:

If you used the transformer AttributeValidator for check if the value is Unicode?

Thanks,

Danilo

Actually, I want to pivot table use to change_type column but it's utf-8 attributes. So I get error when pivot. So I want to ask how to fix it to write pivot table.

I'm unclear what you intend to get with the AttributePivoter. Anyway, I don't think it's a character encoding issue.

If your goal was to calculate the number of features for each unique pair of province and change_type, the StatisticsCalculator would be the way to go, rather than the AttributePivoter.

Example:

Result:

Hi Takashi,

I will use pivot based on change_type in collumn. But chang_type collumn is utf-8. So when pivot I get error at the below. I attached my pivot file excel that I want to do in FME.

I was able to reproduce the same Python error as yours. The AttributePivoter is a Python based transformer, and looks like it doesn't support international characters if Python interpreter version is 2.7. In order to avoid this error, you should change the Python Compatibility parameter to 3.4+ or later.

Note: Python 2.7 support would be removed from FME in the near future.

Probably this workflow generates a result closer to your desired one.

Thank you, Takashi. It seems to have error font in collumns.

Could you tell me how to fix it? Thank you

That's bad... I think there is a defect on internationalization of FME.

I'd recommend you to contact Safe support to ask them to improve FME.

Hi @hoa_le, if the workspace would be run on Windows Vietnamese Edition, the AttributeEncoder might be a workaround in the interim.

Try this one (I cannot test it in my environment).

Hi @takashi. It's still error :(. We use utf-8. Windows Vietnamese can't use.

Sad to hear that. It's definitely a Safe's support case. Please report the problem to Safe as a defect of the current version of FME.

https://www.safe.com/support/report-a-problem/

Hi @hoa_le,

Thanks for reporting the issue. For anyone else following this question, FME handles attribute names with systems default encoding and doesn't have the same encoding application as the data values, which is why the values are displaying correctly but the header row is not. Writing out in Excel, the first header row contains the attribute names if you select to write that out in the parameters.

One potential workaround is to extract the desired first row as data values and re-insert it into the excel output. Takashi's answer from this Q&A might be helpful for that.

Our team is working hard on extending that functionality for the future so that the attribute names will be handled correctly. The issue is FMEENGINE-56578, and we'll let let you know when the issue has been addressed. Thanks everyone!

Thank you so much :)