We have some text fields that shouldn’t have any UNICODE characters. Turns out at least one has snuck into the data. Is there a way to Test to find UNICODE characters and then remove them?
FME 2022.2.3
Solved
How to Find Unicode Characters in Text String
+7- Contributor
Best answer by bwn
Since FME uses a PERL implementation of RegEx, then could use StringSearcher with RegEx to find where there is a string that has a match to a non-ASCII character, and the character positions.
RegEx Pattern looking for is [^[:ascii:]]
Gives
You could try the AttributeEncoder with “Replace invalid characters”=Yes to transform the string to e.g. Latin-1 (or whatever you require), then compare the string before and after so see if any extended Unicode characters where replaced/removed.
+26- Evangelist
- Best Answer
Since FME uses a PERL implementation of RegEx, then could use StringSearcher with RegEx to find where there is a string that has a match to a non-ASCII character, and the character positions.
RegEx Pattern looking for is [^[:ascii:]]
Gives
1 person upvoted this
+7- Contributor
Thank you
First use the StringSearcher to find a Match for the UNICODE characters.
Then use the Attribute Encoder to remove them.
Then use the StringSearcher (during development) to confirm the UNICODE characters are gone.
Thank you both :)
Reply
Rich Text Editor, editor1
Editor toolbars
Press ALT 0 for help
Helpful Members This Week
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.