Question

Trying to Find text and create an attribute flag?

  • 12 August 2016
  • 9 replies
  • 9 views

Badge

I am trying to find a 500 different words in a body of text that's held in a description variable using something like testfilter "contains" but don't want to write out all of the 500 words and just want to do some sort of quick join but having the flexibility of it finding the text anywhere in the description (I can do it manually by adding an entry for each word, e.g. one table holds the descriptions such as "today it will be sunny with showers" and in another table I have my 500 flags in this example "Sunny" and want to create a new variable in the table which has the description value so "Sunny" is in the new value


9 replies

Badge +2

Do you want to join two attributes to produce new variable?

Badge

Do you want to join two attributes to produce new variable?

in the example I am trying to join "Sunny" in the description variable with another varible that has "sunny"

Badge +2

1. Does that mean it is like "find and replace" and you want to collect in new attribute?

2. You will find "Sunny" and you want to replace with "sunny", does that mean both the strings are same irrespective case sensitive

Badge

Sort of, the replace will be just the single string and I don't want description around the string I am looking for.

I'm creating a new attribute, and for the record it will be assigned "Sunny" where "sunny" is contained in the description attribute, likewise another record where I am looking for "snowing" in a description like "today it will be snowing" this record will have "Snowing" assigned in the new attribute. I can get round the case issue by creating a temp attribute that is all upper case so it matches the other table so I don't then have to worry about different types of case.

The only method I have come up with is to use a workspace runner and pass each "string" I am looking through a tester via a published parameter but it means to runnig the check 500 times which is a bit slow but does work. I would have thought there would be a more sensible method where all 500 attributes can be passed through a single check against the records?

Badge +2

Hi,

If my understanding is correct normal tester will do with test condition "Contains" along with keyword as other attribute or published parameter. Updaload a sample workbench based on my understanding

safe.fmw

Badge

it does but I want a quick method of adding the tests to save time processing as there's over 500 tests that's why I went via a list using a published parameter and then I only needed to create one test which each parameter was fed into as published parameter.

Badge +2

Based on my understanding work bench prepared for input of 500 features and 25 test cases, find the updated workbench and supporting excel files attached.

safe-v1.fmwtest.xlsx

Userlevel 2
Badge +17

Hi @denby, assuming that there are two tables - one contains the descriptions ('today it will be snowing' etc.) and another contains the words ('Sunny' etc.), the InlineQuerier that performs joining table may be relatively efficient way. e.g.

select a.description, b.wordfrom descriptions as a inner join words as bon a.description like '%'||b.word||'%'
See also the attached example (FME 2015.1.3+): inlinequerier-example.fmwt
Badge +3

if u have searchkeys in a txt file or so, you can create a regsub or a tcl map string using a stringreplacer.

Resulting can be used for the aforementioned functions.

By using capture and caseinsesitive search it wil be short n easy.

Reply