• EN
Question

TestFilter - Is there a way to "import" values?

  • 22 August 2013
  • 6 replies
  • 28 views
M
Badge
Is there a way to import values when creating expressions in the TestFilter similar to the way that the AttributeValueMapper works? 

 

 

In the AVM, you can import the values of a field. This is especially helpful when I am working with datasets that I don't know the 10's or even 100's of values that I am working with. Additionally, having access to "import" or choose from values in the TestFilter would be extremely helpful and time saving.

 

 

I know that I can run the AVM first, look at, and clean up the data before running to the TestFilter but it would be a very time-saving feature to implement (if there's no quick workaround.)

 

 

Does anyone have an idea if this is possible or would find some iteration of this useful in a future release?

 

 

Regards,

 

 

 

 

Matt

6 replies

takashi
Rank
Userlevel 3
Badge +17
Hi Matt,

 

 

As far as I know, there is no way to import values into the TestFilter.

 

But, I think the test conditions can be fixed generally. I cannot imagine a concrete situation in which you need to import values into the TestFilter. Depending on the actual situation, there could be other approaches.

 

Could you please explain about the test conditions (expressions) which you need to create?

 

Takashi
M
Badge
Hi Takashi,

 

 

I work with a lot of data sets that, plainly, I don't know very intimately. I will know generally what I need to do with certain attributes.

 

 

The approach I use it for is more on the "geoprocessing" end which means that I need to split the data down different swim lanes for various purposes.

 

 

For example, I work in the oil and gas industry mostly and different datasets have LOTS of defining attributes. I want to split a subset of the attributes out and do whatever (insert process/data manipulation, statistics, etc...) on that.

 

 

By not being intimate with the clients data I know roughly the value or values I want and can easily pick them out if I were to see them on a list.

 

 

Secondly, it will allow me to easily catch the human errors (ie typos) within large datasets and stream them together.

 

 

Third, lots of my workbenchs are merged/joined, then split and re-merged/joined back together. I would like to be able to see the current set of attributes of even the merged tables into the single attribute/column.

 

 

Again, the AttributeValueMapper will basically get me there by doing a lookup on the source data and I sometimes use it (or bring into ArcMap) and quickly look at the data. If I have my GIS viewer up, I can quickly get the uniqe values and then manually type them into the TestFilter.

 

 

This seems to be a work around approach and would be nice and convenient to simply to a unique value lookup from within the TestFilter, or the depricated Tester for that matter.

 

 

The nice feature of FME is that it suits all kinds of workflows and I use it in lieu of the conventional GIS analysts who build in ArcGIS' modelbuider, or whatever flavor of "GIS" they use. Using FME I can be a "developer" and and "analyst" in one single push of a button which is why I'm trying dilligently to single-source with FME.

 

 

Including a value lookup would be an immense time-saver and QA/QC feature at the same time.

 

 

Not saying you can change this if the features not built in, I can always put a ticket in with Safe, but if you did know a quick workaround that would be fantastic.

 

 

Thanks again,

 

 

 

Matt

 

 

 

takashi
Rank
Userlevel 3
Badge +17
Hi Matt,

 

 

My understanding is: you want to split the feature flow according to an attribute, but the attribute values used as the filtering conditions are not known beforehand; you need to pick up the specific values from the actual dataset and import them into the workspace before running.

 

 

If my understanding is correct, another possible approach is to create an external table which maps the specific values and constants (i.e. the values used as the filtering conditions). You can merge the table to the features (FeatureMerger), and then filter the merged features (TestFilter).

 

Naturally, you will have to create or modify the external table depending on the source dataset. But you will not need to modify the workspace itself.

 

 

Hope this helps.

 

Takashi
N
Hi Takashi,

 

 

I work with a lot of data sets that, plainly, I don't know very intimately. I will know generally what I need to do with certain attributes.

 

 

The approach I use it for is more on the "geoprocessing" end which means that I need to split the data down different swim lanes for various purposes.

 

 

For example, I work in the oil and gas industry mostly and different datasets have LOTS of defining attributes. I want to split a subset of the attributes out and do whatever (insert process/data manipulation, statistics, etc...) on that.

 

 

By not being intimate with the clients data I know roughly the value or values I want and can easily pick them out if I were to see them on a list.

 

 

Secondly, it will allow me to easily catch the human errors (ie typos) within large datasets and stream them together.

 

 

Third, lots of my workbenchs are merged/joined, then split and re-merged/joined back together. I would like to be able to see the current set of attributes of even the merged tables into the single attribute/column.

 

 

Again, the AttributeValueMapper will basically get me there by doing a lookup on the source data and I sometimes use it (or bring into ArcMap) and quickly look at the data. If I have my GIS viewer up, I can quickly get the uniqe values and then manually type them into the TestFilter.

 

 

This seems to be a work around approach and would be nice and convenient to simply to a unique value lookup from within the TestFilter, or the depricated Tester for that matter.

 

 

The nice feature of FME is that it suits all kinds of workflows and I use it in lieu of the conventional GIS analysts who build in ArcGIS' modelbuider, or whatever flavor of "GIS" they use. Using FME I can be a "developer" and and "analyst" in one single push of a button which is why I'm trying dilligently to single-source with FME.

 

 

Including a value lookup would be an immense time-saver and QA/QC feature at the same time.

 

 

Not saying you can change this if the features not built in, I can always put a ticket in with Safe, but if you did know a quick workaround that would be fantastic.

 

 

Thanks again,

 

 

 

Matt

 

 

 

Any update to this Matt... I am attempting to do something very similar.

 

@matthewbrucker
M
Badge +5

Any update to this Matt... I am attempting to do something very similar.

 

@matthewbrucker

I wonder if the AttributeFilter will help? It's got an import option and allows filtering rather than mapping of data.

G
Badge +3

Maybe usefull technique.

I create a list of the attribute i want to filter on.

Then i use a listsearcher with a conditonal attribute in the "search for".

In the conditional i use expressions and regepx.

I create the regexp value by using a 'external table" of wich i export the"conditions or testvalue to a textfile. Then use Notpad++ to format the data into a string to use in the rexp/expr by , again using regexp.

Of course when sets change you have to do the proces of creating the "mappping"list everytime.

(in my example case it is kinda long term static)

Maybe a pic says more..

Reply


Community Stats

28,803
Posts
109,800
Replies
36,634
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.