Hi!
Hope someone can help me out with this:
I'm trying to create some sort of dynamic way to filter specific set of Attributes. Since the AttributeKeeper needs this input: AttributeName1,AttributeName2,AttributeName3,etc input, I was wandering if I can use a AttriubuteValue for this..but this is where I got lost;) I've created an attribute with concatenated list of Attributes I want to keep. But what would be a good approche to use this value to keep specific set of Attributes?
cheers,
ronald
+2
@rva1 Dynamically filtering attributes in workbench is tricky, because all the subsequent transformers rely on knowing what attributes are coming. Feature caching (Workspace Authoring 101: Feature Caching) helps during authoring. If you can describe why you need to filter the attributes dynamically and the end goal then someone might be able to give some tips.
+29
You could look at building the writers to support dynamic schemas, then you can adjust the schema list to match your requirements. Then you'll pass the schema feature to your writers and they will only use the attributes specified in the schema.
https://community.safe.com/s/article/dynamic-workflows-advanced-example-modifying-the-s
@rva1 Dynamically filtering attributes in workbench is tricky, because all the subsequent transformers rely on knowing what attributes are coming. Feature caching (Workspace Authoring 101: Feature Caching) helps during authoring. If you can describe why you need to filter the attributes dynamically and the end goal then someone might be able to give some tips.
hi mark,
i'll try to explain what i would like to create
I have 2 input files: mdb (1) and xlsx (2).
Mdb contains data i want to transform.
Xlsx is the filter part. This xlsx is the dynamic part of the workspace.
I would like to create seperate tables (3) of each feature_type (this is a predefined number of feature_types) where the specific attributes per feature_type can change in the near future.
In example I use two tabs in the xlsx. tab1 is the filter for fme_feature_type1, tab2 is for filtering type 2. The filtering is done by adding "x" behind an attribute. In fme i use this "x" to filter only desired attributes. These features is can use to create list, grouped by feature_type. And then concatenate this list to: attr1,attr3,attr6 etc format. Since you can copy paste this format in the attribute keeper, i'm looking for a way to do this dynamic, since i'll be changing the template file in the near feature. (so in other words: adding more or less "x").
In the end i could do it without this filtering, but it sure is welcome if i can get a little more overview in the data (and keep it a little lighter to work with). In my case i have to dig through 1.500 unique attributes... Thats a long list;)
Hope this explanation helps;)
cheers,
ronald
You could look at building the writers to support dynamic schemas, then you can adjust the schema list to match your requirements. Then you'll pass the schema feature to your writers and they will only use the attributes specified in the schema.
https://community.safe.com/s/article/dynamic-workflows-advanced-example-modifying-the-s
hi @hkingsbury
i think this is used to dynamic writing. In my case the number of features i'm writing is determined by the reader. The dynamic part is in filtering out data per feature, so the dataset i'm creating with the writer is cleaned up. In short I would like to have AttributeKeeper kinda functionality by using excel, grouped by feature_type... does this make sense at all?:))
+2
hi mark,
i'll try to explain what i would like to create
I have 2 input files: mdb (1) and xlsx (2).
Mdb contains data i want to transform.
Xlsx is the filter part. This xlsx is the dynamic part of the workspace.
I would like to create seperate tables (3) of each feature_type (this is a predefined number of feature_types) where the specific attributes per feature_type can change in the near future.
In example I use two tabs in the xlsx. tab1 is the filter for fme_feature_type1, tab2 is for filtering type 2. The filtering is done by adding "x" behind an attribute. In fme i use this "x" to filter only desired attributes. These features is can use to create list, grouped by feature_type. And then concatenate this list to: attr1,attr3,attr6 etc format. Since you can copy paste this format in the attribute keeper, i'm looking for a way to do this dynamic, since i'll be changing the template file in the near feature. (so in other words: adding more or less "x").
In the end i could do it without this filtering, but it sure is welcome if i can get a little more overview in the data (and keep it a little lighter to work with). In my case i have to dig through 1.500 unique attributes... Thats a long list;)
Hope this explanation helps;)
cheers,
ronald
@rva1 Do you have the example spreadsheet that you can attach?
Hi, I've the absolut identical problem to solve.
I have different features to process and I have a database where is defined wich attributes from wich feature I have to write out.
I tried the same way @rva1 described at the beginning. And I tried to do this by FMEFunctionCaller and give @KeepAttributes(@Value(AttributKeeperValue)) as the FME Function - Parameter. It didn't work either. 😥
The possibility to write per dynamic shema could be a way, but first I must have only the needed attributes at the features before I can write them dynamicly.
So, how can I only keep the needed attributes in this case. And I want to create the workbench as dynamic as possible. So I don't want to define per feature the keeping attributes by hand.
Is there a way to do this?
+29
Hi, I've the absolut identical problem to solve.
I have different features to process and I have a database where is defined wich attributes from wich feature I have to write out.
I tried the same way @rva1 described at the beginning. And I tried to do this by FMEFunctionCaller and give @KeepAttributes(@Value(AttributKeeperValue)) as the FME Function - Parameter. It didn't work either. 😥
The possibility to write per dynamic shema could be a way, but first I must have only the needed attributes at the features before I can write them dynamicly.
So, how can I only keep the needed attributes in this case. And I want to create the workbench as dynamic as possible. So I don't want to define per feature the keeping attributes by hand.
Is there a way to do this?
The general gist of what you'd do is
Hi, another person encountering this same issue.
As background, I'm building a workspace that will read from various WFS sources, and uses SchemaMapper to relabel and setup attributes (each WFS source has own schema map file).
My export format is 12d which handles attributes at the object / feature level and not the data table level, so we only want attributes relevant for that feature.
As an example, I'm in the utilities field using WFS sources that may have attributes for both Pits and Valves in the one dataset. For Pits we want to keep "RL" and "Depth", Valves only keep "RL" as "Depth" is irrelevant / null.
I could assign a Category attribute to each feature, and then assign a Fields attribute that for pits could be "RL,Depth" and for Valves be "RL".
Currently I can hard code AttributeKeepers (one for each category), but that does not allow for a dynamic / changing output (eg, on a different project the output schema could have additional fields for Valve to be kept).
If I try entering the "Fields" value into AttributeKeeper as @Value(Fields) then it fails to use the value as a comma delimited list.
Unless there is a better suggestion, I'm about to try the following workaround:
Could be an issue if the AttributeManager can't delete unexposed attributes.
May also need to copy the "keep field lists" and use that copy as the original field may be deleted during the iteration, then delete the copy once the looping is done.
I'm also aware of using PythonCaller but our other FME users can't develop in python so that is only a last resort.
Seems the issue is Transformers can't reference attributes that are generated at runtime or something like that...
Looks like PythonCaller could be only way to achieve this.
+29
Hi, another person encountering this same issue.
As background, I'm building a workspace that will read from various WFS sources, and uses SchemaMapper to relabel and setup attributes (each WFS source has own schema map file).
My export format is 12d which handles attributes at the object / feature level and not the data table level, so we only want attributes relevant for that feature.
As an example, I'm in the utilities field using WFS sources that may have attributes for both Pits and Valves in the one dataset. For Pits we want to keep "RL" and "Depth", Valves only keep "RL" as "Depth" is irrelevant / null.
I could assign a Category attribute to each feature, and then assign a Fields attribute that for pits could be "RL,Depth" and for Valves be "RL".
Currently I can hard code AttributeKeepers (one for each category), but that does not allow for a dynamic / changing output (eg, on a different project the output schema could have additional fields for Valve to be kept).
If I try entering the "Fields" value into AttributeKeeper as @Value(Fields) then it fails to use the value as a comma delimited list.
Unless there is a better suggestion, I'm about to try the following workaround:
Could be an issue if the AttributeManager can't delete unexposed attributes.
May also need to copy the "keep field lists" and use that copy as the original field may be deleted during the iteration, then delete the copy once the looping is done.
I'm also aware of using PythonCaller but our other FME users can't develop in python so that is only a last resort.
Seems the issue is Transformers can't reference attributes that are generated at runtime or something like that...
Looks like PythonCaller could be only way to achieve this.
What you could do is take one instance of each output schema, and explode it giving you a feature per attribute for each schema. Then remove/alter/add to the schema(s) rebuild into a list and then merge back onto the appropriate features.
If you're having trouble creating the schema in the first instance, have a look at the schema setter:
https://hub.safe.com/publishers/safe-lab/transformers/schemasetter#description
Enter your username or e-mail address. We'll send you an e-mail with instructions to reset your password.
