There are a number of ways to filter and direct data and attribute filter looked like the one. I expected to have a quick and simple option as all I'm really trying to do is send multiple clipped files to the correct transformers and writers. This is by no means intuitive with no easy way to expose the dataset name and direct it accordingly. Instead of the dataset name (admittedly rather long) I get a selection of random names (Source dataset ERS, source dataset ERS9, etc) which I have to try and match with the dataset name. Even though these appeared as options, this then didn't work and I had to add in an attribute creator before attributefilter would pass the data through. Am I just using this transformer incorrectly?
Page 1 / 1
Have you tried the fme attributes fme_dataset, fme_basename and fme_featuretype?
They contain source file name and layer name and can be used to redirect the output (after exposing them on the reader).
FME lists basename as being exposed and I can elect to use it but it doesn't then show up the dataset names just the "dataset source ERS, ERS9, ERS11, etc) and doesn't process the datasets unless I create a new attribute and then populate the attribute with the above source names (ERS, ERS9, ERS11, etc). My knowledge of FME is basic/intermediate but I would have thought this would be simpler.
FME lists basename as being exposed and I can elect to use it but it doesn't then show up the dataset names just the "dataset source ERS, ERS9, ERS11, etc) and doesn't process the datasets unless I create a new attribute and then populate the attribute with the above source names (ERS, ERS9, ERS11, etc). My knowledge of FME is basic/intermediate but I would have thought this would be simpler.
See this page and check if it's fit to what you are going to accomplish.
Separating Output Data with Fanout | FME Workbench Help
@takashi Not what I'm after. I have a number of datasets going into some processes and I want to then direct groups of the processed data to further processes and writers. I think fanout is to create multiple datasets based on criteria from 1 dataset. I have multiple datasets going in and want to have multiple coming out without having many duplicate processes.
@takashi Not what I'm after. I have a number of datasets going into some processes and I want to then direct groups of the processed data to further processes and writers. I think fanout is to create multiple datasets based on criteria from 1 dataset. I have multiple datasets going in and want to have multiple coming out without having many duplicate processes.
Anyway, clarify the conditions and requirement.
- What is the data format of source/destination datasets?
- The "ERS", "ERS9", and "ERS11" (for example) are possible values of "fme_basename" attribute?
- Do you need to write the features into files called "ERS", "ERS9", and "ERS11" according to the value of "fme_basename" attribute?
@takashi Not what I'm after. I have a number of datasets going into some processes and I want to then direct groups of the processed data to further processes and writers. I think fanout is to create multiple datasets based on criteria from 1 dataset. I have multiple datasets going in and want to have multiple coming out without having many duplicate processes.
- Is the schema (geometry type; structure of attributes - attribute names, data types) of the destination datasets identical?
@takashi I should have better outlined my process. I have 6 images (ERS) being clipped. I then want to mosaic them but not all together, I want same years mosaiced together. See image "FME Attribute workflow.png". I don't understand why I need the Attribute creator but Attribute filter doesn't seem to work otherwise. I think I have exposed the basename as per the image "basename exposed" and I then set it as the attribute to filter by but the only options shown in AttrtibuteFilter are the Source Datasets ERS, ERS9, etc. See image "AttributeFilter-Basename.png". I expected to see the dataset name. I have attached my fmw.clip-and-mocaic-images-with-attribute-filter.fmw
Hi @garethgis, I think you can use both Feature Type Fanout and Dataset Fanout effectively.
Since all source ERS file names start with year ("2009", "2014", and "2016"), you can extract the year (first four characters) from "fme_basename", mosaic the clipped rasters grouping by year, and then write them into GeoTIFF files with appropriate file names. Here, you can apply Feature Type Fanout to the GeoTIFF writer feature type in order to define the destination file names. That is, you can set this string expression to the feature type name (Raster File Name) field.
Then, in the Navigator window, configure Dataset Fanout. You can use feature attributes ("_year" in this case) to set the dataset name (folder name in this case) with this string expression.
I don't think the AttributeCreators existing in your workspace are essential in this case.
Since all source ERS file names start with year ("2009", "2014", and "2016"), you can extract the year (first four characters) from "fme_basename", mosaic the clipped rasters grouping by year, and then write them into GeoTIFF files with appropriate file names. Here, you can apply Feature Type Fanout to the GeoTIFF writer feature type in order to define the destination file names. That is, you can set this string expression to the feature type name (Raster File Name) field.
@Value(_year) Veg Ht
Then, in the Navigator window, configure Dataset Fanout. You can use feature attributes ("_year" in this case) to set the dataset name (folder name in this case) with this string expression.
UM@Value(_year)
I don't think the AttributeCreators existing in your workspace are essential in this case.
@takashi That looks pretty slick. I'll work my way through it later today.