Question

Feature Reader w/ Parameter


Badge

Good morning! Has anyone used a FeatureReader in a workspace but set the dataset with a path that included a parameter?

The issue I'm having is because the parameter is there and not the actual file, the reader is not able to determine the structure and I get this message:

Has anyone encountered this?


13 replies

Userlevel 4
Badge +13
Hi @bcrowley10, it looks like the message from FME didn't copy over into your question. Can you try re-posting it?

 

Badge +2

Hi @bcrowley10

Not sure the format or error you are using, but yes I can use a parameter in the path:

and also parameter middle of path:

Badge

Hi @bcrowley10

Not sure the format or error you are using, but yes I can use a parameter in the path:

and also parameter middle of path:

Hi,

 

In FME Desktop 2016 1.2, when I use "$(FME_MF_DIR)020_fichiers_prepares" in the featureWriter in the output directory for Shapfile, the files are written in "$(FME_MF_DIR_USERTYPED)020_fichiers_prepares".

 

 

Using a parameter into a parameter for FeatureReader to read Shapefile does not work anymore. Parameter value of "DestDataset_SHAPE" is "$(FME_MF_DIR)020_fichiers_prepares". The input directory path is "$(DestDataset_SHAPE)\\021_Zone.shp".

 

 

It works only for featureWriter if all the path is in the parameter like this :

 

Output_folder : $(FME_MF_DIR)020_fichiers_prepares

 

In featureWriter output path: $(Output_folder)

 

 

Does anyone have an explanation , both for featureWriter and featureReader?

 

Best regards,

 

 

Userlevel 1
Badge +10

Is it not just for the setup you need to input a path for the feature reader to set up correctly which you can then amend to use a parameter?

Badge +2
Hi,

 

In FME Desktop 2016 1.2, when I use "$(FME_MF_DIR)020_fichiers_prepares" in the featureWriter in the output directory for Shapfile, the files are written in "$(FME_MF_DIR_USERTYPED)020_fichiers_prepares".

 

 

Using a parameter into a parameter for FeatureReader to read Shapefile does not work anymore. Parameter value of "DestDataset_SHAPE" is "$(FME_MF_DIR)020_fichiers_prepares". The input directory path is "$(DestDataset_SHAPE)\\021_Zone.shp".

 

 

It works only for featureWriter if all the path is in the parameter like this :

 

Output_folder : $(FME_MF_DIR)020_fichiers_prepares

 

In featureWriter output path: $(Output_folder)

 

 

Does anyone have an explanation , both for featureWriter and featureReader?

 

Best regards,

 

 

@webservicessia, If you could send us the workspace, we will help take a look at what might be the issue here. Thank you!

 

Badge

Hi @bcrowley10

Not sure the format or error you are using, but yes I can use a parameter in the path:

and also parameter middle of path:

Hi @bcrowley10, @mark_1spatial

 

 

Was there ever a resolution to this as I have the same dilemma and don't want to repost the same problem. Unfortunately the solution from @mark_1spatial means that the feature reader can't read the feature types that you want (due to the user parameter not having a defined path) and then drops the feature types from the reader. It would be good to know if there was a way to lock in those feature types so the process could be repeatable with an input folder parameter and a fixed filename as in @bcrowley10's example or even batch processing it using "*.xml" (or whatever your input data format is).

 

 

Any other ideas for this out in FME land?

 

 

Mike
Badge +2
Hi @bcrowley10, @mark_1spatial

 

 

Was there ever a resolution to this as I have the same dilemma and don't want to repost the same problem. Unfortunately the solution from @mark_1spatial means that the feature reader can't read the feature types that you want (due to the user parameter not having a defined path) and then drops the feature types from the reader. It would be good to know if there was a way to lock in those feature types so the process could be repeatable with an input folder parameter and a fixed filename as in @bcrowley10's example or even batch processing it using "*.xml" (or whatever your input data format is).

 

 

Any other ideas for this out in FME land?

 

 

Mike
Hi @mikegresham1,

 

 

Using FME 2018.1 (build 18552), I was able to read in multiple CSV files using a user parameter for the Dataset parameter. For example:

 

$(Input_Folder)\\*\\*.csv<br> This will allow you to read in multiple folders down one directory from the $(Input_Folder) parameter value. You also able to preview the first feature type read in the FeatureReader's Reader Parameters. Additionally, you can choose which feature types you wish to read if the files have different names by clicking the Feature Types to Read ellipsis.

 

 

The files in the F1-F4 folders are all named Computer_Inventory_Sample whereas the files in the T1-T4 folders have different names. It might be helpful to expose the fme_basename, fme_dataset, and/or fme_feature_type to show exactly where each feature is coming from.

 

 

Lastly, the FeatureReader will allow you to set the Output Port to One per Feature Type if you wish to process each file separately (show in the bottom bookmark on the workspace).

 

 

Feel free to test out this workspace (just have to unzip the data folder and set the input data folder to the FeatureReader directory when you run the workspace.

 

 

Hope this helps.

 

 

- Chris

 

featurereader.zip

 

featurereader-demo.fmw
Userlevel 4
Badge +13
Hi @mikegresham1,

 

 

Using FME 2018.1 (build 18552), I was able to read in multiple CSV files using a user parameter for the Dataset parameter. For example:

 

$(Input_Folder)\\*\\*.csv<br> This will allow you to read in multiple folders down one directory from the $(Input_Folder) parameter value. You also able to preview the first feature type read in the FeatureReader's Reader Parameters. Additionally, you can choose which feature types you wish to read if the files have different names by clicking the Feature Types to Read ellipsis.

 

 

The files in the F1-F4 folders are all named Computer_Inventory_Sample whereas the files in the T1-T4 folders have different names. It might be helpful to expose the fme_basename, fme_dataset, and/or fme_feature_type to show exactly where each feature is coming from.

 

 

Lastly, the FeatureReader will allow you to set the Output Port to One per Feature Type if you wish to process each file separately (show in the bottom bookmark on the workspace).

 

 

Feel free to test out this workspace (just have to unzip the data folder and set the input data folder to the FeatureReader directory when you run the workspace.

 

 

Hope this helps.

 

 

- Chris

 

featurereader.zip

 

featurereader-demo.fmw
I'm guessing that in Chris's example, he has given a value to the Input_Folder parameter over in the navigator, and that FME uses that value to go out and find the files AT DESIGN TIME so we can put their names and structure into the workspace. i.e. the FeatureReader shows the correct attributes and feature type names on its output ports.

 

 

So what this seems to say is that you can certainly use a parameter in the input dataset definition, but during design time you have to give that parameter a value AND have sample input sitting in that directory.

 

 

We could do something where we detect that the parameter doesn't have a value and prompt you for one while the FeatureReader is doing its hard work. Feels like we did that with another xformer in a similar situation, but i can't recall which right now.

 

 

Badge

Hi @ChrisAtSafe, @daleatsafe,

 

I think Dale has hit the nail on the head with this one - the feature reader requires a "sample input" of the feature types for this to work. As Dale says I think you will already have a sample feature type sitting in the user parameter Chris. Although not quite the ideal this can be a good work-around for my purposes. If the Default Value folder location in the folder user parameter registers a feature type with the same structure then at run time you can just change the folder location and the tool should run fine - does that sound right?

 

Thanks for your help with this one.

 

Mike

Badge

Hi @bcrowley10

Not sure the format or error you are using, but yes I can use a parameter in the path:

and also parameter middle of path:

Hi @ChrisAtSafe, @daleatsafe,

 

I think Dale has hit the nail on the head with this one - the feature reader requires a "sample input" of the feature types for this to work. As Dale says I think you will already have a sample feature type sitting in the user parameter Chris. Although not quite the ideal this can be a good work-around for my purposes. If the Default Value folder location in the folder user parameter registers a feature type with the same structure then at run time you can just change the folder location and the tool should run fine - does that sound right?

 

Thanks for your help with this one.

 

Mike

Userlevel 4
Badge +13

Hi @ChrisAtSafe, @daleatsafe,

 

I think Dale has hit the nail on the head with this one - the feature reader requires a "sample input" of the feature types for this to work. As Dale says I think you will already have a sample feature type sitting in the user parameter Chris. Although not quite the ideal this can be a good work-around for my purposes. If the Default Value folder location in the folder user parameter registers a feature type with the same structure then at run time you can just change the folder location and the tool should run fine - does that sound right?

 

Thanks for your help with this one.

 

Mike

Yes, your understanding sounds correct to me. Quite sure this will work.

Badge

Hi @bcrowley10

Not sure the format or error you are using, but yes I can use a parameter in the path:

and also parameter middle of path:

Hi @bcrowley10,

Have a look at my comment about this process further up the thread and see if it's in line with your original question? It's not a complete solution but a reasonable workaround.

 

Cheers,

Mike

Badge

Hello: Sorry, I've been out of the country on vacation. This was so long ago I'm sure I went another direction. Actually, I didn't realize anyone responded to this so my sincerest apologies to those who contributed to this thread. It's clear I was not the only with this question @mikegresham1 had a similar need. Glad this post picked up some steam. The FME community is a great community. Thanks!

Reply