Question

Enable and Disable Feature Readers

  • 27 November 2014
  • 9 replies
  • 19 views

Badge +1
I have three Oracle feature readers in my workspace, one each for dev, qa, and prod. Each Oracle database has a different password.

 

 

I want to make a parameter so that the user chooses which feature reader to read from. I don't want the workspace to read from all three feature readers because the tables in each feature reader are very large. Two of the feature readers should be disabled each time that I run the workspace, based on the user's choice.

 

 

I know that you can dynamically choose feature classes, but can you dynamically enable and disable feature readers as I want to do?

 

 

Thanks

9 replies

Userlevel 2
Badge +17
Hi Jim,

 

 

Unfortunately, there is no way to enable/disable readers dynamically according to user parameter settings, as far as I know.

 

Fortunately, the source is database, so I think you can use the SQLExecutor instead of a reader feature type.

 

How about creating a workflow like this for each table?

 

Creator -> Tester (check the user parameter) [Passed] -> SQLExecutor

 

 

If the database that should be read per a run-time is always limited to one of the three, the TestFilter can also be used.

 

 

Alternatively, if the translation processes for the three databases are separated completely, create three workspaces for each database, and create the 4th workspace having three WorkspaceRunners so that the user can select workspace(s) that should be run.

 

 

Takashi
Userlevel 2
Badge +17
Another thought flashed. If you set the "Where Clause" parameter of the reader to "false", it brings the same effect as disabling the reader. So a scripted parameter could be a workaround.

 

1) Define a private scripted parameter that returns "true" or "false" according to the published user parameter setting.

 

2) Link the "Where Clause" parameter of the reader to the scripted parameter.

 

-----

 

# Script Example for Scripted (Python) Parameter

 

# Assume a published parameter "READ_DB1" (Choice: Yes/No) was defined.

 

return 'true' if FME_MacroValues['READ_DB1'] == 'Yes' else 'false'
Userlevel 2
Badge +17
If users will decide Yes/No for each database separately, "Choice with Alias" type published parameter can also be used. You can link the "Where Clause" to the published parameter directly.

 

Value  |  Display Name

 

true  |  Yes

 

false  |  No
Userlevel 4
Hi,

 

 

if frequently work with the same division of environments. I usually solve it by creating a global config file (ini or xml) that contains connection parameters for all the environments (typically development, test and production). I then create a published parameter of type Choice or Choice with alias that list these environments to the end user.

 

 

The connection properties or the readers/writers are linked to Python scripted parameters that read the relevant part of the config file and return e.g. the username, password, etc. That way you only have one set of readers / writers in each workspace, and all your workspaces can implement the same logic and behave in a consistent way.

 

 

Works perfectly.

 

 

David
Badge +3
You can also simply create a parameter type choice list and use the parametervalue in the feature reader.

 

 

I prefer using SQL executors because they are less limited to the SQL script u can create. (actualy have found hardly any limits in the use of those)

 

It makes it possibel to do dynamic sql reading. To do that you can use SQLreader followed by SQLexecutors in combination with parameters (scripted and otherwise) and attributes.

 

 

(of course u must import attributes before you parametrise the scipts, or manualy input them in teh list).
Badge +1
Okay, I understand the concept of having a user parameter to prompt the user for the environment, and then accessing the selected value through a PythonCaller, and retrieving the username and password from a file and exposing them as attributes exiting the PythonCaller.

 

 

However, I don't see how you can then pass those attributes to an SQLExecutor. In FME 2015 you have to select the format, and that disables the options in the Dataset box. Also you can't put in something like "FME_MacroValues['userName'] or $('userName') in the username / password for the SQLExecutor. It needs a hardcoded username/password on the canvas before the FME script runs.

 

 

Any help that you can give me would be great.
Badge +1
Okay, I understand how to do this. As an example, I want to read Engineering features such as sidewalks, water mains, and water valves. I want to be able to select one of these features through a parameter and let the script figure out how to connect to the correct Oracle service with the correct user name and password, and query the correct table. Here is how.

 

 

First, create an SQLExecutor with any credentials that you have. When this is done, expand the SQLExecutor in the Navigator panel and note that you can link the dataset (Oracle service name), user name, and password from user parameters.

 

 

Second, create a choice public parameter for the features that you want to query.

 

 

 

 

Third, create several scripted Python parameters that use the value of the public parameter in step two. The parameters read from a json-encoded file and retrieve the correct value for the given feature. I'm using the built-in json module for this task.

 

 

 

 

The connections.json file looks like this:

 

 

 

 

So objJson["feature"][feature]["userName"] for the parameter value "Sidewalk" would be "YYYY".

 

 

Fourth, link the dataset, username, and password Python-scripted parameters in the SQLExecutor.

 

 

If you need dynamic values for the SQL in the SQLExecutor, you can make more Python-scripted parameters and put them directly into the SQL Statement box. In my case have retrieved the correct table name in a Python-scripted parameter called "tableName" and then put that value into the SQL Statement window. Sorry, can't show that image, I have exceeded the allowable number of images!

 

 

Hope this helps.
Userlevel 2
Badge +17
Hi Jim,

 

 

Thanks for sharing your solution.

 

I've never used JSON for FME workspaces, but looks like it will be helpful to many scenarios.

 

I'll try using JSON if I got a chance.

 

 

Takashi
Userlevel 4
Hi,

 

 

I usually go for plain old INI-files (https://docs.python.org/2/library/configparser.html) rather than json for the configuration, other than that your solution is pretty identical to my usual way of doing this.

 

 

David

Reply