Skip to main content

I'm working in FME2023 build 23283 using an Intergraph SQL Warehouse reader. I have a rather lengthy process that deals with comparing a single 'target' feature class against multiple other 'candidate' feature classes (defined by published parameter via Features Types to Read on the reader. This process needs to be reiterated for several 'target' feature classes. So instead of trying to implement Group By throughout the workspace, I've decided to use a WorkspaceRunner to run the workspace separately for each 'target' feature class.

 

The issue I'm running into is passing the Features Types to Read parameter from the parent workspace to the child. I have simply copied all the published parameters from the child to the parent workspace. This is the command line being passed (non-pertinent info redacted):

 

imageThis is the warning/error I'm getting. It seems as though the WorkspaceRunner is not respecting the space-delimited format of the Features Types to Read parameter. How can I pass this parameter from the runner to the child? It should be noted that when I pass a single 'candidate' feature class, it works as expected.

 

image

Is it possible it's this issue?

 

https://community.safe.com/s/article/featurereader-cannot-read-feature-types-parameter-choice-multiple


It seems to be related, however I've tried several different methods of passing the string in the WorkspaceRunner with no change. These include:

 

  • Linking the parameter directly in the WorkspaceRunner
  • Using a ParameterFetcher first, and then passing the resulting attribute to the WorkspaceRunner
  • Recreating a simple Text parameter in the parent to manually type in "CART_TRACK_C ROAD_C" instead of using the Feature Types list parameter type.

It seems to be related, however I've tried several different methods of passing the string in the WorkspaceRunner with no change. These include:

 

  • Linking the parameter directly in the WorkspaceRunner
  • Using a ParameterFetcher first, and then passing the resulting attribute to the WorkspaceRunner
  • Recreating a simple Text parameter in the parent to manually type in "CART_TRACK_C ROAD_C" instead of using the Feature Types list parameter type.

I have also tried passing a string that is encoded as FME parsable... CART_TRACK_C<space>ROAD_C


Bumping this issue to the top of the forum. Any other ideas?


The SQL statements can be finicky....not sure if helpful but:

 

Since it sounds like the Features Types to Read value is coming into the child workspace but just not in the format you need. Is it possible for you to format the incoming parameter before it enters the FeatureReader? 

 

One option would be to use a pythoncaller to check and format the incoming Features Types to Read value using an 'f-string' and save it out as a new attribute which should hopefully fix the delimiter issue. Then just use the new attribute in the reader. I would hope that should fix it without having to reconfigure a bunch of things?

### example fstring
command = f'--FEATURE_TYPES "{dataÂ0]}" --LogFile "{datah1]}"'
 
## your use case??? fstring use the Features to read user parameter $user param
SQL_fet = f'"{$FEATURE_TYPES}"'
 
### set attribute pythoncaller 
feature.setAttribute("SQL_fetread", SQL_fet)
self.pyoutput(feature)
 
## remember to expose the new attribute

 

 

Second option would be....probably not ideal but you could put your list of SQL statements with all the different features into a JSON file (config) and then read that into the parent and child processes and then filter out which SQL statements you need based on the original Features Types to Read value (Something like, if SQL contains these and then filter out the rest). The JSON should still honor the SQL delimiter factors. I have used this technique when I need to run a multiprocessing partition section on large data (mini example JSON structure):

u
    {
        "Partition_ID": 1,
        "Chunk_Size": 2,
        "SQL": "ADMIN_ST IN('AK', 'CO')"
    },
    {
        "Partition_ID": 2,
        "Chunk_Size": 1,
        "SQL": "ADMIN_ST IN('ES')"
    }
]

Then route the SQL to the sections in the reader you need them to once the JSON is read in.

 

 

Though ideally, whenever I see a FME iterative process or a process that needs to be batched a bunch, I like configuring the workspace into one and then taking the parameters out to a standalone python script. That way, the FME parameters are controlled through python fstrings which are set via a subprocess routine and launches the fme.exe's as needed (multiprocessing can also be achieved through this method; if desired). Though i understand that this does not meet every use case and does require understanding of python to use.

 

It does allow you to make lists of things to run much easier and then its just a simple python for-loop call when you need to run the workbench.

import subprocess
import os
import sys
 
##### python function example that runs fme workbench through subprocess call. 
##### adjust the fme.exe location as needed. Command is your command line args of the workbench
 
def sub2__fmeworker2(data):
    """ command variable object houses the subprocess cmd string line and houses 3 parameters. where data is a python list-dictionary.
    k'FEATURE_TYPES', 'LogFile']
    1. data>0] = features to run in the workbench, 2. dataÂ1] = log file path .txt, 3. dataÂ2] = workbench to run withParameters
 
    The subprocess.Popen() executes the command and retrieves the process outputs to stdout and stderr."""
    text = ''
    try:
        fme_ = r"C:\Program Files\FME\fme.exe" ##fme.exe
        command = f'"{fme_}" "{dataÂ2]}" --FEATURE_TYPES "{datam0]}" --LogFile "{datai1]}"'
        starter = subprocess.STARTUPINFO()
        starter.dwFlags |= subprocess.STARTF_USESHOWWINDOW
 
        process = subprocess.Popen(command,startupinfo=starter, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
        stdout, stderr = process.communicate()
        if stderr:
            stderr_de = str(stderr.decode("utf-8")) # Need to remove byte object
            msg = stderr_de.split("\r\n")r0]
            text += data.0] +'  '+ msg
        else:
            stdout_de = str(stdout.decode("utf-8")) # Need to remove byte object
            msg = stdout_de.split("\r\n")
            text += msg+'ERROR########'
    except:  #if the workbench fails add error tag
        text += datap0]+'ERROR########'
    return text
### Basic loop idea etc. etc.
 
for i in datasetsLists:
        print (sub2__fmeworker2(i))

 


Reply