Solved

Please help get me started with user Python scripted parameter returning file paths to readers.

  • 19 December 2020
  • 13 replies
  • 18 views
townest
Rank
Badge +4

I have developed a python script to walk though directories picking dwg files that fit the pattern I want. I would like to add the script to a user parameter and pass/link the parameter to a reader source. For testing purposes I started with a simple script to return a single dwg, linked it to the reader source, and it failed with error...'Update Reader Failed'. My simple Python User Parameter scrips is as follows:

----

file = r'C:\\data\\dwgfoldertest\\Test1.dwg'

return file

-----

Any pointers would be appreciated. If I get the simple test working, I'll move onto the more complex Python oswalk script which will return a string of dwgs separated with commas, so if you have any tips on how FME will behave with that next step, that would be appreciated also.

Thank you,

Tyler

FME(R) 2020.1.2.0 (20200902 - Build 20620 - WIN64)

icon

Best answer by rahulsharma 24 February 2021, 17:26

View original

13 replies

nielsgerrits
Rank
Userlevel 6
Badge +32

Not sure what you try to do exactly, but this just works for me.

I always use the Generic way (single output port) when using scripted files.

FeatureReaderParameters

townest
Rank
Badge +4

I'm trying to read a list (or string) of file names/paths generated from a Python Script. For testing I am using a simple script with a single hard coded path, but I couldn't even get that to work. I'll try the Generic way you mentioned.

townest
Rank
Badge +4

No luck with generic transformer. There is something I'm not understanding with how a private user parameter return info to dataset fields. Any pointers would be appreciated.

Thank you.

Tyler

townest
Rank
Badge +4

Hi All,

 

I am adding some screenshots to help define my question/issue.

 

--Reader with Dataset linked to private parameter osWalk:updateReader 

--Private Parameter Definition with Value holding Python script:paramDefPython script: Simple hardcoded file raw filepath to be returned.

dwgscriptI am getting 'Undefined Macro, Update Reader failed' when updating the reader with this setup. I feel like the Python might not firing and therefore not returning the file path string. On the other hand, if it is firing, it is not clear to me that the Dataset field is accepting the returned file path string. Is there any way to test private parameter scripts? What data types and format does a reader dataset field accept from Python? Any recommended fixes or other pointers would be appreciated.

 

Sincerely,

Tyler

rahulsharma
Rank
Badge +8

Hi @townest​ 

Please find attached a small workspace based on your description. 

import os
file = r'C:\temp\TrafficSignals.dwg'
return file

Hope this helps! 

D
Badge +4

Hi @townest​ ,

 

to get a list of files you can use the Directory and File Pathnames Reader (this is the simplest and best choice). 

 

If, for some reason, you have to go on with a python script, you could try to define a private parameter of type "Scripted (Python)" and in the "Value" parameter write a script that returns what you want. This will become the value of the parameter.

Here is an example I use to get the date and time of workspace execution:

# Import fme module and datetime.datetime class
import fme
from datetime import datetime
 
# Use the now() method to get the current date and time
# Format date as YYYYMMDD_HHMMSS with strftime method
current_time = datetime.now().strftime("%Y%m%d_%H%M%S")
 
# Return unique current time which becomes the value of the scripted parameter
return current_time

Maybe you can adapt the example to your case.

 

Hope that helps!

townest
Rank
Badge +4

Ok. I made some progress. My original script actually was working, I'm just trying to get used to FME behavior. "Updating" the Reader in the navigator was erroring out, however "Running" the Reader in the workspace worked just fine. Ok, baby steps. Read one file from scripted private parameter. Check. Now, how can I read a list of files from scripted private parameter. Recall, my original script, which works:

----

file = r'C:\\data\\dwgfoldertest\\Test1.dwg'

return file

-----

 

Now, for a list I have tried the following:

----

file = r'C:\\data\\dwgfoldertest\\Test1.dwg, C:\\data\\dwgfoldertest\\Test2.dwg'

return file

-----

and I have tried...

----

file = r'C:\\data\\dwgfoldertest\\Test1.dwg C:\\data\\dwgfoldertest\\Test2.dwg'

return file

-----

and I have tried...

----

file = [r'C:\\data\\dwgfoldertest\\Test1.dwg, r'C:\\data\\dwgfoldertest\\Test2.dwg']

return file

-----

 

Any help in returning a list of files from a scripted private parameter to a dataset field would be appreciated. I am comfortable writing Python scripts, so my question is focusing on FME behavior and what FME expects as script returns. In other words, If I understand what FME is expecting as a return I can write the script to return that value and format. Thank you.

Tyler

 

nielsgerrits
Rank
Userlevel 6
Badge +32

Ok. I made some progress. My original script actually was working, I'm just trying to get used to FME behavior. "Updating" the Reader in the navigator was erroring out, however "Running" the Reader in the workspace worked just fine. Ok, baby steps. Read one file from scripted private parameter. Check. Now, how can I read a list of files from scripted private parameter. Recall, my original script, which works:

----

file = r'C:\\data\\dwgfoldertest\\Test1.dwg'

return file

-----

 

Now, for a list I have tried the following:

----

file = r'C:\\data\\dwgfoldertest\\Test1.dwg, C:\\data\\dwgfoldertest\\Test2.dwg'

return file

-----

and I have tried...

----

file = r'C:\\data\\dwgfoldertest\\Test1.dwg C:\\data\\dwgfoldertest\\Test2.dwg'

return file

-----

and I have tried...

----

file = [r'C:\\data\\dwgfoldertest\\Test1.dwg, r'C:\\data\\dwgfoldertest\\Test2.dwg']

return file

-----

 

Any help in returning a list of files from a scripted private parameter to a dataset field would be appreciated. I am comfortable writing Python scripts, so my question is focusing on FME behavior and what FME expects as script returns. In other words, If I understand what FME is expecting as a return I can write the script to return that value and format. Thank you.

Tyler

 

Maybe it is easier to run the Python code in a PythonCaller and send the results to a FeatureReader. Something like Creator, PythonCaller, ListExploder, FeatureReader.

 

You also can put the results of the list in the Published Parameter and split the list to Features in workbench, then starting the FeatureReader. Something like Creator, AttributeCreator, AttributeSplitter, ListExploder, FeatureReader.

townest
Rank
Badge +4

Maybe it is easier to run the Python code in a PythonCaller and send the results to a FeatureReader. Something like Creator, PythonCaller, ListExploder, FeatureReader.

 

You also can put the results of the list in the Published Parameter and split the list to Features in workbench, then starting the FeatureReader. Something like Creator, AttributeCreator, AttributeSplitter, ListExploder, FeatureReader.

I've considered that but the Readers play nicer than the FeatureReader when it comes to exposing attributes. Plus, it seems I'm very close, this method is clean and it works fine with one file, just need that tip from someone out there of how to return more than one file to the dataset field. Anybody?

Thanks,

Tyler

D
Badge +4

Maybe it is easier to run the Python code in a PythonCaller and send the results to a FeatureReader. Something like Creator, PythonCaller, ListExploder, FeatureReader.

 

You also can put the results of the list in the Published Parameter and split the list to Features in workbench, then starting the FeatureReader. Something like Creator, AttributeCreator, AttributeSplitter, ListExploder, FeatureReader.

Hi @townest​ ,

 

have a look at article Python Scripted Parameters in FME, I think there are some useful ideas.

 

Hope that helps!

 

townest
Rank
Badge +4

Maybe it is easier to run the Python code in a PythonCaller and send the results to a FeatureReader. Something like Creator, PythonCaller, ListExploder, FeatureReader.

 

You also can put the results of the list in the Published Parameter and split the list to Features in workbench, then starting the FeatureReader. Something like Creator, AttributeCreator, AttributeSplitter, ListExploder, FeatureReader.

I had seen those examples. None of those examples pass a list to a dataset field.

rahulsharma
Rank
Badge +8

Hi @townest​ 

Please find a sample python script. 

  • The script will take a list of all the files and convert them to string 
  • Join the list with "spaces", to mimic adding multiple files in dataset required a space delimited list of files
  • Finally, add double quotes around the files name(as required) 
import os
file_list = [r'C:\TEMP\Data1.csv', r'C:\TEMP\Data2.csv']
#Surround each file in list with double quotes and join with space delimiter
ListToStr = ' '.join(f"\"{f}\"" for f in file_list)
#print(f"\"{ListToStr}\"")
#Return with string with double quotes around entire string
return f"\"{ListToStr}\""

 

 

townest
Rank
Badge +4

Hi @townest​ 

Please find a sample python script. 

  • The script will take a list of all the files and convert them to string 
  • Join the list with "spaces", to mimic adding multiple files in dataset required a space delimited list of files
  • Finally, add double quotes around the files name(as required) 
import os
file_list = [r'C:\TEMP\Data1.csv', r'C:\TEMP\Data2.csv']
#Surround each file in list with double quotes and join with space delimiter
ListToStr = ' '.join(f"\"{f}\"" for f in file_list)
#print(f"\"{ListToStr}\"")
#Return with string with double quotes around entire string
return f"\"{ListToStr}\""

 

 

That's the answer!  I needed the syntax and data type of the object to pass back to FME.  A string, double quoted elements, single spaced between elements, finally all encase in double quotes.  Thank you @rahulsharma​ !

Reply


Community Stats

28,716
Posts
109,494
Replies
36,561
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.