Skip to main content
Solved

How to feed result of FILECOPY operation to a Reader?


mattwilkie
Supporter
Forum|alt.badge.img+11

I have a workspace that copies a file from one location to another. Now I want to use the copied file as input to a reader. How do I do that?

 

how-use-copy-output

Best answer by mattwilkie

Thanks @caracadrian​. What I figured out from your pointers:

 For a complete working process chain one needs:

Creator >> FeatureWriter set to `filecopy` >> AttributeExposer with manually entered values of `filecopy_source_dataset, filecopy_dest_dataset, filecopy_filename` >> fed to StringConcatenator in order to build the full path name to >> feed to FeatureReader.

Don't forget to enable Output Ports on the FeatureWriter, it defaults to None. Otherwise you'll only be able to read the Summary in AttributeExposer.

complete-filecopy-to-reader2021-04-16 10_59_43-AttributeExposer Parameters2021-04-16 11_00_16-StringConcatenator ParametersCaveat:

In filecopy you must define `filecopy_dest_dataset`. It's tempting to use `_dataset` from Summary instead, which is taken from the writer's "dataset" parameter, but the later StringConcatenator won't use it as expected. (Instead of a single combined path output it will yield 2 output rows. One for the connector path of Attribute Exposer and one from the Summary connector.)

In StringConcatenator it's up to you to ensure the combined path string is valid (there doesn't seem to be an equivalent to Python's os.path.join() which validates the path segments and connectors to ensure no invalid characters).

 

This took a couple hours to sort out. I guess folks must usually resort to doing this kind of thing directly in sub process command shell or python script as it's pretty fiddly doing this in workbench.

 

View original
Did this help you find an answer to your question?

4 replies

caracadrian
Contributor
Forum|alt.badge.img+23
  • Contributor
  • April 15, 2021

Use the FeatureWriter set to File Copy instead of "classic" writer.

It will output attributes for source and destination datasets, as well as the destination filename.


mattwilkie
Supporter
Forum|alt.badge.img+11
  • Author
  • Supporter
  • April 15, 2021
caracadrian wrote:

Use the FeatureWriter set to File Copy instead of "classic" writer.

It will output attributes for source and destination datasets, as well as the destination filename.

Umm. I don't see these output attributes. Where are they located?

2021-04-15 16_05_27-_NONE → NONE (Untitled) - FME Workbench 2020.22021-04-15 16_02_31-_NONE → NONE (Untitled) - FME Workbench 2020.2 

 


caracadrian
Contributor
Forum|alt.badge.img+23
  • Contributor
  • April 16, 2021

Use AttributeExposer and enter them manually


mattwilkie
Supporter
Forum|alt.badge.img+11
  • Author
  • Supporter
  • Best Answer
  • April 16, 2021

Thanks @caracadrian​. What I figured out from your pointers:

 For a complete working process chain one needs:

Creator >> FeatureWriter set to `filecopy` >> AttributeExposer with manually entered values of `filecopy_source_dataset, filecopy_dest_dataset, filecopy_filename` >> fed to StringConcatenator in order to build the full path name to >> feed to FeatureReader.

Don't forget to enable Output Ports on the FeatureWriter, it defaults to None. Otherwise you'll only be able to read the Summary in AttributeExposer.

complete-filecopy-to-reader2021-04-16 10_59_43-AttributeExposer Parameters2021-04-16 11_00_16-StringConcatenator ParametersCaveat:

In filecopy you must define `filecopy_dest_dataset`. It's tempting to use `_dataset` from Summary instead, which is taken from the writer's "dataset" parameter, but the later StringConcatenator won't use it as expected. (Instead of a single combined path output it will yield 2 output rows. One for the connector path of Attribute Exposer and one from the Summary connector.)

In StringConcatenator it's up to you to ensure the combined path string is valid (there doesn't seem to be an equivalent to Python's os.path.join() which validates the path segments and connectors to ensure no invalid characters).

 

This took a couple hours to sort out. I guess folks must usually resort to doing this kind of thing directly in sub process command shell or python script as it's pretty fiddly doing this in workbench.

 


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings