Question

Output directory and file name based on input directory and input attributes

A

Folks,

I've read over numerous similar questions and they all slip just past the mark I'm trying to hit. This link is the closest I got to something like what I'm after. https://knowledge.safe.com/questions/4653/naming-an-output-file-based-on-parts-of-the-reader.html

Just a quick summary on the workspace. I'm reading a number of Mapinfo table and writing to a single GML file.

I should explain I'm still a little lost as to how to use the following transformers but the link describes using FileNamePartExtractor to extract the directory path.

The issue I've run into is my Readers were opened as a group(directory). When I try to use FileNamePartExtractor I get ALL of the filenames and their paths in the "_dirpath" output port. In a sense this makes sense but it doesn't help me. I was able to make up a filename based on attributes using StringConcatenator. However I'm still not clear how the final output string I called "_fullpathfilename" to be exposed in the "fanout" for the writer.

I hope I've explained my problem well enough.

As always advice appreciated.

12 replies

redgeographics
Rank
Userlevel 4
Badge +25

If you want to fanout the name of the GML file you have to look in the navigator. Underneath the writer you'll see an option "Fanout Dataset" where you can construct the path- and filename of the GML. You should be able to use the attribute there.

(you'll see a different list of attributes of course).

You can also use the text editor interface to construct the filename.

A

If you want to fanout the name of the GML file you have to look in the navigator. Underneath the writer you'll see an option "Fanout Dataset" where you can construct the path- and filename of the GML. You should be able to use the attribute there.

(you'll see a different list of attributes of course).

You can also use the text editor interface to construct the filename.

Thanks @redgeographics. That's my expectation but not what I'm seeing. I see a bunch of attributes but not my attribute based on the FileNamePartExtractor and StringConcatentor. TBH I'm not sure what I'm supposed to do with them. I placed them and then added and Inspector to look at them and notwithstanding the 21 paths because I have 21 MI Tables I'm not getting anything. I tried hot wiring it and just putting "_fullpathfilename" and @value(_fullpathfilename) but all I get is a file called "_fullpathfilename" or "@value(_fullpathfilename)". Any clues how you expose the attribute? i should add none of the standard _dirpath, _rootname etc from the FileNamePartExtractor are visible either.

 

redgeographics
Rank
Userlevel 4
Badge +25
Thanks @redgeographics. That's my expectation but not what I'm seeing. I see a bunch of attributes but not my attribute based on the FileNamePartExtractor and StringConcatentor. TBH I'm not sure what I'm supposed to do with them. I placed them and then added and Inspector to look at them and notwithstanding the 21 paths because I have 21 MI Tables I'm not getting anything. I tried hot wiring it and just putting "_fullpathfilename" and @value(_fullpathfilename) but all I get is a file called "_fullpathfilename" or "@value(_fullpathfilename)". Any clues how you expose the attribute? i should add none of the standard _dirpath, _rootname etc from the FileNamePartExtractor are visible either.

 

If the attribute is visible in the Data Inspector but not selectable in Workbench try an AttributeExposer to expose it. That way you can use it.

 

 

If you don't see the attribute make sure the output from the FileNamePartExtractor and StringConcatenator is actually connected to the writer. Only attributes going in to the writer can be used for the fanout (they don't have to be actually written to the output though).

 

 

Can you post a screenshot of your workspace? (or the actual workspace)

 

 

takashi
Rank
Userlevel 2
Badge +17
Thanks @redgeographics. That's my expectation but not what I'm seeing. I see a bunch of attributes but not my attribute based on the FileNamePartExtractor and StringConcatentor. TBH I'm not sure what I'm supposed to do with them. I placed them and then added and Inspector to look at them and notwithstanding the 21 paths because I have 21 MI Tables I'm not getting anything. I tried hot wiring it and just putting "_fullpathfilename" and @value(_fullpathfilename) but all I get is a file called "_fullpathfilename" or "@value(_fullpathfilename)". Any clues how you expose the attribute? i should add none of the standard _dirpath, _rootname etc from the FileNamePartExtractor are visible either.

 

For example, if all the features have these two common attributes

 

  • _directory = "output/data"
  • _filename = "result.gml"
the writer with this Fanout setting writes the features into "C:/root/output/data/result.gml".

 

  • Destination Fanout Directory: C:/root
  • Fanout Expression: @Value(_directory)/@Value(_filename)
Note that FME is case sensitive on function names. That is, @Value function should start with @ and upper case V always.
A
If the attribute is visible in the Data Inspector but not selectable in Workbench try an AttributeExposer to expose it. That way you can use it.

 

 

If you don't see the attribute make sure the output from the FileNamePartExtractor and StringConcatenator is actually connected to the writer. Only attributes going in to the writer can be used for the fanout (they don't have to be actually written to the output though).

 

 

Can you post a screenshot of your workspace? (or the actual workspace)

 

 

"Only attributes going in to the writer can be used for the fanout (they don't have to be actually written to the output though)."

 

That was the clue I needed. I still have the issue of a group of tables all wanting to contribute their paths to the "_dirpath" so I end up with a massively long string but now I can access the attribute.

 

Any clues on how I might get just one of the "_dirpath" from my group of MI tables?

 

 

A
For example, if all the features have these two common attributes

 

  • _directory = "output/data"
  • _filename = "result.gml"
the writer with this Fanout setting writes the features into "C:/root/output/data/result.gml".

 

  • Destination Fanout Directory: C:/root
  • Fanout Expression: @Value(_directory)/@Value(_filename)
Note that FME is case sensitive on function names. That is, @Value function should start with @ and upper case V always.
Thanks @takashi and @redgeographics. I think my last stumbling block is actually the FilenamePartExtractor "Source Filename" parameter. I presume this needs to be some kind of variable as to "hard code" and actual filename and path would defeat the purpose of using this process to automate the filename and placement of the output file.

 

I used $(SourceDataset_MAPINFO) and since I added all the files as a group I get ALL of the filename paths. I need a system variable or similar that targets one specific file in the workspace. I presumed that if you connected the FileNamePartExtractor to a particular reader there would be a variable exposed that would target that Readers source. Thoughts?

takashi
Rank
Userlevel 2
Badge +17
For example, if all the features have these two common attributes

 

  • _directory = "output/data"
  • _filename = "result.gml"
the writer with this Fanout setting writes the features into "C:/root/output/data/result.gml".

 

  • Destination Fanout Directory: C:/root
  • Fanout Expression: @Value(_directory)/@Value(_filename)
Note that FME is case sensitive on function names. That is, @Value function should start with @ and upper case V always.
The FilenamePartExtractor parses the string value specified in the Source Filename parameter by splitting it with the path delimiter character (/ or \\). Naturally it wouldn't work as expected if the specified string was not the full path of a single file. I don't think the $(SourceDataset_MAPINFO) is a single full path, since you have select multiple files as the dataset.

 

Why not set "fme_dataset" to the parameter? You can expose "fme_dataset" in the reader feature type.

 

 

redgeographics
Rank
Userlevel 4
Badge +25
"Only attributes going in to the writer can be used for the fanout (they don't have to be actually written to the output though)."

 

That was the clue I needed. I still have the issue of a group of tables all wanting to contribute their paths to the "_dirpath" so I end up with a massively long string but now I can access the attribute.

 

Any clues on how I might get just one of the "_dirpath" from my group of MI tables?

 

 

Are those _dirpath values all the same?

 

 

A
Are those _dirpath values all the same?

 

 

They are the same @redgeographics. However the whole thing is a bust. When I use the fanout a hidden trap emerged. My GML isn't pure GML. The schema is actually a blend of XML with GML embedded in it. When I used the fanout the GML got separated from the XML. Made a real mess. I'm suspicious I may need a custom function to do what I want to do and that's really not my area of expertise.

 

redgeographics
Rank
Userlevel 4
Badge +25

Okay, if you're trying to write to a hybrid XML/GML format take a look at the XMLTemplater. It involves a lot more work though.

A

Okay, if you're trying to write to a hybrid XML/GML format take a look at the XMLTemplater. It involves a lot more work though.

Thanks for continuing to look at this @redgeographics. The writer is largely doing what I want to do so that's not the issue as such. I was attempting to use the "fanout" as an exploit to achieve the automated naming and file placement of the final XML/GML.

 

 

Is there no way to provide an automation for the file naming and output directory that doesn't use the "fanout"?

 

redgeographics
Rank
Userlevel 4
Badge +25
Thanks for continuing to look at this @redgeographics. The writer is largely doing what I want to do so that's not the issue as such. I was attempting to use the "fanout" as an exploit to achieve the automated naming and file placement of the final XML/GML.

 

 

Is there no way to provide an automation for the file naming and output directory that doesn't use the "fanout"?

 

You could use a FeatureWriter transformer instead of a writer, that should give you similar options for making the filename (and path) dynamic

 

 

Reply


Community Stats

28,666
Posts
109,340
Replies
36,535
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.