Skip to main content
Question

Dynamic Schema and file suffixes

  • September 9, 2013
  • 6 replies
  • 6 views

Forum|alt.badge.img+1
Hi

 

 

I am using Dynamic Schema to translate a large number of shapefiles from one coordinate system to another, but keeping the same schema. 

 

 

I am able to do this successfully however I cannot work out how to suffix each translated shapefile so as to differentiate the translated files from the non-translated files.

 

 

I have tried setting the Fanout Parameter to Yes, but it only suffixes the folder with the translated files, not the actual shapefiles themselves.

 

 

Does anyone know how to do this?

 

 

Thankyou

 

 

6 replies

david_r
Celebrity
  • September 9, 2013
Hi,

 

 

look at the Fanout Dataset under the Writer in the Navigator tree (and not under the Fanout option in the output feature type). You can specify a suffix there:

 

 

 

 

David

Forum|alt.badge.img+1
  • Author
  • September 9, 2013
Thanks David,

 

 

That is where I am setting the suffix, but it is only appending it to the output folder and not the actual shapefiles.

takashi
Evangelist
  • September 9, 2013
Hi,

 

 

When you specify a source reader schema as the schema source of the writer, I don't think you can change the feature type name (i.e. the destination file name) in the workflow. If I am wrong, please correct.

 

 

One possible solution is to rename the destination file names using the Shutdown Python Script after the translation completing. For example:

 

-----

 

# Assume there are no sub-directories in the destination directory

 

# and also every file name has an extension (.*).

 

# If you defined the suffix as a user parameter, you can get it with:

 

# suffix = FME_MacroValues['<parameter name>']

 

import os   dir = FME_MacroValues['DestDataset_SHAPE'] suffix = '-fixed' for src in os.listdir(dir):     t = src.split('.')     dest = '%s%s.%s' % ('.'.join(t[0:-1]), suffix, t[-1])     os.rename('%s/%s' % (dir, src), '%s/%s' % (dir, dest))

 

-----

 

 

Takashi

 


fmelizard
Safer
Forum|alt.badge.img+19
  • Safer
  • September 10, 2013
Hi,

 

I agree with Takashi, the properties are extracted from the input schema  in a dynamic translation. 

 

The option provided to use the end python is a neat one.

 

 

Itay

Forum|alt.badge.img+1
  • Author
  • September 11, 2013
Hi guys,

 

 

thankyou for your answers.  I actually worked out how to add a suffix while still using Dynamic schema by using the String Concatenator transformer and changing the Feature Type Name in the writer properties to "From Attribute".

 

 

When i run the workbench it works for the first 5 files, but when it gets to the next lot of files which are considerably larger >100MB each, it crashes and errors come up relating to lack of memory space.

 

 

When i do the same translation on a single file >100 MB without using Dynamic Schema, I don't get this memory issue and it works successfully.

 

 

Therefore I am wondering if using the Dynamic Schema definition somehow does something to memory?  If so, is there a way to fix this so I can still use Dynamic Schema definition and process the large files?

 

 

Thanks

takashi
Evangelist
  • September 11, 2013
Hi,

 

 

I didn't know "From Attribute" option for "Feature Type Name" parameter.

 

I confirmed that the option works well. Thank you for sharing the information.   I currently have no idea about the memory issue. If you have not solve it yet, how about posting as a new question?

 

 

Takashi

Reply


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings