Skip to main content
Question

Write CSV file where filename includes date/timestamp of translation - currently getting multiple CSVs

  • October 26, 2016
  • 7 replies
  • 107 views
tim_wood
Contributor
Forum|alt.badge.img+8

I'm removing duplicates from a File Geodatabase (FGDB) feature class using Sorter then DuplicateFilter, then writing the Unique features back to the same FGDB feature class.

I want to write the Duplicate features to a CSV log file with the date/time of the translation in the file name so I have a CSV listing the duplicates removed each time the Workspace is run.

First I tried including @Timestamp(^Y^m^d^H^M^S) in the CSV File Name in the Writer but I ended up with multiple CSVs, presumably because features were being written as they were received by the Writer. I haven't noticed this problem with FGDBs but is that because of difference in the way the FGDB Writer works?

I then tried a Creator followed by a TimeStamper running parallel to the actual data processing with the output from the TimeStamper going to the CSV Writer along with the data. But what I get is two CSV files - one with no date/time stamp in the name and all the data, and another with the date/time stamp in the name but no data.

Would "Append to file" in the CSV Writer properties solve this? If so, how can I ensure that the CSV file with the date/time stamp in the name is created first?

7 replies

tim_wood
Contributor
Forum|alt.badge.img+8
  • tim_woodContributor
  • Author
  • Contributor
  • October 26, 2016
I know I could record the date/time against each feature and append to a single log file but I'd prefer the clarity of separate files.

 

ebygomm
Influencer
Forum|alt.badge.img+32
  • ebygommInfluencer
  • Influencer
  • October 26, 2016

You need to merge the timestamper with the rest of your data, e.g. with a FeatureMerger

david_r
Evangelist
  • david_rEvangelist
  • October 26, 2016

I agree with @egomm, but I would highly recommend setting "Suppliers first" to yes to avoid it blocking the data flow and consuming huge amounts of memory if you have a lot of data.

Another way to do it is to use a private Python scripted parameter to give you the timestamp at the start of the translation, e.g.

from datetime import datetime
return datetime.now().strftime('%Y%m%d%H%M%S')

Useful if you need the same timestamp many places in your workspace and you want to avoid all those FeatureMergers.

ebygomm
Influencer
Forum|alt.badge.img+32
  • ebygommInfluencer
  • Influencer
  • October 26, 2016

You could also use a variable setter and a variable retriever

takashi
Influencer
  • takashiInfluencer
  • October 26, 2016

Yes, there is more than one way. Another approach: once create the destination CSV file with temporary name (e.g. "temp.csv") by a FeatureWriter, and then rename it to the timestamp. i.e. move the file finally.

A
ebygomm wrote:

You need to merge the timestamper with the rest of your data, e.g. with a FeatureMerger

I am having the same issue and tried this solution but the I'm stumped on the join. I can choose _timestamp in supplier but what do I choose to join on Requester? If I choose a unique identifier nothing outputs from the Merged.

erik_jan
Contributor
Forum|alt.badge.img+17
  • erik_janContributor
  • Contributor
  • August 28, 2019
anthonystokes wrote:

I am having the same issue and tried this solution but the I'm stumped on the join. I can choose _timestamp in supplier but what do I choose to join on Requester? If I choose a unique identifier nothing outputs from the Merged.

You need an unconditional join (1 = 1) to get the timestamp on all features.

Reply


Helpful Members This Week

Community Stats

31,135
Posts
118,046
Replies
38,777
Members

Latest FME

Terms & ConditionsCookie settings

Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.