I would suggest that you use DuplicateFilter Transformer (doc). You may need to put Sorter if front of that. Specify key attributes to compare/find duplicates, then changes will be output via a 'unique' port. To differentiate b/w files, you could set an attribute 'Source' or expose fme_dataset (which I believe will = to filename).
I hope it helps.
Sasha
from the look of it you have not set the transformer parameters, hence the yellow color of the ... button on the thransformer.
I would suggest that you use DuplicateFilter Transformer (doc). You may need to put Sorter if front of that. Specify key attributes to compare/find duplicates, then changes will be output via a 'unique' port. To differentiate b/w files, you could set an attribute 'Source' or expose fme_dataset (which I believe will = to filename).
I hope it helps.
Sasha
I'm using v.2013 so it doesn't have the DuplicateFilter transformer.
I used sorter and DuplicateRemover and got 2 csv output files but both are blank..
from the look of it you have not set the transformer parameters, hence the yellow color of the ... button on the thransformer.
Ah you're right. Just changed that but now I'm not getting any output files?
Ah you're right. Just changed that but now I'm not getting any output files?
It looks like you accidentally disabled the transformer, It's grey instead of blue.
It looks like you accidentally disabled the transformer, It's grey instead of blue.
I did indeed. Enabled it then ran but only got unchanged output this time and was still blank.
Hi @crow132, I would recommend you to investigate how the CSV reader has read the source tables. Connect two Inspector transformers to the reader feature types for each and run. You can then see the contents of the tables read by the reader with FME Data Inspector (or FME Universal Viewer in 2013?).
Hi @crow132, I would recommend you to investigate how the CSV reader has read the source tables. Connect two Inspector transformers to the reader feature types for each and run. You can then see the contents of the tables read by the reader with FME Data Inspector (or FME Universal Viewer in 2013?).
I've ran it with the inspectors attached but the output shows me nothing. The readers do pick up the field names (name, job and school) so not sure what I'm missing?
I've ran it with the inspectors attached but the output shows me nothing. The readers do pick up the field names (name, job and school) so not sure what I'm missing?
FME Universal Viewer was not good to inspect non-spatial features. Change inspection application to FME Data Inspector, and then retry.
To change the option, open the FME Options dialog (Workbench menu: Tools > FME Options ) and check "Use FME Data Inspector".
FME Universal Viewer was not good to inspect non-spatial features. Change inspection application to FME Data Inspector, and then retry.
To change the option, open the FME Options dialog (Workbench menu: Tools > FME Options ) and check "Use FME Data Inspector".
Ok so I changed that and this is the output. Stil get a blank csv output file even though the reader is reading the input files. Each of those tabs down the bottom return 1 row from the input csv files.
I've ran it with the inspectors attached but the output shows me nothing. The readers do pick up the field names (name, job and school) so not sure what I'm missing?
Looks like the reader has read the tables correctly. I guess you have set the three attributes (name, job, school) to the Group By parameter in the Inspector. If you clear the parameter, entire tables appear in the Table View and you can compare the two tables.
Looks like the reader has read the tables correctly. I guess you have set the three attributes (name, job, school) to the Group By parameter in the Inspector. If you clear the parameter, entire tables appear in the Table View and you can compare the two tables.
Yes that worked, I can see the full tables of the two input csv files. However I'm still getting blank output files for deleted/unchanged/added. I'm thinking (like with python) that because each output has more than one (unchanged 5 and the others 2) output link after the transformer that they're overwriting themselves and this turns up with blank csvs (well it happens with python I've found). Would that be correct?
I've ran it with the inspectors attached but the output shows me nothing. The readers do pick up the field names (name, job and school) so not sure what I'm missing?
Next, inspect the output features from the ChangeDetector with the Inspectors as well.
I expect that:
- The output features from the ChangeDetector have your desired attributes,
- but the CSV writer won't write them into the destination files.
If so, the schema setting of writer feature types could be wrong. The writer feature type setting should look like this.
Next, inspect the output features from the ChangeDetector with the Inspectors as well.
I expect that:
- The output features from the ChangeDetector have your desired attributes,
- but the CSV writer won't write them into the destination files.
If so, the schema setting of writer feature types could be wrong. The writer feature type setting should look like this.
Ahhh yes you are correct. Those are blank for all 3 writers. If I manually added in their names (name, job, school) my output CSVs come out correctly with visible data. Although I shouldn't have to type the user attribute names out, surely there is an easier way, especially with lots of attributes? In the format parameters tab I tried inherit from writer, yes and append from the source dataset but all came up with blank csvs.
Ahhh yes you are correct. Those are blank for all 3 writers. If I manually added in their names (name, job, school) my output CSVs come out correctly with visible data. Although I shouldn't have to type the user attribute names out, surely there is an easier way, especially with lots of attributes? In the format parameters tab I tried inherit from writer, yes and append from the source dataset but all came up with blank csvs.
Yes, there is a way not to type individual attributes manually, and the FME Knowledge Center provides a good tutorial.
Tutorial: Dynamic Workflows
Although the articles in the tutorial are created with FME 2016+ and details are different from FME 2013, basic concept is common to every version. Good luck!
FME Universal Viewer was not good to inspect non-spatial features. Change inspection application to FME Data Inspector, and then retry.
To change the option, open the FME Options dialog (Workbench menu: Tools > FME Options ) and check "Use FME Data Inspector".
Agreed. As far as I've understood it, the Universal Viewer will no longer be available starting FME 2017.
