Skip to main content

I'm trying to sort through huge amounts of data.

I have already run a directory file reader and have a huge CSV full of all the possible files in a directory including their name, path and extension etc.

I now would like to push those file types into a new folder.

This is for non-spatial, so a different folder for csv's, xsl, xlsx, doc, jpeg etc etc

By doing this I can then investigate these futher.

Work so far:

Reader: Read all files in directory and sub-directory that are, for example, a .csv

I get an error and it won't create the reader, so I create it as a single CSV and then change the directory to the folder with \\**\\* at the and so it looks through all sub-directories.

Writer is set to Dynamic and the same file type.

I get errors from any of the above workings.

Alternatively I feel like the original csv I have with all the directories included (including file path and extension) could be utilised for this?

When I pull that CSV in it shows up like this instead of broken down, so I can't run a TestFilter.

Any help appreciated.

If you just need to move/copy files to a new location it's not necessary to read them. The filecopywriter will let you do this from the csv file you're already produced.


If you just need to move/copy files to a new location it's not necessary to read them. The filecopywriter will let you do this from the csv file you're already produced.

Hi, could you expand on this a bit more? I can't work out how that works.


So if your start point is a csv file with information about files, their extensions and locations

Read the csv file

test the attribute that holds the extension so you only keep the csv files

create 3 new attributes

filecopy_source_dataset - the location of the original file to be moved

filecopy_dest_dataset -the new folder you want the file moved to

filecopy_dest_filename - the name you want the new file to be called (this can just be the same as the existing)

send to the file copy writer

 


Reply