Directory and File Pathnames
Directory and File Pathnames
The Directory and File Path Reader is definately the way to go, you can also look to join this metadata onto a spatial object that reflects the extents of your files and then colour them based on if they have been modified or not. If you fast forward 18 minutes into this movie you'll see an example of what I mean
http://youtu.be/ejszB6VWnKE hope it helps.
Cheers, Dave
The Directory and File Path Reader is definately the way to go, you can also look to join this metadata onto a spatial object that reflects the extents of your files and then colour them based on if they have been modified or not. If you fast forward 18 minutes into this movie you'll see an example of what I mean http://youtu.be/ejszB6VWnKE hope it helps.
Cheers, Dave
Hi, thanks for the answer SigTill this is exactly what I was after.
Dave, also very interesting, I think I'll intergrate your spatial solution too. They actually want an visual tree showing the sub-folders going into sub-folders then listing files that have are colour coded depending on frequency access.
Thanks for your help guys.
Great, this is almost what I was looking for...I would however like to complicate a bit
Is it possible to read the modified date from inside a zip file? So basically treat the zip file as a folder and give attributes of the contents inside...
I would like to get modified dates, but the date of the zip file seldom is the same as the actual data inside. And I would rather not open the zips.
I'm trying to do something similar to the above, but the difference being that I don't want to specify a folder in the directory and file path reader, I want to specify certain file names. I want to use Directory and file path reader because I would like the date modified and file size as attributes but it wont let me do this for specific files. I have over 1000 files I need these attributes for, and the files themselves are spread across many folders and subfolders and I don't want to lump all these files into one folder. have tried using FilenamePartExtractor then attempting to use "_dirpath" attribute in the Dataset field of the FeatureReader transformer where the format is set to Directory and file paths but this doesn't work - FME keeps wanting me to specify the folder! I'm finding this quite frustrating as this as I want to tack these attributes onto my existing workspace which examines the geometry of the files (using a .bat file). Any suggestions would be greatly received or perhaps I should create a new question for this.
I'm trying to do something similar to the above, but the difference being that I don't want to specify a folder in the directory and file path reader, I want to specify certain file names. I want to use Directory and file path reader because I would like the date modified and file size as attributes but it wont let me do this for specific files. I have over 1000 files I need these attributes for, and the files themselves are spread across many folders and subfolders and I don't want to lump all these files into one folder. have tried using FilenamePartExtractor then attempting to use "_dirpath" attribute in the Dataset field of the FeatureReader transformer where the format is set to Directory and file paths but this doesn't work - FME keeps wanting me to specify the folder! I'm finding this quite frustrating as this as I want to tack these attributes onto my existing workspace which examines the geometry of the files (using a .bat file). Any suggestions would be greatly received or perhaps I should create a new question for this.
I think this question deserves it's own page .
But maybe Python is the way to go. If you can specify one parent folder, it is possible to loop through all subfolders and files and test if this is the file you are looking for.
Of course this isn't efficient since it will loop through all files and folders but using the os.path.getsize()- and os.path.getmtime()-functionality, you can retrieve the size and last modification time of the file easily.
Great, this is almost what I was looking for...I would however like to complicate a bit
Is it possible to read the modified date from inside a zip file? So basically treat the zip file as a folder and give attributes of the contents inside...
I would like to get modified dates, but the date of the zip file seldom is the same as the actual data inside. And I would rather not open the zips.
You will most likely have to use something like Python for this. There are some good examples on this site.
I'm trying to do something similar to the above, but the difference being that I don't want to specify a folder in the directory and file path reader, I want to specify certain file names. I want to use Directory and file path reader because I would like the date modified and file size as attributes but it wont let me do this for specific files. I have over 1000 files I need these attributes for, and the files themselves are spread across many folders and subfolders and I don't want to lump all these files into one folder. have tried using FilenamePartExtractor then attempting to use "_dirpath" attribute in the Dataset field of the FeatureReader transformer where the format is set to Directory and file paths but this doesn't work - FME keeps wanting me to specify the folder! I'm finding this quite frustrating as this as I want to tack these attributes onto my existing workspace which examines the geometry of the files (using a .bat file). Any suggestions would be greatly received or perhaps I should create a new question for this.
Do you know the names of the files ahead of time? If so, is it possible to add these to a spreadsheet and then use a FeatureMerger of Joiner to join them to the incoming features from the Directory and File Pathnames Reader. Any that join are the ones you want then...
Does that help?
The Directory and File Path Reader is definately the way to go, you can also look to join this metadata onto a spatial object that reflects the extents of your files and then colour them based on if they have been modified or not. If you fast forward 18 minutes into this movie you'll see an example of what I mean http://youtu.be/ejszB6VWnKE hope it helps.
Cheers, Dave
Thanks Dave this video was very useful. I must admit I had seen it before but not fully taken it in. I just need to specify a number of high level folders in the Directory and File Path Reader and then merge the output on say a concatenated filename/path field. Also, I should probably use a workspace runner to collect the file path/date modified metadata before running my geometry workspace. I was hoping for a way to feed the folder path (as an attribute) into the Directory and File Path Reader (via feature reader) as a way of limiting the amount of unnecessary searching/trawling through subfolders given that in this instance I know exactly where the files are! Thanks again!
Do you know the names of the files ahead of time? If so, is it possible to add these to a spreadsheet and then use a FeatureMerger of Joiner to join them to the incoming features from the Directory and File Pathnames Reader. Any that join are the ones you want then...
Does that help?
I've replied to the wrong comment - sorry. Yes I have all the file names and paths I need in a spread sheet. I've used this spread sheet to create a bat file which takes care of the geometry summary workspace. As mentioned above I was hoping to feed the same information to the Directory and File path reader.
Very late answer here but I was just doing something similar so thought I'd share. The Directory and Path reader can be used for this via the FeatureReader transformer. Just set the dataset to the containing folder and then set the path filter to the filename. These values can be stored in attributes of the features entering the FeatureReader and then the attribute values can be used as input for the reader parameters.
