Solved

Is this possible to extract the content of a zip file and explore / sort it's content using FME ?

A

Hello,

 

To automate some of my tasks using FME, I'm currently trying to make FME download some .7z files and extract their content (composed of subfolders, txt files, pdf and shp).

 

I managed to make FME download the archive and store it into a temporary folder just as I wanted but know I want to make FME extract the content of the archive so I can then sort it's content (I want to go inside the subfolders and only extract shp files, then work on them).

 

I've already tried to use some transformers like "ZipExtractor" or "Unzipper" but they don't seem to make what I'd like...

 

If someone know a way to extract and sort zipfile's content using FME please help me !

 

Regards,

Andy METAYER

icon

Best answer by nielsgerrits 9 March 2022, 21:00

View original

10 replies

nielsgerrits
Rank
Userlevel 6
Badge +32

I use the ZipExtractor to do this. Can you explain why this does not work for you?

 

As an alternative a Reader / FeatureReader can read directly from zip.

See Reading from Archive Files.

A

Thank you for your answer !

 

I'll try to explain my problem more clearly !

 

I have an archive wich I want to download from a ftp (so I use "FTPCaller")

 

The content of the archive wich I require is located on the 4th sub folder of the archive so I want to explore this archive using FME to only get the SHP I need.

 

Here are some screenshots of my script and transformers...

 

scriptftpcallerextractor 

When I launch the script with those transformers activated I get those errors:

 

error1 

When I run the script with the FTPCaller activating (using a previously downloaded of the .7z) I get no error but 0 feature output...

 

error2 

I don't understand where my mistake is and I've tried to resolve this problem for the entire day so I hope you have a solution !

Regards,

Andy METAYER

 

nielsgerrits
Rank
Userlevel 6
Badge +32

Thank you for your answer !

 

I'll try to explain my problem more clearly !

 

I have an archive wich I want to download from a ftp (so I use "FTPCaller")

 

The content of the archive wich I require is located on the 4th sub folder of the archive so I want to explore this archive using FME to only get the SHP I need.

 

Here are some screenshots of my script and transformers...

 

scriptftpcallerextractor 

When I launch the script with those transformers activated I get those errors:

 

error1 

When I run the script with the FTPCaller activating (using a previously downloaded of the .7z) I get no error but 0 feature output...

 

error2 

I don't understand where my mistake is and I've tried to resolve this problem for the entire day so I hope you have a solution !

Regards,

Andy METAYER

 

This probably has to do with FeatureCaching? Do you see the downloaded file in the explorer?

A

This probably has to do with FeatureCaching? Do you see the downloaded file in the explorer?

I do ! And as you can see in the 4th screen, feature catching is activated!

A

This probably has to do with FeatureCaching? Do you see the downloaded file in the explorer?

I tried to run the script after disabling FeatureCatching but there is no change... Would this be possible for you to try to make the same script as I did (creator > ftpcaller > zipextractor) with this ftp link "ftp://RPG_ext:quoojaicaiqu6ahD@ftp3.ign.fr/RPG_2-0__SHP_LAMB93_R11_2020-01-01.7z" (Origin: https://geoservices.ign.fr/rpg) so you can tell me if problems comes from me or something else ?

 

Thank you in advance

nielsgerrits
Rank
Userlevel 6
Badge +32

This probably has to do with FeatureCaching? Do you see the downloaded file in the explorer?

I do not understand why but I am able to reproduce your issue with this archive.

ERROR |Specified zip file or destination root folder is invalid.

But when using a FeatureReader Esri Shapefile, dataset 

@Value(_downloaded_file_path)[**\*.shp]

it does read the data. Sample attached.

A

This probably has to do with FeatureCaching? Do you see the downloaded file in the explorer?

I still have one more question ! How did you set the Output ports per feature type ? I tried to reproduce your script to understand it but can't manage to set the output ports.

 

When I try to access "Feature Types to Read" it opens a dialog box saying that I need to select the dataset

 

dataset 

And when I just copy the features types from your script (ILOTS_ANONYMES PARCELLES_GRAPHIQUES) and try to validate another dialog box is opened

 

output 

I don't understand, If I launch your script everything works perfectly but when I try to reproduce it it doesn't work at all wich is a big problem as long as I'll have to do this kind of scripts again in the future...

 

If I try to avoid the FTPCaller by using a downloaded .7z as dataset it works perfectly so maybe the problems come from the caller ?

 

If you have an idea please let me know,

Regards

 

A

This probably has to do with FeatureCaching? Do you see the downloaded file in the explorer?

I think that it doesn't work with the attribute paths, I don't get why it works on yours but it's the only clue I have..

nielsgerrits
Rank
Userlevel 6
Badge +32

This probably has to do with FeatureCaching? Do you see the downloaded file in the explorer?

For the Generating Output Ports part, I manually unzipped the shapes and selected those as source dataset.

 

But you have to do that every time you edit the FeatureReader. Personally, I prefer to use the Generic Output Port, split the features based on fme_feature_type and expose attributes using an AttributeExposer in the workspace later. But I mostly work on server workspaces trying to handle any dataset a user throws at it.

A

This probably has to do with FeatureCaching? Do you see the downloaded file in the explorer?

Oh ok I get it ! Thank your for your answer, I'll try t use this method !

Reply


Community Stats

28,709
Posts
109,484
Replies
36,557
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.