Skip to main content
Solved

Read Features from .zip or .tar.gz

V
Forum|alt.badge.img

Hi,

My customers upload .zip or .tar.gz files to our server, I have a CSV reader, when I set the url (https://mydomain.com/customer1.zip) of a zip file with only 1 file, FME server works fine.

Now, a customer send us .tar.gz files with several files in it, for this spacific workspace I need to grab only one file form the package, there is a way to specify which will be the file in the url to process? I tried with https://mydomain.com/customer2.tar.gz\\students.txt with no luck, I manually convert to a zip file https://mydomain.com/customer2.zip\\students.txt but nothing works.

Thanks

VictorV

Best answer by pallegama

Hi @velasquezvictor,

I agree with @stalknecht of using HTTPCaller in this case. Since you are looking for using same parameter for local and remote files, you can use a workflow as depicted in the attached workspace template.

zipfile-reader.fmwt

In this workflow, source file path (local or remote) should be passed through 'Source' Published Parameter and the first Tester identifies and divert remote paths to download the zip/gz file before passing it to FeatureReader. This workspace works on both FME Desktop and FME Server.

Alternatively, you can think of writing a Startup Python script to do this.

Regards, Priyantha Pallegama

View original
Did this help you find an answer to your question?

5 replies

stalknecht
Contributor
Forum|alt.badge.img+19
  • stalknechtContributor
  • Contributor
  • July 23, 2017

Hi Victor you should use a slash instead of a ". "

https://mydomain.com/customer2.tar/gzstudents.txt

V
Forum|alt.badge.img
stalknecht wrote:

Hi Victor you should use a slash instead of a ". "

https://mydomain.com/customer2.tar/gzstudents.txt

Hi I use https://mydomain.com/customer2.tar.gz/students.txt for gzip and https://mydomain.com/customer2.zip/students.txt for the zip file a none of them works. I receive the error: 

 

 

CSV reader: Failed to open file 'https://mydomain.com/customer2.tar.gz/students.txt' for reading. Please ensure that the file exists and you have sufficient privileges to read it

 

I verified and the zip/tar.gz file has students.txt inside.

 

 

stalknecht
Contributor
Forum|alt.badge.img+19
  • stalknechtContributor
  • Contributor
  • July 27, 2017
velasquezvictor wrote:
Hi I use https://mydomain.com/customer2.tar.gz/students.txt for gzip and https://mydomain.com/customer2.zip/students.txt for the zip file a none of them works. I receive the error: 

 

 

CSV reader: Failed to open file 'https://mydomain.com/customer2.tar.gz/students.txt' for reading. Please ensure that the file exists and you have sufficient privileges to read it

 

I verified and the zip/tar.gz file has students.txt inside.

 

 

Do you run it from FME Server?

 

 

You can use the HTTPCaller to download the file locally and then use a FeatureReader to read it.

 

 

V
Forum|alt.badge.img
stalknecht wrote:
Do you run it from FME Server?

 

 

You can use the HTTPCaller to download the file locally and then use a FeatureReader to read it.

 

 

 

Hi,

 

 

I Would Like to use the same parameter for the reader to load locally, temporary upload or a remote file.
pallegama
Contributor
Forum|alt.badge.img+8
  • pallegamaContributor
  • Contributor
  • Best Answer
  • July 27, 2017

Hi @velasquezvictor,

I agree with @stalknecht of using HTTPCaller in this case. Since you are looking for using same parameter for local and remote files, you can use a workflow as depicted in the attached workspace template.

zipfile-reader.fmwt

In this workflow, source file path (local or remote) should be passed through 'Source' Published Parameter and the first Tester identifies and divert remote paths to download the zip/gz file before passing it to FeatureReader. This workspace works on both FME Desktop and FME Server.

Alternatively, you can think of writing a Startup Python script to do this.

Regards, Priyantha Pallegama

Reply


Helpful Members This Week

Community Stats

31,176
Posts
118,181
Replies
38,791
Members

Latest FME

Terms & ConditionsCookie settings

Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.