Solved

Open multiple datasets in multiple folders on FTP server directly ?

lifalin2016
Rank
Userlevel 1
Badge +22

Hi,

As far as I can see, it's not possible to open a FTP based dataset directly in a reader (MapInfo TAB) ?

And FTPCaller only allows a single file to be downloaded at any time ?

I have multiple folders with multiple datasets (consisting of multiple files) to import, and doesn't want to do this one file at a time.

The optimal solution would be to open the datasets with a wildcard directly in a reader. Just like how you can within a ZIP.

How to proceed ?

Cheers.

icon

Best answer by danilo_fme 6 May 2018, 18:32

View original
-- Cheers, Lars I.

3 replies

danilo_fme
Rank
Userlevel 4
Badge +30

Hi @lifalin2016,

I believe this post can to help you Use PythonCller to get list of files in Directory

Thanks,

Danilo

lifalin2016
Rank
Userlevel 1
Badge +22

Hi @lifalin2016,

I believe this post can to help you Use PythonCller to get list of files in Directory

Thanks,

Danilo

Thanks danilo_fme.

 

 

I originally fetched the data manually, and read them from my HDD instead.

 

 

However, I just revisited all my unanswered question, and decided to test the FTP Python code.

 

 

It took awhile to get it to work, though, but it does now.

 

 

Unfortunately one still needs to download multiple files to get a single dataset for FME to read (ESRI Shape, MapInfo TAB etc.), but at least it's now automateable.

 

 

Cheers.

 

-- Cheers, Lars I.
lifalin2016
Rank
Userlevel 1
Badge +22

Hi @lifalin2016,

I believe this post can to help you Use PythonCller to get list of files in Directory

Thanks,

Danilo

Here's the Python code in case someone else wants to tinker:

 

import fme
import fmeobjects
import ftplib

class FeatureProcessor(object):
    def __init__(self):
        pass

    def input(self,feature):
        pass
        CSVlist = []
        _ftp = ftplib.FTP("ftp.safe.com")
        _ftp.login() #anonymously
        _ftp.cwd("fmebuilds")
        try:
            CSVlist = _ftp.nlst()
        except ftplib.error_perm, resp:
            if str(resp) == "550 No files found":
                print("No files found")
            else:
                pass
                #raise
        _ftp.quit()
        pass
        for i,v in enumerate(CSVlist):
            _var = "_list_CSVs{{{}}}".format(i)
            feature.setAttribute(_var, v)
        self.pyoutput(feature)
        
    def close(self):
        pass 

 

-- Cheers, Lars I.

Reply


Community Stats

28,723
Posts
109,514
Replies
36,561
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.