Skip to main content
Question

Read .txt file from online source


pflegpet
Contributor
Forum|alt.badge.img+8

Hi,

I'm trying to get the links from this URL using the CSV reader but it does not work: https://data.geo.admin.ch/ch.swisstopo-vd.amtliche-vermessung/meta.txt Using an HTTP Caller did not work either. If I paste the link in a browser it works fine. Any idea what I am doing wrong? Thank you for your help!

4 replies

redgeographics
Celebrity
Forum|alt.badge.img+49

I'm not able to process this either. Viewing in a browser works fine, from there you can save it as a local file and process that, which works too, but that's probably not a feasible long-time workflow.

The HTTPCaller gives this error: HTTPCaller(HTTPFactory): HTTP/FTP transfer error: 'Unrecognized or bad HTTP Content or Transfer-Encoding'

Which makes me wonder if it's something on the server side that's throwing off FME.


david_r
Celebrity
  • May 25, 2018

I can confirm that even FME 2018 doesn't seem to manage reading from this URL, neither the CSV reader, Text File reader or the HTTPCaller. It may be worth signalling to Safe support as a possible bug.

As a workaround, try the following in a PythonCreator:

import fmeobjects
import urllib2 

class FeatureCreator(object):
    def __init__(self):
        pass
       
    def input(self,feature):
        data = urllib2.urlopen('https://data.geo.admin.ch/ch.swisstopo-vd.amtliche-vermessung/meta.txt')
        for line in data:
            url, date = line.split(' ')
            newFeature = fmeobjects.FMEFeature()
            newFeature.setAttribute('url', url)
            newFeature.setAttribute('date', date)
            self.pyoutput(newFeature)

    def close(self):
        pass

Remenber to expose the attributes 'url' and 'date'.


Forum|alt.badge.img
david_r wrote:

I can confirm that even FME 2018 doesn't seem to manage reading from this URL, neither the CSV reader, Text File reader or the HTTPCaller. It may be worth signalling to Safe support as a possible bug.

As a workaround, try the following in a PythonCreator:

import fmeobjects
import urllib2 

class FeatureCreator(object):
    def __init__(self):
        pass
       
    def input(self,feature):
        data = urllib2.urlopen('https://data.geo.admin.ch/ch.swisstopo-vd.amtliche-vermessung/meta.txt')
        for line in data:
            url, date = line.split(' ')
            newFeature = fmeobjects.FMEFeature()
            newFeature.setAttribute('url', url)
            newFeature.setAttribute('date', date)
            self.pyoutput(newFeature)

    def close(self):
        pass

Remenber to expose the attributes 'url' and 'date'.

Also you can download the Zip file using python too to a given location, if you wanted to.

 

 

 


pflegpet
Contributor
Forum|alt.badge.img+8
  • Author
  • Contributor
  • May 29, 2018
david_r wrote:

I can confirm that even FME 2018 doesn't seem to manage reading from this URL, neither the CSV reader, Text File reader or the HTTPCaller. It may be worth signalling to Safe support as a possible bug.

As a workaround, try the following in a PythonCreator:

import fmeobjects
import urllib2 

class FeatureCreator(object):
    def __init__(self):
        pass
       
    def input(self,feature):
        data = urllib2.urlopen('https://data.geo.admin.ch/ch.swisstopo-vd.amtliche-vermessung/meta.txt')
        for line in data:
            url, date = line.split(' ')
            newFeature = fmeobjects.FMEFeature()
            newFeature.setAttribute('url', url)
            newFeature.setAttribute('date', date)
            self.pyoutput(newFeature)

    def close(self):
        pass

Remenber to expose the attributes 'url' and 'date'.

Thank you very much, the python workaround works like a charm. 

 

 


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings