Skip to main content
Question

Read .txt file from online source

  • May 25, 2018
  • 4 replies
  • 302 views

pflegpet
Contributor
Forum|alt.badge.img+8

Hi,

I'm trying to get the links from this URL using the CSV reader but it does not work: https://data.geo.admin.ch/ch.swisstopo-vd.amtliche-vermessung/meta.txt Using an HTTP Caller did not work either. If I paste the link in a browser it works fine. Any idea what I am doing wrong? Thank you for your help!

This post is closed to further activity.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.

4 replies

redgeographics
Celebrity
Forum|alt.badge.img+62

I'm not able to process this either. Viewing in a browser works fine, from there you can save it as a local file and process that, which works too, but that's probably not a feasible long-time workflow.

The HTTPCaller gives this error: HTTPCaller(HTTPFactory): HTTP/FTP transfer error: 'Unrecognized or bad HTTP Content or Transfer-Encoding'

Which makes me wonder if it's something on the server side that's throwing off FME.


david_r
Celebrity
  • May 25, 2018

I can confirm that even FME 2018 doesn't seem to manage reading from this URL, neither the CSV reader, Text File reader or the HTTPCaller. It may be worth signalling to Safe support as a possible bug.

As a workaround, try the following in a PythonCreator:

import fmeobjects
import urllib2 

class FeatureCreator(object):
    def __init__(self):
        pass
       
    def input(self,feature):
        data = urllib2.urlopen('https://data.geo.admin.ch/ch.swisstopo-vd.amtliche-vermessung/meta.txt')
        for line in data:
            url, date = line.split(' ')
            newFeature = fmeobjects.FMEFeature()
            newFeature.setAttribute('url', url)
            newFeature.setAttribute('date', date)
            self.pyoutput(newFeature)

    def close(self):
        pass

Remenber to expose the attributes 'url' and 'date'.


Forum|alt.badge.img
  • May 25, 2018

I can confirm that even FME 2018 doesn't seem to manage reading from this URL, neither the CSV reader, Text File reader or the HTTPCaller. It may be worth signalling to Safe support as a possible bug.

As a workaround, try the following in a PythonCreator:

import fmeobjects
import urllib2 

class FeatureCreator(object):
    def __init__(self):
        pass
       
    def input(self,feature):
        data = urllib2.urlopen('https://data.geo.admin.ch/ch.swisstopo-vd.amtliche-vermessung/meta.txt')
        for line in data:
            url, date = line.split(' ')
            newFeature = fmeobjects.FMEFeature()
            newFeature.setAttribute('url', url)
            newFeature.setAttribute('date', date)
            self.pyoutput(newFeature)

    def close(self):
        pass

Remenber to expose the attributes 'url' and 'date'.

Also you can download the Zip file using python too to a given location, if you wanted to.

 

 

 


pflegpet
Contributor
Forum|alt.badge.img+8
  • Author
  • Contributor
  • May 29, 2018

I can confirm that even FME 2018 doesn't seem to manage reading from this URL, neither the CSV reader, Text File reader or the HTTPCaller. It may be worth signalling to Safe support as a possible bug.

As a workaround, try the following in a PythonCreator:

import fmeobjects
import urllib2 

class FeatureCreator(object):
    def __init__(self):
        pass
       
    def input(self,feature):
        data = urllib2.urlopen('https://data.geo.admin.ch/ch.swisstopo-vd.amtliche-vermessung/meta.txt')
        for line in data:
            url, date = line.split(' ')
            newFeature = fmeobjects.FMEFeature()
            newFeature.setAttribute('url', url)
            newFeature.setAttribute('date', date)
            self.pyoutput(newFeature)

    def close(self):
        pass

Remenber to expose the attributes 'url' and 'date'.

Thank you very much, the python workaround works like a charm.