Skip to main content
Archived

Stop fetching huge web based data when setting up workspace

Related products:Transformers
  • October 4, 2016
  • 1 reply
  • 26 views
lifalin2016
Influencer
Forum|alt.badge.img+40

I'm trying to create a workspace that fetches (among other data) all addresses as a CSV from a web portal. I need to do this with the FeatureReader and a generated URL for "Dataset".

It seems that FME is trying to download the whole dataset (with 3+ million records/lines) when I'm setting up the workspace.

It would be beneficial to have FME _not_ read a huge web based dataset when setting up the workspace, i.e. in design mode. In any case, it has no practical application, since I do have to manually type in the CSV field names even though they're given as line #1 in the dataset (and I did check the "Field names follow Header" in parameters).

E.g. by abstaining to read any dataset that starts with "http://", or at least just read a suitable small fraction.

Cheers

-- Cheers, Lars I.
This post is closed to further activity.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.

1 reply

M
Forum|alt.badge.img+2
  • markatsafe
  • October 11, 2016
This is a good idea. We could add an option to cache any data that came from a web source such as ftp or http.

 

 

Community Stats

32,752
Posts
124,208
Replies
41,264
Members
Terms & ConditionsAccessibility statement
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.