Skip to main content
Solved

WFS service fails to write to featurewriter

  • August 7, 2018
  • 13 replies
  • 105 views
R
Forum|alt.badge.img

I am attempting to write from a WFS FeaturerReader directly to a FeatureWriter, using the tip outlined in this post:

https://knowledge.safe.com/questions/69654/writing-table-with-featurereader-featurewriter-pro.html

This works for some datasets, but when I try to use a particular dataset, I get an error message:

Bulk copy failed on table 'public.road_works_line_temp' using delimiter ':'. Error was 'ERROR:  extra data after last expected column CONTEXT:  COPY road_works_line_temp, line 1: "NULL:featureMembers:rwe_line.143236:143236:Sporting/social event:NULL: road closed:NULL:Pending:NULL..."'

The wfs url is 

http://api.vicroads.vic.gov.au/vicroads/wfs?SERVICE=WFS&VERSION;=1.1.0&REQUEST;=GetCapabilities&AUTH;=eyJraWQiOiJRRFFQWDZVSDlQRExOOU9GQVowMlNFRFVYIiwic3R0IjoiYWNjZXNzIiwiYWxnIjoiSFMyNTYifQ.eyJqdGkiOiJuUzRMM0MzSURvdzRsakNNc3R2Y0IiLCJpYXQiOjE0OTE1Mzg2NDQsImlzcyI6Imh0dHBzOi8vYXBpLnN0b3JtcGF0aC5jb20vdjEvYXBwbGljYXRpb25zLzRBeTd5RjJtUUNpQlpxUHo5Q3lRVTgiLCJzdWIiOiJodHRwczovL2FwaS5zdG9ybXBhdGguY29tL3YxL2FjY291bnRzL20xckJyQ1BGMXJCREdVUG9oN2gxZiIsImV4cCI6MTU1NDYxMDY0NH0.9NmwBbkw_5pe_0LRaPENBCBo3j6KsPY-V2OR7Qq1rOw

 

The feature type that fails is rwe_line (rwe_point also fails). If I use a different data set from the same service (e.g. erc_line or erc_point), the import process works without any problems.

From the error message, it looks like something is going wrong with the colon delimiter.

Could anyone suggest a fix for this issue.

Best answer by jkr_wrk

I was able to reproduce the issue with FME 2018.0.0.3 and 2018.1.0.0 versions and Postgres 9.5.13 /PostGIS 2.3.1

I solved the issue setting the Bulk Insert setting to No.

 

Open the FeatureWriter, Click on Parameters... Advanced Bulk Inserts: No.

@nielsgerrits, can you check if you have this parameter set to No?

This post is closed to further activity.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.

13 replies

R
Forum|alt.badge.img
  • rohan
  • Author
  • August 7, 2018

Actually, the same problem occurs if I don't use the tip I pointed to with a generic output port and just write to the table directly with the output port defined as rwe_line.

takashi
Celebrity
  • takashiCelebrity
  • August 7, 2018

Actually, the same problem occurs if I don't use the tip I pointed to with a generic output port and just write to the table directly with the output port defined as rwe_line.

What is the destination writer format? What happens if you send the features to the Inspector instead of the writer?

 

 

Why not inspect features with Visual/Data Preview and Feature/Record Information before writing them into a destination dataset?
R
Forum|alt.badge.img
  • rohan
  • Author
  • August 7, 2018
Sorry, I should have mentioned - the destination format is postGIS. I am attempting to import the layer into a postGIS database.

 

I should mention that the same thing happens if I use a Writer. The problem doesn't therefore seem to be due to any quirks in my implementation. Basically, I just can't import the layer
takashi
Celebrity
  • takashiCelebrity
  • August 7, 2018
Sorry, I should have mentioned - the destination format is postGIS. I am attempting to import the layer into a postGIS database.

 

I should mention that the same thing happens if I use a Writer. The problem doesn't therefore seem to be due to any quirks in my implementation. Basically, I just can't import the layer
I was able to reproduce the same error, regardless of whether the writer schema is dynamic or not. I think there could be a defect in the PostGIS writer. Please report the problem to Safe support: Report a Problem

 

 

Why not inspect features with Visual/Data Preview and Feature/Record Information before writing them into a destination dataset?
R
Forum|alt.badge.img
  • rohan
  • Author
  • August 7, 2018
OK thanks, I have reported the problem.

 

nielsgerrits
VIP
Forum|alt.badge.img+64
@rohan what versions of FME / PostGIS do you use? I wasn't able to reproduce the problem using 2018.1 and PostGIS 2.1. Maybe the datasource changed? Could you check?

 

Buckle your seatbelt Dorothy, cause Kansas is going bye-bye...
R
Forum|alt.badge.img
  • rohan
  • Author
  • August 8, 2018
I am using postGIS 2.3.3. and postgreSQL version 9.6.3. I checked the upload again just now and it still fails for me.

 

nielsgerrits
VIP
Forum|alt.badge.img+64
I am using postGIS 2.3.3. and postgreSQL version 9.6.3. I checked the upload again just now and it still fails for me.

 

What version FME?
Buckle your seatbelt Dorothy, cause Kansas is going bye-bye...
R
Forum|alt.badge.img
  • rohan
  • Author
  • August 8, 2018
The FME version is 2018.0.0.1 (20180328 - Build 18295 - WIN64)

 

 

jkr_wrk
Influencer
Forum|alt.badge.img+36
  • jkr_wrkInfluencer
  • Best Answer
  • August 8, 2018

I was able to reproduce the issue with FME 2018.0.0.3 and 2018.1.0.0 versions and Postgres 9.5.13 /PostGIS 2.3.1

I solved the issue setting the Bulk Insert setting to No.

 

Open the FeatureWriter, Click on Parameters... Advanced Bulk Inserts: No.

@nielsgerrits, can you check if you have this parameter set to No?

💡 Did you know... The FeatureWriter is more intuitive. 😏
nielsgerrits
VIP
Forum|alt.badge.img+64

I was able to reproduce the issue with FME 2018.0.0.3 and 2018.1.0.0 versions and Postgres 9.5.13 /PostGIS 2.3.1

I solved the issue setting the Bulk Insert setting to No.

 

Open the FeatureWriter, Click on Parameters... Advanced Bulk Inserts: No.

@nielsgerrits, can you check if you have this parameter set to No?

Bulk Insert: Yes

 

FME Desktop 2018.1 / Postgres 9.4 / PostGIS 2.1

 

Workspace attached.

 

 

Buckle your seatbelt Dorothy, cause Kansas is going bye-bye...
jkr_wrk
Influencer
Forum|alt.badge.img+36
  • jkr_wrkInfluencer
  • August 8, 2018

Bulk Insert: Yes

 

FME Desktop 2018.1 / Postgres 9.4 / PostGIS 2.1

 

Workspace attached.

 

 

I can confirm your Workspace gives an error on my database. Looks like it has to do with the newer Postgres database then?

 

 

FME 2018.1.0.0 / Postgres 9.5.13 / PostGIS 2.3.1

 

 

Bulk copy failed on table 'public.rwe_line' using delimiter ':'. Error was 'ERROR: extra data after last expected column CONTEXT: COPY rwe_line, line 1: "NULL:featureMembers:rwe_line.147022:147022:Road construction:NULL: :NULL:Pending:NULL:2018-06-28T23\\..."

 

 

 

 

💡 Did you know... The FeatureWriter is more intuitive. 😏
R
Forum|alt.badge.img
  • rohan
  • Author
  • August 21, 2018

Thanks to everyone for their help. I reported the issue to Safe and they confirm that it is a bug. Setting the bulk inserts setting to No does fix the problem in the short-term.

Community Stats

32,207
Posts
122,305
Replies
39,891
Members

Latest FME

Terms & ConditionsCookie settingsAccessibility statement
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.