Solved

WFS service fails to write to featurewriter

R
Badge

I am attempting to write from a WFS FeaturerReader directly to a FeatureWriter, using the tip outlined in this post:

https://knowledge.safe.com/questions/69654/writing-table-with-featurereader-featurewriter-pro.html

This works for some datasets, but when I try to use a particular dataset, I get an error message:

Bulk copy failed on table 'public.road_works_line_temp' using delimiter ':'. Error was 'ERROR:  extra data after last expected column CONTEXT:  COPY road_works_line_temp, line 1: "NULL:featureMembers:rwe_line.143236:143236:Sporting/social event:NULL: road closed:NULL:Pending:NULL..."'

The wfs url is 

http://api.vicroads.vic.gov.au/vicroads/wfs?SERVICE=WFS&VERSION;=1.1.0&REQUEST;=GetCapabilities&AUTH;=eyJraWQiOiJRRFFQWDZVSDlQRExOOU9GQVowMlNFRFVYIiwic3R0IjoiYWNjZXNzIiwiYWxnIjoiSFMyNTYifQ.eyJqdGkiOiJuUzRMM0MzSURvdzRsakNNc3R2Y0IiLCJpYXQiOjE0OTE1Mzg2NDQsImlzcyI6Imh0dHBzOi8vYXBpLnN0b3JtcGF0aC5jb20vdjEvYXBwbGljYXRpb25zLzRBeTd5RjJtUUNpQlpxUHo5Q3lRVTgiLCJzdWIiOiJodHRwczovL2FwaS5zdG9ybXBhdGguY29tL3YxL2FjY291bnRzL20xckJyQ1BGMXJCREdVUG9oN2gxZiIsImV4cCI6MTU1NDYxMDY0NH0.9NmwBbkw_5pe_0LRaPENBCBo3j6KsPY-V2OR7Qq1rOw

 

The feature type that fails is rwe_line (rwe_point also fails). If I use a different data set from the same service (e.g. erc_line or erc_point), the import process works without any problems.

From the error message, it looks like something is going wrong with the colon delimiter.

Could anyone suggest a fix for this issue.

icon

Best answer by jkr_wrk 8 August 2018, 09:52

View original

13 replies

R
Badge

Actually, the same problem occurs if I don't use the tip I pointed to with a generic output port and just write to the table directly with the output port defined as rwe_line.

takashi
Rank
Userlevel 2
Badge +17

Actually, the same problem occurs if I don't use the tip I pointed to with a generic output port and just write to the table directly with the output port defined as rwe_line.

What is the destination writer format? What happens if you send the features to the Inspector instead of the writer?

 

 

R
Badge
Sorry, I should have mentioned - the destination format is postGIS. I am attempting to import the layer into a postGIS database.

 

I should mention that the same thing happens if I use a Writer. The problem doesn't therefore seem to be due to any quirks in my implementation. Basically, I just can't import the layer
takashi
Rank
Userlevel 2
Badge +17
Sorry, I should have mentioned - the destination format is postGIS. I am attempting to import the layer into a postGIS database.

 

I should mention that the same thing happens if I use a Writer. The problem doesn't therefore seem to be due to any quirks in my implementation. Basically, I just can't import the layer
I was able to reproduce the same error, regardless of whether the writer schema is dynamic or not. I think there could be a defect in the PostGIS writer. Please report the problem to Safe support: Report a Problem

 

 

R
Badge
OK thanks, I have reported the problem.

 

nielsgerrits
Rank
Userlevel 6
Badge +32
@rohan what versions of FME / PostGIS do you use? I wasn't able to reproduce the problem using 2018.1 and PostGIS 2.1. Maybe the datasource changed? Could you check?

 

R
Badge
I am using postGIS 2.3.3. and postgreSQL version 9.6.3. I checked the upload again just now and it still fails for me.

 

nielsgerrits
Rank
Userlevel 6
Badge +32
I am using postGIS 2.3.3. and postgreSQL version 9.6.3. I checked the upload again just now and it still fails for me.

 

What version FME?
R
Badge
The FME version is 2018.0.0.1 (20180328 - Build 18295 - WIN64)

 

 

jkr_wrk
Rank
Userlevel 3
Badge +18

I was able to reproduce the issue with FME 2018.0.0.3 and 2018.1.0.0 versions and Postgres 9.5.13 /PostGIS 2.3.1

I solved the issue setting the Bulk Insert setting to No.

 

Open the FeatureWriter, Click on Parameters... Advanced Bulk Inserts: No.

@nielsgerrits, can you check if you have this parameter set to No?

Sample: Every Nth Feature (N=1)
nielsgerrits
Rank
Userlevel 6
Badge +32

I was able to reproduce the issue with FME 2018.0.0.3 and 2018.1.0.0 versions and Postgres 9.5.13 /PostGIS 2.3.1

I solved the issue setting the Bulk Insert setting to No.

 

Open the FeatureWriter, Click on Parameters... Advanced Bulk Inserts: No.

@nielsgerrits, can you check if you have this parameter set to No?

Bulk Insert: Yes

 

FME Desktop 2018.1 / Postgres 9.4 / PostGIS 2.1

 

Workspace attached.

 

 

jkr_wrk
Rank
Userlevel 3
Badge +18

Bulk Insert: Yes

 

FME Desktop 2018.1 / Postgres 9.4 / PostGIS 2.1

 

Workspace attached.

 

 

I can confirm your Workspace gives an error on my database. Looks like it has to do with the newer Postgres database then?

 

 

FME 2018.1.0.0 / Postgres 9.5.13 / PostGIS 2.3.1

 

 

Bulk copy failed on table 'public.rwe_line' using delimiter ':'. Error was 'ERROR: extra data after last expected column CONTEXT: COPY rwe_line, line 1: "NULL:featureMembers:rwe_line.147022:147022:Road construction:NULL: :NULL:Pending:NULL:2018-06-28T23\\..."

 

 

 

 

Sample: Every Nth Feature (N=1)
R
Badge

Thanks to everyone for their help. I reported the issue to Safe and they confirm that it is a bug. Setting the bulk inserts setting to No does fix the problem in the short-term.

Reply


Community Stats

28,707
Posts
109,478
Replies
36,551
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.