Skip to main content

I am attempting to write from a WFS FeaturerReader directly to a FeatureWriter, using the tip outlined in this post:

https://knowledge.safe.com/questions/69654/writing-table-with-featurereader-featurewriter-pro.html

This works for some datasets, but when I try to use a particular dataset, I get an error message:

Bulk copy failed on table 'public.road_works_line_temp' using delimiter ':'. Error was 'ERROR:  extra data after last expected column CONTEXT:  COPY road_works_line_temp, line 1: "NULL:featureMembers:rwe_line.143236:143236:Sporting/social event:NULL: road closed:NULL:Pending:NULL..."'

The wfs url is 

http://api.vicroads.vic.gov.au/vicroads/wfs?SERVICE=WFS&VERSION;=1.1.0&REQUEST;=GetCapabilities&AUTH;=eyJraWQiOiJRRFFQWDZVSDlQRExOOU9GQVowMlNFRFVYIiwic3R0IjoiYWNjZXNzIiwiYWxnIjoiSFMyNTYifQ.eyJqdGkiOiJuUzRMM0MzSURvdzRsakNNc3R2Y0IiLCJpYXQiOjE0OTE1Mzg2NDQsImlzcyI6Imh0dHBzOi8vYXBpLnN0b3JtcGF0aC5jb20vdjEvYXBwbGljYXRpb25zLzRBeTd5RjJtUUNpQlpxUHo5Q3lRVTgiLCJzdWIiOiJodHRwczovL2FwaS5zdG9ybXBhdGguY29tL3YxL2FjY291bnRzL20xckJyQ1BGMXJCREdVUG9oN2gxZiIsImV4cCI6MTU1NDYxMDY0NH0.9NmwBbkw_5pe_0LRaPENBCBo3j6KsPY-V2OR7Qq1rOw

 

The feature type that fails is rwe_line (rwe_point also fails). If I use a different data set from the same service (e.g. erc_line or erc_point), the import process works without any problems.

From the error message, it looks like something is going wrong with the colon delimiter.

Could anyone suggest a fix for this issue.

Actually, the same problem occurs if I don't use the tip I pointed to with a generic output port and just write to the table directly with the output port defined as rwe_line.


Actually, the same problem occurs if I don't use the tip I pointed to with a generic output port and just write to the table directly with the output port defined as rwe_line.

What is the destination writer format? What happens if you send the features to the Inspector instead of the writer?

 

 


Sorry, I should have mentioned - the destination format is postGIS. I am attempting to import the layer into a postGIS database.

 

I should mention that the same thing happens if I use a Writer. The problem doesn't therefore seem to be due to any quirks in my implementation. Basically, I just can't import the layer
Sorry, I should have mentioned - the destination format is postGIS. I am attempting to import the layer into a postGIS database.

 

I should mention that the same thing happens if I use a Writer. The problem doesn't therefore seem to be due to any quirks in my implementation. Basically, I just can't import the layer
I was able to reproduce the same error, regardless of whether the writer schema is dynamic or not. I think there could be a defect in the PostGIS writer. Please report the problem to Safe support: Report a Problem

 

 


OK thanks, I have reported the problem.

 


@rohan what versions of FME / PostGIS do you use? I wasn't able to reproduce the problem using 2018.1 and PostGIS 2.1. Maybe the datasource changed? Could you check?

 


I am using postGIS 2.3.3. and postgreSQL version 9.6.3. I checked the upload again just now and it still fails for me.

 


I am using postGIS 2.3.3. and postgreSQL version 9.6.3. I checked the upload again just now and it still fails for me.

 

What version FME?
The FME version is 2018.0.0.1 (20180328 - Build 18295 - WIN64)

 

 


I was able to reproduce the issue with FME 2018.0.0.3 and 2018.1.0.0 versions and Postgres 9.5.13 /PostGIS 2.3.1

I solved the issue setting the Bulk Insert setting to No.

 

Open the FeatureWriter, Click on Parameters... Advanced Bulk Inserts: No.

@nielsgerrits, can you check if you have this parameter set to No?


I was able to reproduce the issue with FME 2018.0.0.3 and 2018.1.0.0 versions and Postgres 9.5.13 /PostGIS 2.3.1

I solved the issue setting the Bulk Insert setting to No.

 

Open the FeatureWriter, Click on Parameters... Advanced Bulk Inserts: No.

@nielsgerrits, can you check if you have this parameter set to No?

Bulk Insert: Yes

 

FME Desktop 2018.1 / Postgres 9.4 / PostGIS 2.1

 

Workspace attached.

 

 


Bulk Insert: Yes

 

FME Desktop 2018.1 / Postgres 9.4 / PostGIS 2.1

 

Workspace attached.

 

 

I can confirm your Workspace gives an error on my database. Looks like it has to do with the newer Postgres database then?

 

 

FME 2018.1.0.0 / Postgres 9.5.13 / PostGIS 2.3.1

 

 

Bulk copy failed on table 'public.rwe_line' using delimiter ':'. Error was 'ERROR: extra data after last expected column
CONTEXT: COPY rwe_line, line 1: "NULL:featureMembers:rwe_line.147022:147022:Road construction:NULL: :NULL:Pending:NULL:2018-06-28T23\\..."

 

 

 

 


Thanks to everyone for their help. I reported the issue to Safe and they confirm that it is a bug. Setting the bulk inserts setting to No does fix the problem in the short-term.


Reply