Skip to main content
  • EN

Good morning

I have a problem I currently am unable to resolve, which is a csv file containing address data and additional spatial and non-spatial information related to the individual addresses.

The csv file is ordered according to a Record Identifier.

However each Record Identifier has a different number of columns populated with data. The minimum number of columns is 19, the maximum 29

If i attach a Tester to set the 1st column to a particular Record Identifier eg 23 then attach an Inspector, the result always shows 19 columns, which is the number of columns the first record in the file has (which is Record Identifier 11).

If i export to an .xlsx file using a template of the source Record Identifier data i want to write to, only the first 19 columns are populated, the other 10 columns are blank even if that particular Record Identifier uses the additional 10 columns.

Can anyone suggest how i can force the tester to recognise the number of columns the Record Identifier has, or an alternative transformer that may provide the results i need?

thanks

 

SImon Hume

Ignoring the tester for the moment, when you add the CSV reader do you see all the columns in the Preview? What is Maximum Rows to Scan set to? I'm presuming this is LPG/LSG type data, you'll see that the number of columns read is dependent on the maximum rows to scan attribute

CaptureCapture

@ebygomm​ 

thanks for your prompt answer. I set the Maximum Rows to Scan to blank and when i run the writer it shows all the columns and data

You're correct, it is LLPG data: I am trying to set up a workspace in FME to split a COU into 3 distinct COU's based on three current Authority areas that will disappear in April 2021. Potentially, the split COU's may be needed to supply local LLPG's and systems that rely on the LLPG data. The FME Workspace is only a backup as hopefully it won't be required

thanks again

 

 

@ebygomm​ 

thanks for your prompt answer. I set the Maximum Rows to Scan to blank and when i run the writer it shows all the columns and data

You're correct, it is LLPG data: I am trying to set up a workspace in FME to split a COU into 3 distinct COU's based on three current Authority areas that will disappear in April 2021. Potentially, the split COU's may be needed to supply local LLPG's and systems that rely on the LLPG data. The FME Workspace is only a backup as hopefully it won't be required

thanks again

 

 

With LLPG data I always found it was better to work with it as a text file. My experience is probably 6+ years out of date now though. But i know this was the approach I took when i had to split a very large change only file into smaller chunks.

Reply


Community Stats

30,160
Posts
114,491
Replies
37,811
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.