Skip to main content

Heya,

 

I'm having some trouble writing a two csv tables (share a relationship via a common ID in both tables) to a esri feature class and related table (linked via relationship class - global ID / parent globalID) which have a set pre-configured schema.

Here are the steps i have taken and some screen shots of my process to load the csv data to the feature class schema (part 1/2).

Part 1/2 - load geocoded main csv table data into esri feature class

 

 

The csv default for each field is read as a text field, thus the Esri reprojector is trying to pass all these text fields into a set schema. Do I need to include a schema mapper? If so, what does that table look like and in what format does it need to be in. Or, do I need to define all the fields in the CSV reader?

 

The csv is being created correctly no problem and the data are same in the same projection and in the same region as I view them all in the preview with feature caching. The writer to write to Awhina Schema cannot be run - it's greyed out. But I did manage to run it once after some modifications and the log file had a lot of errors - a warning about this coordinate system no existing and a date error:

"2020-05-11 15:27:29| 0.9| 0.0|WARN |Coordinate system named _LL-WGS84_0 does not exist."

The Esri date fields are date but the csv is passing datetime - not sure if that is an issue?

 

example of csv date format above

csv writer above

 

Part 2/2 - read csv related table and load into esri related table

 

 

In the above model, the csv write to a gdb with no set schema and writes all the features but cannot run to the related table that has set schema.

Thanks for any help you may be able to provide, I'm a bit new to FME so may not be using the best approach here.

Cheers,

Simon

The easiest thing to do is add the table/feature class as a reader and then copy the attributes from the reader to the output by right clicking the output and click "Copy Attributes from Feature Type". Saves a lot of time trying to set things up.

 

Also, whatever you set the "Type" to is what it will write as. Text will be converted to numbers automatically.

 

On a side note, whenever you aren't sure of what you are about to be writing. It's best to add a _1 to the feature class name on the writer to create a new feature class. Then you can run the data into a temp space, verify the results, and remove the _1 when you are sure it's what you want.

 


@allasimon FME can't create a relationship class in a Geodatabase. The relationship class must already exist in your geodtabase, or use a template. The tutorial Introduction to Working with Geodatabase Relationship Classes will take you thru the steps. If you're not familiar with FMe's Geodb writer and templates then How to use an Esri Template Geodatabase


The easiest thing to do is add the table/feature class as a reader and then copy the attributes from the reader to the output by right clicking the output and click "Copy Attributes from Feature Type". Saves a lot of time trying to set things up.

 

Also, whatever you set the "Type" to is what it will write as. Text will be converted to numbers automatically.

 

On a side note, whenever you aren't sure of what you are about to be writing. It's best to add a _1 to the feature class name on the writer to create a new feature class. Then you can run the data into a temp space, verify the results, and remove the _1 when you are sure it's what you want.

 

Hi @jlbaker2779,

 

Thanks for your suggestions above. I have already done that and the writer has the imported schema from the feature class I eventually want to write to. Could it have something to do with the date field?

feature class written above but with no schema definition. Then used as a reader to write to defined schema below.

imported schema definition above

***An error occurred while attempting to convert the '' component of a date. A possible reason for this is that the data in the date field may be corrupt

 

Some of the other errors that follow in the log are:

An error occurred while attempting to convert the '' component of a date. A possible reason for this is that the data in the date field may be corrupt

Storing feature(s) to FME feature store file `Q:\\log.ffs'

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Feature Type: `Welfare_MainTable_test01'

Attribute(encoded: windows-1252) : `Call Identification Number' has value `00-00-02'

Attribute(encoded: windows-1252) : `CallIdentificationNumber' has value `00-00-02'

Attribute(encoded: UTF-8) : `GeoCoded_y_n' has value `No'

Attribute(encoded: windows-1252) : `OBJECTID' has value `'

Attribute(64 bit real) : `_latitude' has value `5815849.601'

Attribute(encoded: UTF-8) : `_longitude' has value `1940095.127 '

Attribute(encoded: windows-1252) : `accomnumberofadults' has value `'

Attribute(encoded: windows-1252) : `accomnumberofchildren' has value `'

Attribute(encoded: windows-1252) : `accomrequestreason' has value `BACKFILL'

Attribute(encoded: windows-1252) : `accomrequirements' has value `'

Attribute(encoded: windows-1252) : `additionalnotes' has value `'

Attribute(encoded: windows-1252) : `additionalrequirements' has value `'

Attribute(encoded: windows-1252) : `animalagencynotes' has value `'

Attribute(encoded: windows-1252) : `animaldateofreferral' has value `'

Attribute(encoded: windows-1252) : `animalneednotes' has value `'

 

Thanks again,

Simon


@allasimon FME can't create a relationship class in a Geodatabase. The relationship class must already exist in your geodtabase, or use a template. The tutorial Introduction to Working with Geodatabase Relationship Classes will take you thru the steps. If you're not familiar with FMe's Geodb writer and templates then How to use an Esri Template Geodatabase

Hi @markatsafe,

 

I did see that tutorial so maybe I'll give it a go. I thought that I could write both files indidually to the desired feature class and table schema, create the relationship in arc afterward using the call id fields in both tables, then change this to a globalID/parent global id relationship afterward?

 

But I am having serious issues pushing all the fields defined as text in the output csv or feature class to a feature class with the imported desired feature schema.

 

Thanks,

Simon


Hi @jlbaker2779,

 

Thanks for your suggestions above. I have already done that and the writer has the imported schema from the feature class I eventually want to write to. Could it have something to do with the date field?

feature class written above but with no schema definition. Then used as a reader to write to defined schema below.

imported schema definition above

***An error occurred while attempting to convert the '' component of a date. A possible reason for this is that the data in the date field may be corrupt

 

Some of the other errors that follow in the log are:

An error occurred while attempting to convert the '' component of a date. A possible reason for this is that the data in the date field may be corrupt

Storing feature(s) to FME feature store file `Q:\\log.ffs'

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Feature Type: `Welfare_MainTable_test01'

Attribute(encoded: windows-1252) : `Call Identification Number' has value `00-00-02'

Attribute(encoded: windows-1252) : `CallIdentificationNumber' has value `00-00-02'

Attribute(encoded: UTF-8) : `GeoCoded_y_n' has value `No'

Attribute(encoded: windows-1252) : `OBJECTID' has value `'

Attribute(64 bit real) : `_latitude' has value `5815849.601'

Attribute(encoded: UTF-8) : `_longitude' has value `1940095.127 '

Attribute(encoded: windows-1252) : `accomnumberofadults' has value `'

Attribute(encoded: windows-1252) : `accomnumberofchildren' has value `'

Attribute(encoded: windows-1252) : `accomrequestreason' has value `BACKFILL'

Attribute(encoded: windows-1252) : `accomrequirements' has value `'

Attribute(encoded: windows-1252) : `additionalnotes' has value `'

Attribute(encoded: windows-1252) : `additionalrequirements' has value `'

Attribute(encoded: windows-1252) : `animalagencynotes' has value `'

Attribute(encoded: windows-1252) : `animaldateofreferral' has value `'

Attribute(encoded: windows-1252) : `animalneednotes' has value `'

 

Thanks again,

Simon

I see. Use a dateformatter and set the output format as %Y%m%d. That will convert it to YYYYMMDD. That should have no problem writing. Make sure you catch the rejected if any fail and adjust accordingly.


Hi @markatsafe,

 

I did see that tutorial so maybe I'll give it a go. I thought that I could write both files indidually to the desired feature class and table schema, create the relationship in arc afterward using the call id fields in both tables, then change this to a globalID/parent global id relationship afterward?

 

But I am having serious issues pushing all the fields defined as text in the output csv or feature class to a feature class with the imported desired feature schema.

 

Thanks,

Simon

@allasimon I've never tried that approach but there's no reason it shouldn't work - it would be a lot faster during the data load since relationship classes really slow down the inserts.

But if you're having trouble with the straight load of the tables that's something different altogether. If you can add some sample data and an export of your Geodb schema and your workspace we should be able to sort it out (or to mark @ safe.com if the data is more sensitive)


Reply