Skip to main content

Hi there,

 

I'm still a bit new to FME so this might be dumb question. Sorry for that.

What I'm trying to do is read a CSV file which contains X and Y fields and then after a few transformations write to an Esri Geodatabase (Postgresql with SDE).

First I've set my reader as described in 1st Method here.

Now at the end of my workbench I wonder what to do so that the ArcSde Geodb writer writes the Geometry point to an attribute named Shape (of type st_geometry) which is not in the csv ? Do I need to add this field in an AttributeManager ? If so what should the value be ?

 

Thanks in advance.

 

Hello @letoff, if your workspace is creating the points as suggested the linked document, you just need to send the data to the Esri Geodatabase Writer. The SDE option is greyed out in the screenshot below as we don't have the ESRI version of FME.

 

 

Or use a FeatureWriter but you will still need the appropriate version of FME.


You might need to use a VertexCreator to tell FME which fields in the CSV contain the actual X/Y coordinates.


You do not explicitly need to create the SHAPE field, FME will do it for you and it also automatically writes the geometry there.

Simply create the Geodatabase Writer and specify and specify the output feature type to be a point.


You might need to use a VertexCreator to tell FME which fields in the CSV contain the actual X/Y coordinates.

Thanks for answering but since my CSV Reader has the coordinates columns identified I thought the VertexCreator was not mandatory as mentionned in the link I provided yesterday. (It's in methode 2 btw)

 


Hello @letoff, if your workspace is creating the points as suggested the linked document, you just need to send the data to the Esri Geodatabase Writer. The SDE option is greyed out in the screenshot below as we don't have the ESRI version of FME.

 

 

Or use a FeatureWriter but you will still need the appropriate version of FME.

Thanks for answering Dean,

 

Well I forgot to mention but of course I have the Esri version of FME Desktop (2019.1.1) and the Feature Writer already sends all the datas to the Sde Geodatabase except the shape column which is not set, reason why I'm asking here. All values of the shape are null in the database whereas it's supposed to be something like '240000000100000001000400E59304000C0000000100000089EC82D7F114A591D3DFB311' which is a geometry point encoded in Sde format.

 


It sounds like there's an issue on the reader side of things. First, look in the log to see if there are any warnings about not being able to create geometries out of your x/y columns. This could e.g. be an issue with floating point separators (point vs comma) and other weird stuff.

Secondly, try redirecting all your output to the FME Inspector and verify that the points are actually showing. If you have no geometries in the Inspector, then you won't have any geometries in the output table either. If that is the case, you need to track back in your workspace to find the issue. Using the VertexCreator as suggested by @danullen could be a good option to improve control of your input data.


You do not explicitly need to create the SHAPE field, FME will do it for you and it also automatically writes the geometry there.

Simply create the Geodatabase Writer and specify and specify the output feature type to be a point.

Thank you for your answer David,

 

This is exactly what I was expecting from FME. Unfortunately as I mentionned earlier, the shape column remains desperatly empty (null) in the Geodatabase. I don't know what I'm doing wrong and I suppose that something is missing in the way I've set my writer which looks like this:

 

I've even tried to set the attribute definition to Automatic or Manual (I've not tried the Dynamic) without any success:

 

 

 

 

Any help would be very appreciated.

 

Thanks in advance.


It sounds like there's an issue on the reader side of things. First, look in the log to see if there are any warnings about not being able to create geometries out of your x/y columns. This could e.g. be an issue with floating point separators (point vs comma) and other weird stuff.

Secondly, try redirecting all your output to the FME Inspector and verify that the points are actually showing. If you have no geometries in the Inspector, then you won't have any geometries in the output table either. If that is the case, you need to track back in your workspace to find the issue. Using the VertexCreator as suggested by @danullen could be a good option to improve control of your input data.

Thanks a lot David, you're completely right ! I don't know what happened because it used to functioned but since I've spend a lot of time on this workbench (this is my first one) something might have gone wrong without me noticing.

Unfortunately there are no warning in the log and of course no geometry created but I clearly remembered I saw the points on the Stamen map in the Graphics view, so I might have made an error somewhere in the meantime.

 

Btw since I have several different coordinate systems in my CSV file, what would you suggest I put in the reader ? Is XY-KM ok for that ?

It's precisely the point of this workbench to reproject them all into web-mercator.

 

And again thanks a lot for your help. I really appreciate.


Thanks a lot David, you're completely right ! I don't know what happened because it used to functioned but since I've spend a lot of time on this workbench (this is my first one) something might have gone wrong without me noticing.

Unfortunately there are no warning in the log and of course no geometry created but I clearly remembered I saw the points on the Stamen map in the Graphics view, so I might have made an error somewhere in the meantime.

 

Btw since I have several different coordinate systems in my CSV file, what would you suggest I put in the reader ? Is XY-KM ok for that ?

It's precisely the point of this workbench to reproject them all into web-mercator.

 

And again thanks a lot for your help. I really appreciate.

Concerning the coordinate systems, I guess there are several opinions on how to handle them in your scenario. Personally, I never define the coordinate system in either the reader or the writer, as I find the implicit reprojection done by FME in this case to be a bit opaque and not always handled exactly as I need.

So I take extra care to very specifically reproject each input data source as necessary using transformers such as the CsMapReprojector or similar, and also making sure that all the output features have the correct coordinate system set before they hit the writer, using the CoordinateSystemSetter. That way the writer will automatically adapt itself to whatever is defined in the workspace, and not the other way around. But again, personal preference and all that.


It sounds like there's an issue on the reader side of things. First, look in the log to see if there are any warnings about not being able to create geometries out of your x/y columns. This could e.g. be an issue with floating point separators (point vs comma) and other weird stuff.

Secondly, try redirecting all your output to the FME Inspector and verify that the points are actually showing. If you have no geometries in the Inspector, then you won't have any geometries in the output table either. If that is the case, you need to track back in your workspace to find the issue. Using the VertexCreator as suggested by @danullen could be a good option to improve control of your input data.

End of story: I simply suppressed the CSV reader from my workbench then recreated it and now it works as expected ! Thanks a lot everybody for your help.

On an other side this is not a particularly pleasant experience and I still wonder how I can compare the two workbenchs: the one with the old reader vs the new one which works. I know they are just text files so I will probably try to find the culprit parameter as I really want to understand what I did wrong.


End of story: I simply suppressed the CSV reader from my workbench then recreated it and now it works as expected ! Thanks a lot everybody for your help.

On an other side this is not a particularly pleasant experience and I still wonder how I can compare the two workbenchs: the one with the old reader vs the new one which works. I know they are just text files so I will probably try to find the culprit parameter as I really want to understand what I did wrong.

I'm glad that you found the solution. As for your question, I would recommend either of these two solutions to check if the workspace is properly handling the geometry:

  1. Either using a VertexCreator rather than the reader-specific geometry generator. If there is a problem creating the geometry, the <Rejected> port will be triggered, terminating the workspace with an error.
  2. Or using a GeometryFilter somewhere in your workflow and check for NULL geometries, using a Terminator to end the workspace with an error message if detected.

I'm glad that you found the solution. As for your question, I would recommend either of these two solutions to check if the workspace is properly handling the geometry:

  1. Either using a VertexCreator rather than the reader-specific geometry generator. If there is a problem creating the geometry, the <Rejected> port will be triggered, terminating the workspace with an error.
  2. Or using a GeometryFilter somewhere in your workflow and check for NULL geometries, using a Terminator to end the workspace with an error message if detected.

This is something I was already doing in my workbench but differently as I was checking the columns containing x and y values. But your solution is certainly better. Thanks again for all your help David.


Reply