Issue Geometry Type when using a dynamic GPKG writer

I think you uploaded the wrong example here. this is the same CustomTransformer workspace from one of your previous questions.

Ah, you are absolutely right, sorry.

I just changed the upload to the right file.

I would say this is a bug - or at least if not a bug then falling short of what it should be doing.

The geometry (if derived from schema) should come from the fme_geometry{} list attribute. In this case there is only fme_point so I am at a loss as to why FME would make it a multipoint.

when reading the GeopackageType back into FME for sure it's coming in as multipoint. You need a deaggregator to get your point back. This seems wrong.

And yeah, when you set it to Point directly in the write I see what you mean. The type in the file is GEOMETRY. This also seems wrong to me, especially comparing how it's treated in Automatic mode. When reading back into FME though I do see that it is a point and not a multi-point. It's hard to say how other applications might treat/support this output. I can definitely imagine an application which only supports geopackage files with layers with properly defined geometry types. Do you think this would be acceptable to your client?

I would submit a ticket to Safe directly via support (https://community.safe.com/s/submit-case) on this with your reproduction workspace and link to this discussion.

Check, tnx again for the feedback virtualcitymatt! I just created a case (C692709).

That the GPKG writer creates (geopackage_)multipoint when the schema indicates fme_point geometry is indeed a bit odd, and I believe (geopackage_)point would be more applicable. However, I personally believe that it's more of an issue that there also seems to be no way to manually set the preferred geometry type at the dynamic GPKG writer... 😞

Maybe the GPKG geometryType (geopackage_)geometry was chosen for dynamic writing as a more generic type... As you mention FME seems to pick this up fine, but indeed that may not be the case for every application. In my case our 'data broker' has implemented a geometry type requirement for the intake of GPKG files. See also 'RQ14' at Voor data-aanbieders - PDOK (It's in Dutch, I hope auto/google translate works for you if preferred/necessary).

This document is a great reason that Safe should get this sorted and I'm 100% with you that you should be able to choose the geometry type here.

One thing I tried which which may or may not work for the delivery here is follow the writing with an SQL updater to update the geometry type in the table.

It's 100% hacky and there may be other things are effected but could be worth a shot?

​

Haha, tnx. I'll just patiently wait for their feedback on this.

With regards to your suggestion. Using an SQLExecutor to update the geometry type in the underlying SQLite table could indeed be a workaround. However, in my case there was an easier workaround; I just generated the entire dataset again using my original workspace. And another alternative could be to not use the dynamic writer, but set the feature type definition in the writer in the classic 'Manual' way.

So while I appreciate your suggestions, I'm not necessarily looking for a workaround here. I just felt my responsibility to share my experience on this (as I thought it wasn't correct behavior), so that the community and Safe is aware of this, and hopefully Safe can address this is a future release.

Thanks @thijsknapen​ what version of FME are you using? It seems we have discovered this and have fixed it for FME 2022.1, are you able to upgrade to a newer version? Internal reference: FMEENGINE-73547

Hi @Evie Lapalme​ 

That's odd. My workspace was created with version '2022.1.0.0'. I just retested for versions '2022.2.2.0' and '2023.0.0.1', but I'm still getting the same results.

Can you confirm/show that when you run the attached workspace, at the inspectors 'GPKG_geom_1-3' you are getting different results than shown below;

And if so, what version of FME are you using?

I was also using FME 2022.2.5 when I tested your workspace

Hi @Evie Lapalme​ ,

Is it possible to provide an update on this case/ticket (C692709)?

If I look at the ticket myself, all I see on the status is 'Closed - With Development'.

I'm curious to hear if this has indeed been identified/characterized as a bug, and if so, when we can expect to see a fix for it.

Sorry for the late reply. Unfortunately this issue is apparently a limitation of FME and due to design decisions will not be fixed. Some more details and explanation below. Sorry for this inconvenience!

We do not distinguish the multipoint as a distinct type from point as GeoPackage does. The automatic mapping prefers the Geopackage Multipoint type over the singular version because we report both multi points and points as an fme_point. If we were to create a table with the simple point type and then write a multipoint to the file. Other formats will have a similar mapping where a multi takes precedence over a singular geometry type. This is especially important where formats treat multis and their single versions as distinct types. FME treats multis the same as singular (eg, multiline vs. lines, multipolygons vs polygons), and so when mapping to a destination format, we have to make the assumption that multis can sneak into the output so we choose a mapping that is guaranteed to work.

Hi @Evie Lapalme​ ,

Thanks for the response. That is an explanation to the case/situation (lets type it as 'B');

Using a dynamic GPKG writer with the Geometry parameter set to 'From Schema Definition' (which was 'fme_point') results in a GPKG with 'geometry_type_name' = 'MULTIPOINT'.

However as I already mentioned I don't see that as the (main) issue. See e.g. in the text of the question;

"When I investigated a bit further, it seems to me that a dynamic GPKG writer assumes multi geometry (in my case multi point), in case the writer is configured to use the Geometry 'From Schema Definition' (fme_point). Ok, I can get my head around that."

and my comment to virtualcitymatt;

"That the GPKG writer creates (geopackage_)multipoint when the schema indicates fme_point geometry is indeed a bit odd, and I believe (geopackage_)point would be more applicable. However, I personally believe that it's more of an issue that there also seems to be no way to manually set the preferred geometry type at the dynamic GPKG writer..."

Can you maybe also provide feedback on what I would call the main issue (lets call it case/situation 'A');

Using a dynamic GPKG writer with the Geometry parameter set to 'geopackage_point' results in a GPKG with 'geometry_type_name' = 'GEOMETRY',

whereas using a manual GPKG writer with the Geometry parameter set to 'geopackage_point' results in a GPKG with 'geometry_type_name' = 'POINT'.

I thought I had this all pretty well described/shown in the sample workspace I attached. Did you open/inspect this workspace (and preferably run it for yourself)?

Apologies, I believe I may have conflated the two issues together myself as well as the developers on the issue. I have now emphasized this difference to the developers and will update you, but as previously stated it does sound like a bug to me at least. I'm not sure there would be an quick fix or easy workaround regardless. (beyond using the regular/manual GPKG writer)

As well, the 'Closed - With Development' status on the case means that the developers are actively investigating this issue and you will be automatically updated through the case when a fix is implemented and in what version. Hope this helps :)

Hi @Evie Lapalme​ , thanks for emphasizing the main issue to the developers, and the way in which this is different from the second observation/remark (type B).

In the mean type I'll use the manual GPKG writer as a workaround (not ideal, as there are many feature types and attributes, but its manageable).

For the long term solution, I'll wait patiently on the developers. I understand that resolving tickets like this takes time...

I'm already happy with the fact that it is being addressed, i.e. that there seems to be a forecast that a fix for this will be introduced in a new FME version.

Hi ​@evieatsafe ,

Is there an update on this? As I tried it out with FME 2024.1.3 it stills seems to be an issue (unless there is a workaround that works for the dynamic settings that I missed )

Hi ​@tva it looks like this has been delivered in FME 2024.1

I’m currently investigating if/why this might not have been released, or if there is a potential regression. (internal tracking FMEENGINE-84774)

To clarify on dynamic writing, you should be setting your Geometry = From Schema Definition in the dynamic writer to get the correct geometry set for each feature.