I have some questions about how to make a CityGML more efficient.
Context: I have about 1.8m pipelines and cables. I want to visualise these as 3D cylinder/pipes (not actual ‘FMEPipeGeometry’), regardless of the ‘is it necesary to do this’. All features also have attributes like diameter and such. I want to write this to CityDB and CityGML. This results in huge filesize (somwhere like 15gb or 20gb).
What I did: My thought process was it would be more efficient to use relative geoetries (a circle) that I extrude/scale/offset/etc to the proper location using the transformation matrix. Of course I cut the original geometry into pairs of 2 vertices. I wanted to reaggregate this to 1 feature once I had the relative geometry up and working. This all worked just fine until I discovered that in CityGML schema the cardinality is 0-1 on features with relative geometry. This means I cannot gather all separate bits of the cables again, giving me an even more absurd amount of features. I also want to keep the attributes, meaning each bit of cable would have to get these attributes. A lot more unneccessary data. Sizewise it seems to be better to forget about instancing and make each object its unique geometry.
The question: what can I try to make my CityGML more efficient? Except for switching to CityJSON which seems to reduce filesize by 6x, haha. I can generalize the geometries a little to reduce the amount of vertices. But I thought maybe there are possibilities to still use each relative geometry and in each of these geometries (of 2vertices) refer to relative attributes if that exists? So i dont have doubled attributes.
Its not really a specific issue that needs to be solved, I just want to ask the question to improve my understanding/knowledge of CityGML/FME.
Thanks in advance.
Page 1 / 1
Or maybe should I be looking into 3DTiles instead?
@virtualcitymatt and @redgeographics can probably provide some insight here
Thanks for the ping @hkingsbury,
CityGML does have the concept of CityObjectGroups which would probably let you group your parts of pipe into a group of objects - I’ll be honest though, CityObjectGroups as a feature has pretty limited support among tools which support CityGML partly because it’s so seldom used.
I’ve definitely seen Pipes created with relative geometries (e.g., the ones you can see here ) But indeed the issue is that it creates a single feature for each section of pipe which is on a different orientation. I think the geometry used was just a 1m length section of pipe with a 1m diameter. The transformation matrix was use to rotate/scale the sections as needed. In the linked map you can click on the sections of pipe to see how many there are.
For instancing I would also recommend that you include the offset of the transformation in the actual insert point and just use the matrix to transform the scale and rotation. this helps performance when doing spatial filters.
Indeed 3DTiles makes sense for visualization - CityGML (or CityJSON) data are often the inputs for conversion to 3DTiles. And for sure if converting to 3DTiles (or any other format) then you would definitely benefit from relative geometries when it comes to rendering.
You mentioned CityDB - by that do you mean the 3DCityDB? or something else. If you’re working with CityGML then I definitely would recommend looking into the 3DCityDB for CtiyGML storage.
Cables and Pipes though are always a bit tricky because there’s no real definition of how to handle these in CityGML - No utility module. You’re just stuck with GenericCityObjects. If you’re really intersted you could look into creating a new Application Domain Extension (ADE) for CityGML. There might already be one but I don’t know of one off the top of my head.
Large file size just comes with the territory of CityGML I’m afraid - That’s why I’m a big pusher of the 3DCityDB.
Hi @virtualcitymatt , thanks for your detailed reply.
Yes, I mean 3DCityDB. We are working on getting it up and running. I am aware of some ADE's for utilitynetworks but they don't seem very complete or active.
Could you explain why this issue would be resolved if I used 3DCityDB? What does it do differently (apart from being a database compared to a static written file of course) to make it better? If I understand correctly: if I would write my feature to a 3DCityDB instead of a CityGML, wouldn't I still have a separate record with attributes for each 2-vertex part of one pipe? Or can I aggregate all geometry-instances into 1 FME Aggregate which holds the attributes and push this to the CityDB?
Thanks for your tip on the offset of the transformation. The way I have it now it makes a referencePoint geometry; I believe this is also what you suggest(?).
I tried today to make CityObjectGroups from FME but I can't precisely figure out how to actually group them and have 1 group with a lot of groupmembers (2-vertex chopped part of pipes). I also can't find very much documentation on how to get it working.
Hi @virtualcitymatt , thanks for your detailed reply.Yes, I mean 3DCityDB. We are working on getting it up and running. I am aware of some ADE's for utilitynetworks but they don't seem very complete or active.
Yeah 100%, it’s still quite academic at this point - And with CityGML 3 now here it would make more sense to make something targeted at that CityGML version.
Could you explain why this issue would be resolved if I used 3DCityDB? What does it do differently (apart from being a database compared to a static written file of course) to make it better? If I understand correctly: if I would write my feature to a 3DCityDB instead of a CityGML, wouldn't I still have a separate record with attributes for each 2-vertex part of one pipe? Or can I aggregate all geometry-instances into 1 FME Aggregate which holds the attributes and push this to the CityDB?
No, you’re right you end up with the same thing - it was more just to avoid the static files. Keep in mind that you will need to still output CityGML (or CityJSON) if you plan to import the data into the 3DCityDB - you can’t directly write features to the 3DCityDB with FME. There is a CLI for the 3DCityDB client though so you can automate steps if you need to. Also do try and keep a consistent gml:id/feautreId for your features. This makes it much easier to maintain in the future when the source data gets modified. I would also recommend that you add a prefix to the gml:id in order to easily distinguish between types of generic objects. Generic Attributes are OK, however, when it comes down to working with the database it makes it much easier when writing queries if you can just use the gml:id.
Thanks for your tip on the offset of the transformation. The way I have it now it makes a referencePoint geometry; I believe this is also what you suggest(?).
Right - I just say it because I’ve seen some cases where the referencePoint has been 0,0 and the offset has been applied in the transformation matrix.
I tried today to make CityObjectGroups from FME but I can't precisely figure out how to actually group them and have 1 group with a lot of groupmembers (2-vertex chopped part of pipes). I also can't find very much documentation on how to get it working.
Yeah - I don’t actually know anyone who has attempted to do this. I just know that the schema supports this on some level.
One thing to consider is taking advantage of the external reference component in CityGML to link the pipes to a master pipe/cable ID. Depending on your application you could leverage this to dynamically request the attributes from the original source. Definitely not ideal.
In general the typical solution is to have a separate generic city object for each section of pipe. The limitation is also true for trees/SolitaryVegetationObjects. One model per feature. I’ve seen users want to create semantic objects with variable Crown, Trunk and Root elements but this isn’t supported so each component is it’s own feature.
Hi @virtualcitymatt, thanks for your again detailed response.
Interesting about the limitation of not being able to write directly to a 3DCityDB using FME. I did not expect that! If in essence 3DCityDB is based on a PgSQL with a custom schema, why is it not possible for FME?
Based on your help I'm not going to spend time on figuring out CityObjectGroups then, I will just accept my fate and produce very large static CityGML files where each straight part of a pipe is an instanced geometry with a lot of attributes
Hi @virtualcitymatt, thanks for your again detailed response.
Interesting about the limitation of not being able to write directly to a 3DCityDB using FME. I did not expect that! If in essence 3DCityDB is based on a PgSQL with a custom schema, why is it not possible for FME?
Based on your help I'm not going to spend time on figuring out CityObjectGroups then, I will just accept my fate and produce very large static CityGML files where each straight part of a pipe is an instanced geometry with a lot of attributes
Yeah - the database schema has about 60ish tables in there all linked in various ways.I guess in theory you could write directly to the database I just don’t think it’s really worth it. You would need to update a lot of tables and make sure all the keys are properly linked and all the sequences are correctly updated etc etc. the same is also true when it comes to reading form it. You can read from it with FME but it’s quite a task to properly support all the features. It’s not as easy as just connecting up a reader to PostGIS and getting the data. You have to access multiple tables and perform various joins.
On that note though, the 3DCityDB team are working on a new version of the database which is much, much more simplified - The schema is pretty stable now but the client for still needs some work to be able to support all of the features which version 4 offers and there is currently no GUI for it. It’s still not as easy as just pointing a PostGIS reader at it but it’s closer.
Yes, skip the CityObjectGroups. If you ever get a solution to create the instanced geometries from the lines I’d love to take a look and what you were able to put together.
Also, great that you are working on creating CityGML. If you have more questions about CityGML feel free to send me a PM on here and I will do my best to try and answer
I 'finished’ my workspace. I am slightly disappointed with being unable to use relative geometries inside a multigeometry feature (aggregate). I wanted to include the pipes as 2D lines as LOD0/1 and the 3D pipes as LOD2. But this is not possible if LOD2 is the instanced geometry.
I also used different shapes of pipes, in essence they are 1x1meter geometries. I add these as instance, and then scale it by the pipe segment length and the diameter (attribute driven). Then I offset it to the starting point (referencepoint). Then I use rotator to correct the horizontal (2D) orientation. Then I finish it by doing a 3D Rotation (with the reference point as its origin to rotate around). The 3D Rotation is used to offset it with the DEM; the pipes don't have individual Z values from the source data so I can only assume a standard depth per type of pipe in combination with DEM.
See image and sample GML (saved as TXT because GML cant be attached here) for your mentioned interest, @virtualcitymatt .