Skip to main content
Question

Output buildings into individual .obj files

G
Forum|alt.badge.img

So I have a large CityGML file, which is the LoD2 model of New York City. I now would like to export all buildings into individual .obj files. Currently I am facing two challenges:

  • The file is too large (30G) to be loaded into FME data inspector directly.
  • I do not know how to extract individual buildings in a programmatic way.

I already loaded the data file into a 3D City Database using the importer / exporter. But still confused how to solve the two technical challenges above. Any advice would be helpful. Thanks!

11 replies

scyphers
Participant
Forum|alt.badge.img+5
  • scyphersParticipant
  • Participant
  • January 5, 2019

Hi @greenstone1564,

Two ways:

Depending whether or not your buildings have surface materials, if they do NOT you could introduce a counter or make use of some existing building ID attribute in your GML data to fanout the .obj files. This will broadcast one .obj files for every distinct UID, this parameter setting is available in the OBJ writer on your canvas.

 

OR if you have photo tessellated buildings with attached .obj materials you need to address/keep you can change the writer fanout (in Navigator Pane) from No...

 

To have some sort of fancy directory level creation step as to not overwrite the materials directories on each UID (as shown above), this may not be an issue coming from a GML file.

This will broadcast a directory for each building UID and an UID.obj in that folder, if materials exist they will cascade into those directories. (NYC wow so many folders) but if you need to break them down that is two ways to shard the buildings out as individual files OR files in directories.

:)

jovitaatsafe
Safer
Forum|alt.badge.img+11

Hi @greenstone1564,

 

For the first problem of file being very large, virtualcitymatt's answer on this Q&A has some good insight about tiling the dataset in 3D City Database by exporting from it. According to 3D City DB's documentation, you should be able to export to CityGML and tile the dataset. There's a bit more on it on pages 142 and 171 in the pdf about exporting bounding boxes. I'm afraid I don't have personal experience in using it though.

Once you have it tiled to smaller and more manageable files, you can write out separate obj files by building using a fanout. Setting the writer feature type name to the attribute gml_parent_id should do it. I'd suggest testing this by reading in a select number of features by setting the Max Features to Read in the Reader in the Navigator before running a whole dataset through. Hope that helps a bit!

G
Forum|alt.badge.img
jovitaatsafe wrote:

Hi @greenstone1564,

 

For the first problem of file being very large, virtualcitymatt's answer on this Q&A has some good insight about tiling the dataset in 3D City Database by exporting from it. According to 3D City DB's documentation, you should be able to export to CityGML and tile the dataset. There's a bit more on it on pages 142 and 171 in the pdf about exporting bounding boxes. I'm afraid I don't have personal experience in using it though.

Once you have it tiled to smaller and more manageable files, you can write out separate obj files by building using a fanout. Setting the writer feature type name to the attribute gml_parent_id should do it. I'd suggest testing this by reading in a select number of features by setting the Max Features to Read in the Reader in the Navigator before running a whole dataset through. Hope that helps a bit!

Thanks a lot! This is very helpful and probably saved me days of reading the documentation. I knew the answer you linked to, but had trouble finding exactly how to do tiled export. The pointing to page 171 is very helpful, though the documentation leaves a little gaps and I'm still trying to figure that out.

 

A remaining issue is, suppose I output 100 tiles and successfully loaded them into FME data inspector. Then to export all buildings in these 100 tiles, does this mean I have to manually use the fanout on every .gml tile? Just to make sure I am not misunderstanding anything here.

virtualcitymatt
Celebrity
Forum|alt.badge.img+35
greenstone1564 wrote:

Thanks a lot! This is very helpful and probably saved me days of reading the documentation. I knew the answer you linked to, but had trouble finding exactly how to do tiled export. The pointing to page 171 is very helpful, though the documentation leaves a little gaps and I'm still trying to figure that out.

 

A remaining issue is, suppose I output 100 tiles and successfully loaded them into FME data inspector. Then to export all buildings in these 100 tiles, does this mean I have to manually use the fanout on every .gml tile? Just to make sure I am not misunderstanding anything here.

This doesn't have to be a manual process. You can use the WorkspaceRunner transformer to run the process as a batch. I would also recommend that you use a FeatureWriter as the writer instead of a normal Writer. As I understand it the Performance of the 'fanout' can be better. Here's an example of how to use the Worksapce runner: https://knowledge.safe.com/articles/1469/batch-processing-using-the-workspacerunner-1.html

 

Essentially you have a child workspace which reads in a tile and uses the fanout to output one file per building. The parent workapce simple reads the filenames of the tiles using a PATH reader and passes this Information into the WorkspacRunner.

 

 

Just out of curiosity can I ask why you're wanting to have a separate .obj for each building?

 

 

P.S - Don't load these into the FME Data Inspector. Just use FME Workbench for this
G
Forum|alt.badge.img
virtualcitymatt wrote:

This doesn't have to be a manual process. You can use the WorkspaceRunner transformer to run the process as a batch. I would also recommend that you use a FeatureWriter as the writer instead of a normal Writer. As I understand it the Performance of the 'fanout' can be better. Here's an example of how to use the Worksapce runner: https://knowledge.safe.com/articles/1469/batch-processing-using-the-workspacerunner-1.html

 

Essentially you have a child workspace which reads in a tile and uses the fanout to output one file per building. The parent workapce simple reads the filenames of the tiles using a PATH reader and passes this Information into the WorkspacRunner.

 

 

Just out of curiosity can I ask why you're wanting to have a separate .obj for each building?

 

 

P.S - Don't load these into the FME Data Inspector. Just use FME Workbench for this

I see. I'll look into that (in fact how to use FME workbench - I'm very new to this and yet to understand all functionalities!) Thanks!

 

As for why I need individual buildings: I would like to write a machine learning model and try generate buildings. so each building will be a single training sample.
jovitaatsafe
Safer
Forum|alt.badge.img+11
greenstone1564 wrote:

I see. I'll look into that (in fact how to use FME workbench - I'm very new to this and yet to understand all functionalities!) Thanks!

 

As for why I need individual buildings: I would like to write a machine learning model and try generate buildings. so each building will be a single training sample.

Neat project! Just wanted to mention that we have free training courses that you can register for if you're interested. The next live online 2-day Desktop Basics course is tomorrow actually (Jan8th at 8am PST), but we also have them recorded to do at your own pace whenever. Also there's some good videos from our webinars here for topic-specific things. Happy FME'ing!

G
Forum|alt.badge.img
jovitaatsafe wrote:

Neat project! Just wanted to mention that we have free training courses that you can register for if you're interested. The next live online 2-day Desktop Basics course is tomorrow actually (Jan8th at 8am PST), but we also have them recorded to do at your own pace whenever. Also there's some good videos from our webinars here for topic-specific things. Happy FME'ing!

Thanks for pointing me to the resources Jovita. I just found them too and am watching the tutorial now :)

G
Forum|alt.badge.img
jovitaatsafe wrote:

Hi @greenstone1564,

 

For the first problem of file being very large, virtualcitymatt's answer on this Q&A has some good insight about tiling the dataset in 3D City Database by exporting from it. According to 3D City DB's documentation, you should be able to export to CityGML and tile the dataset. There's a bit more on it on pages 142 and 171 in the pdf about exporting bounding boxes. I'm afraid I don't have personal experience in using it though.

Once you have it tiled to smaller and more manageable files, you can write out separate obj files by building using a fanout. Setting the writer feature type name to the attribute gml_parent_id should do it. I'd suggest testing this by reading in a select number of features by setting the Max Features to Read in the Reader in the Navigator before running a whole dataset through. Hope that helps a bit!

Thanks for this very helpful tip, I eventually am able to get the .obj files for every building based on gml_parent_id. One remaining issue is, now every building has three obj files: GroundSurface, Roofsurface, and Wallsurface. Together they will form a closed geometry... But how can I "merge" them into a single .obj file ? Is there a convenient way to do this in FME? Thanks!

 

Edit: I think I find the solution --- use an aggregator. Leave the comment here just in case helpful to others.

 

virtualcitymatt
Celebrity
Forum|alt.badge.img+35
greenstone1564 wrote:

I see. I'll look into that (in fact how to use FME workbench - I'm very new to this and yet to understand all functionalities!) Thanks!

 

As for why I need individual buildings: I would like to write a machine learning model and try generate buildings. so each building will be a single training sample.

Ah, very interesting indeed!

 

 

Good luck with that project, really neat idea. If you're interested in more training data check out these download portals where you can get more buildings if needed.

https://download-berlin3d.virtualcitymap.de/?lang=en&lang=en#/export - City of Berlin (Germany)

 

https://3drotterdam.nl/#/export# - City of Rotterdam (Netherlands)

 

https://soest.virtualcitymap.de/#/export - City of Soest (Germany)

 

http://195.216.117.146/#/export - City of Poznan (Poland)

 

 

You can also contact these cities directly to see if you can get a copy of the full datasets. Germany and the Netherlands are great sources for 3D Buildings if you are needing more training data.
jovitaatsafe
Safer
Forum|alt.badge.img+11
greenstone1564 wrote:

Thanks for this very helpful tip, I eventually am able to get the .obj files for every building based on gml_parent_id. One remaining issue is, now every building has three obj files: GroundSurface, Roofsurface, and Wallsurface. Together they will form a closed geometry... But how can I "merge" them into a single .obj file ? Is there a convenient way to do this in FME? Thanks!

 

Edit: I think I find the solution --- use an aggregator. Leave the comment here just in case helpful to others.

 

That's definitely one possible solution, another alternative is to make sure that all of the reader feature types (Roof and walls and ground) are connected to a single writer feature type. Glad to hear you're having success in your project!
G
Forum|alt.badge.img
jovitaatsafe wrote:
That's definitely one possible solution, another alternative is to make sure that all of the reader feature types (Roof and walls and ground) are connected to a single writer feature type. Glad to hear you're having success in your project!

Thanks Jovita. My issue is solved perfectly.

Reply


Helpful Members This Week

Community Stats

31,096
Posts
117,900
Replies
38,853
Members

Latest FME

Terms & ConditionsCookie settings

Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.