Question

Esri i3s data converted by FME Desktop is not available in ArcGIS Portal


use FME Desktop 2018 to convert a revit data into Esri I3S format(Indexed 3D Scene Layer).

the exported data can be viewed in ArcGIS Pro 2.1 and sucessfully published as a scene service in ArcGIS Portal 10.6.

but the scene service from this exported i3s data cann't be viewed incorretly in Portal scene viewer.

thanks all!


21 replies

Badge +2

This sounds more like a scene service configuration issue than an i3s issue. Can you confirm that you have created some i3s data in ArcPro, created a related scene service and can view that in your portal. More on Esri scene service here.

If you attach a small example of the i3s you have created in FME and cannot register in ArcPro then perhaps someone can give you additional ideas.

Badge +13

@MarkAtSafe

We, over here, have been wrestling with the same problem.

  • We convert an assortment of 3D files to .slpk using FME 2018.1.0.3 (and 2019 -build 19136)
  • We publish them as hosted-scenelayer to ArcGIS Online using the RestAPI and/or the ArcGIS python library
  • These hosted layers can be viewed in ArcGIS Pro
  • These hosted layers can NOT be viewed in the ArcGIS online viewer.
    • The viewer pans to the right spot on the globe, but does not show anything

We also followed another route:

  • Convert various 3D files to MultiPatch objects in a ESRI geodatabase (file geodb)
  • Publishing these as a hosted-scenelayer using ArcGIS Pro
  • These can be viewed in ArcGIS Pro and in the scene viewer.

On close inspection there is one difference between the .slpk files, when looking at the Properties / source tab in ArcGIS Pro.

The Pro-generated .slpk files have a vertical coordinate system (NAVD 1988 / epsg:5703). The FME-generated .slpk files do not have a vertical coordinate system.

This feels like a reason why the scene-viewer pans to the right spot, but does not show a 3D object.

An ArcGIS Pro scene, in which one views a scenelayer service, has a default vertical coordinate system... probably making up for the missing one in the FME generated .slpk.

Anyone know a way to work around this and add a vertical coordinate system on the .i3s writer, using the current FME-versions?

Kind regards,

Martin

Badge +2

@MarkAtSafe

We, over here, have been wrestling with the same problem.

  • We convert an assortment of 3D files to .slpk using FME 2018.1.0.3 (and 2019 -build 19136)
  • We publish them as hosted-scenelayer to ArcGIS Online using the RestAPI and/or the ArcGIS python library
  • These hosted layers can be viewed in ArcGIS Pro
  • These hosted layers can NOT be viewed in the ArcGIS online viewer.
    • The viewer pans to the right spot on the globe, but does not show anything

We also followed another route:

  • Convert various 3D files to MultiPatch objects in a ESRI geodatabase (file geodb)
  • Publishing these as a hosted-scenelayer using ArcGIS Pro
  • These can be viewed in ArcGIS Pro and in the scene viewer.

On close inspection there is one difference between the .slpk files, when looking at the Properties / source tab in ArcGIS Pro.

The Pro-generated .slpk files have a vertical coordinate system (NAVD 1988 / epsg:5703). The FME-generated .slpk files do not have a vertical coordinate system.

This feels like a reason why the scene-viewer pans to the right spot, but does not show a 3D object.

An ArcGIS Pro scene, in which one views a scenelayer service, has a default vertical coordinate system... probably making up for the missing one in the FME generated .slpk.

Anyone know a way to work around this and add a vertical coordinate system on the .i3s writer, using the current FME-versions?

Kind regards,

Martin

Hi @martinkoch, are there textures on the I3S model? We are aware of an existing issue with texture coordinates, which could cause similar problems. Would you mind attaching a small sample of the I3S that demonstrate the problem? So we could confirm it's related to our known issue? Thank you!

Badge +13

Hi @martinkoch, are there textures on the I3S model? We are aware of an existing issue with texture coordinates, which could cause similar problems. Would you mind attaching a small sample of the I3S that demonstrate the problem? So we could confirm it's related to our known issue? Thank you!

Sorry for responding late.

There is some styling involved in this (colors) but no textures as I know of. This issue has been reported and discussed bij a team member of mine under support case C140844. We reported this multi-channel ;-). It seemed to fit in the problem reported by @pkusir

Badge +2

Sorry for responding late.

There is some styling involved in this (colors) but no textures as I know of. This issue has been reported and discussed bij a team member of mine under support case C140844. We reported this multi-channel ;-). It seemed to fit in the problem reported by @pkusir

Thanks for the case number @martinkoch. We'll make sure to get back to you as soon as possible.

Badge

@martinkoch @XiaomengAtSafe @MarkAtSafe Hello, we have the same problem. And I've tried to manually add a vertical coordinate system to the slpk file's 3dSceneLayer.json file. Basically by unzipped the slpk file then unzipped the json.gz file to add vcswkid and lastestvcswkid, then put them back all together.the good news is that we can still open it in arcgis pro and it did shows a vertical coordinate system, but still no show in portal. Also we tried your way to put them into a geodatabase then to portal, well,this worked.

Badge

@martinkoch @XiaomengAtSafe @MarkAtSafe Hello, we have the same problem. And I've tried to manually add a vertical coordinate system to the slpk file's 3dSceneLayer.json file. Basically by unzipped the slpk file then unzipped the json.gz file to add vcswkid and lastestvcswkid, then put them back all together.the good news is that we can still open it in arcgis pro and it did shows a vertical coordinate system, but still no show in portal. Also we tried your way to put them into a geodatabase then to portal, well,this worked.

But in that way, it actually published a esriGeometryMultiPatch instead of a slpk. And if we use the gp tool to convert feature in that geodatabase into a slpk , then it has a vertical coordinate system and no difference from any original arcgis slpk.

So my conclusion is, this is still a problem of how does the fme write\\generate the slpk file.

 

Badge +9

But in that way, it actually published a esriGeometryMultiPatch instead of a slpk. And if we use the gp tool to convert feature in that geodatabase into a slpk , then it has a vertical coordinate system and no difference from any original arcgis slpk.

So my conclusion is, this is still a problem of how does the fme write\\generate the slpk file.

 

Hi @karltk,

Would you be able to try the latest 2019 beta as a test? We've recently made some improvements that may resolve this issue. I'd love to know if there is still a problem that our development team needs to be aware of.

 

Brian
Badge

Hi @karltk,

Would you be able to try the latest 2019 beta as a test? We've recently made some improvements that may resolve this issue. I'd love to know if there is still a problem that our development team needs to be aware of.

 

Brian

Sorry Brian @BrianAtSafe ♦ , but this is 2019, you should see my other post, 2018's output slpk file can't be used at all.

Badge

@martinkoch @XiaomengAtSafe @MarkAtSafe @BrianAtSafe ♦

Hi everyone

 

I found the problem that the slpk file can't be loaded into arcgis portal.

When the fme generate a slpk file, it only generate a single node.

while the arcgis pro's output have many layers

 

That not only result in a very inefficient way for online usage,but also concluded another fact.

 

With everything in one layer, that layer's texture file is huge.

 

In this particular case, 80MB.

 

And with only one layer, arcgis portal have to load all 80MB's texture file at the very beginning.

That's simply impossible.

 

Userlevel 1
Badge +10

@martinkoch @XiaomengAtSafe @MarkAtSafe @BrianAtSafe ♦

Hi everyone

 

I found the problem that the slpk file can't be loaded into arcgis portal.

When the fme generate a slpk file, it only generate a single node.

while the arcgis pro's output have many layers

 

That not only result in a very inefficient way for online usage,but also concluded another fact.

 

With everything in one layer, that layer's texture file is huge.

 

In this particular case, 80MB.

 

And with only one layer, arcgis portal have to load all 80MB's texture file at the very beginning.

That's simply impossible.

 

Hi @karltk:

Thanks for providing us with your findings! We are investigating this and will hopefully provide an update here on this thread shortly.

In the meantime, would you mind providing us with some additional information to help with our investigation:

  1. With respect to the nodes, have you adjusted the i3S Reader Parameter "Maximum Number of Faces per Node"?
  2. Are you able to view it using ArcGIS Online?
  3. Have you tried a reduced dataset (i.e. one object)? If that doesn't work, are you able to provide us with a sample of your .slpk file?

 

Badge

Hi @karltk:

Thanks for providing us with your findings! We are investigating this and will hopefully provide an update here on this thread shortly.

In the meantime, would you mind providing us with some additional information to help with our investigation:

  1. With respect to the nodes, have you adjusted the i3S Reader Parameter "Maximum Number of Faces per Node"?
  2. Are you able to view it using ArcGIS Online?
  3. Have you tried a reduced dataset (i.e. one object)? If that doesn't work, are you able to provide us with a sample of your .slpk file?

 

Hello@Nampreet IS there an i3s reader? I assumed you mean the writer.

 

And I didn't adjust Maximum Number of Faces per Node at the first time.But then I tried and the default is 60000, while the minimum is 10000.

 

So I set at 10000, and it still get only one node.

 

And believe it or not, the texture file is even larger in this case, it's 101MB.

 

For your second question, still no show in arcgis online.

 

And this is a small sample, a obj file with all texture files still less then 50MB.

I can't attach the file here so uploaded to google drive, it's generated by FME 2019.0.0.0 (20181203 - Build 19166 - WIN64)

 

Userlevel 1
Badge +10

Hello@Nampreet IS there an i3s reader? I assumed you mean the writer.

 

And I didn't adjust Maximum Number of Faces per Node at the first time.But then I tried and the default is 60000, while the minimum is 10000.

 

So I set at 10000, and it still get only one node.

 

And believe it or not, the texture file is even larger in this case, it's 101MB.

 

For your second question, still no show in arcgis online.

 

And this is a small sample, a obj file with all texture files still less then 50MB.

I can't attach the file here so uploaded to google drive, it's generated by FME 2019.0.0.0 (20181203 - Build 19166 - WIN64)

 

Thanks for the sample data @karltk and for your patience as we looked into the matter. I see that the sample .slpk you provided is in a Mercator projection. In your FME workspace, try adding a Reprojector and set the Destination Coordinate System to a Lat/Long coordinate system such as "LL-WGS84". Let me know if you have any luck with it in ArcGIS Online after reprojecting it.

With respect to ArcGIS Portal, we tried uploading .slpk files to Portal on ArcGIS 10.5.1 and 10.6 without any success. We were able to successfully upload our .slpk files to Portal on ArcGIS 10.6.1. It might be related to the issue reported here: https://community.esri.com/thread/210708-cityengine-slpk-and-portal-for-arcgis. What version of ArcGIS Portal are you using? Are you able to upgrade to 10.6.1?

Badge

Thanks for the sample data @karltk and for your patience as we looked into the matter. I see that the sample .slpk you provided is in a Mercator projection. In your FME workspace, try adding a Reprojector and set the Destination Coordinate System to a Lat/Long coordinate system such as "LL-WGS84". Let me know if you have any luck with it in ArcGIS Online after reprojecting it.

With respect to ArcGIS Portal, we tried uploading .slpk files to Portal on ArcGIS 10.5.1 and 10.6 without any success. We were able to successfully upload our .slpk files to Portal on ArcGIS 10.6.1. It might be related to the issue reported here: https://community.esri.com/thread/210708-cityengine-slpk-and-portal-for-arcgis. What version of ArcGIS Portal are you using? Are you able to upgrade to 10.6.1?

I don't understand why wgs84? it need web mercator to show online.

 

Never the less I tried wgs84 and no still no good.

 

Even worse now it's not properly showed in ArcGIS Pro too.

That's when the scene is wgs84

 

 

That's the scene in web mercator.

 

And we are using both 10.6 and 10.6.1.

 

Userlevel 1
Badge +10

I don't understand why wgs84? it need web mercator to show online.

 

Never the less I tried wgs84 and no still no good.

 

Even worse now it's not properly showed in ArcGIS Pro too.

That's when the scene is wgs84

 

 

That's the scene in web mercator.

 

And we are using both 10.6 and 10.6.1.

 

Hi @karltk, would it be possible for you to share the FME workspace you are using to generate your i3s (.slpk) file with a small sample of your source data? You can do this by saving your workspace as a Template (.fmwt file, which will package up your source data and workspace together). We would like to replicate the issue to determine what is happening.

If you prefer not to share your data/workspace here, feel free to submit a Support Case and we can investigate this further through the Case.

Badge

@Nampreet

@martinkoch @XiaomengAtSafe @MarkAtSafe @BrianAtSafe ♦

Hi everyone

 

Quick update, in ArcGIS Pro 2.3, there is a new gp tool, Validate Scene Layer Package, to check if the slpk file is ok.

 

And it's quick useful. Find lots of problems in the slpk file generated by FME 2019.

 

I think you better take a look into this. Thank you.
Badge +2

@Nampreet

@martinkoch @XiaomengAtSafe @MarkAtSafe @BrianAtSafe ♦

Hi everyone

 

Quick update, in ArcGIS Pro 2.3, there is a new gp tool, Validate Scene Layer Package, to check if the slpk file is ok.

 

And it's quick useful. Find lots of problems in the slpk file generated by FME 2019.

 

I think you better take a look into this. Thank you.

Thank you @karltk, for the update with this new info. @nampreet is doing some testing and will work with the development team on this issue. And we will post updates here, when there is some progress.

Userlevel 1
Badge +10

@Nampreet

@martinkoch @XiaomengAtSafe @MarkAtSafe @BrianAtSafe ♦

Hi everyone

 

Quick update, in ArcGIS Pro 2.3, there is a new gp tool, Validate Scene Layer Package, to check if the slpk file is ok.

 

And it's quick useful. Find lots of problems in the slpk file generated by FME 2019.

 

I think you better take a look into this. Thank you.

Hi @karltk, Sorry to hear that you're still experiencing issues with your i3S transformations. We unfortunately cannot determine what is happening with your i3S output without looking at your source data and worksapce. Are you able to share a sample of your source data and your workspace with us? If not here, we can help you through a Support Case. You can open submit a Support Case here.

I spoke to a developer and he said that from the validation report you provided, obj2-validatescenelayerpacka.txt, it doesn't look like an slpk that was generated by FME, because FME doesn't write out textures in DDS format. Are you able to confirm that?

He also suggested that another possibility is that the data might be "below ground." He has seen this problem with some datasets, where all buildings are at 0 elevation (ie, on the ellipsoid). If you have terrain enabled in ArcGIS Pro/Online, etc., the terrain will cover the buildings. So that's another possibility. If this is the case, you can try something like getting the center point of each feature, and offset it upwards to the elevation value from a DEM.

There are changes coming to the i3S Reader in FME 2019. FME 2019 is currently in Beta and therefore not complete. However, I'm informed that there will be further changes to the i3S Writer in the next Beta Release.

Again, we'll be glad to help you determine what is happening but we will need to take a look at your source and workspace.

Userlevel 1
Badge +10

@martinkoch @XiaomengAtSafe @MarkAtSafe @BrianAtSafe ♦

Hi everyone

 

I found the problem that the slpk file can't be loaded into arcgis portal.

When the fme generate a slpk file, it only generate a single node.

while the arcgis pro's output have many layers

 

That not only result in a very inefficient way for online usage,but also concluded another fact.

 

With everything in one layer, that layer's texture file is huge.

 

In this particular case, 80MB.

 

And with only one layer, arcgis portal have to load all 80MB's texture file at the very beginning.

That's simply impossible.

 

FYI: The issue with the large (in this case 80mb) texture should already be fixed in FME 2019.0 Beta.

Badge +6

@Nampreet

@martinkoch @XiaomengAtSafe @MarkAtSafe @BrianAtSafe ♦

Hi everyone

 

Quick update, in ArcGIS Pro 2.3, there is a new gp tool, Validate Scene Layer Package, to check if the slpk file is ok.

 

And it's quick useful. Find lots of problems in the slpk file generated by FME 2019.

 

I think you better take a look into this. Thank you.

Thanks for pointing this out. This should be fixed in 2019.1 beta b19604 and will be in the 2019.1 release. Our apologies this took so long to deal with. Part of the complexity is that this is a write only format that is used within a complex third party system (ArcPro scene layers) so it can be hard to isolate and reproduce problems.

Badge +1

Thanks for pointing this out. This should be fixed in 2019.1 beta b19604 and will be in the 2019.1 release. Our apologies this took so long to deal with. Part of the complexity is that this is a write only format that is used within a complex third party system (ArcPro scene layers) so it can be hard to isolate and reproduce problems.

Video_2020-01-09_135712.zipUpgrading from 2018.1 to 2019.2 fixed the issue of missing textures in ArcGIS Pro and missing data in ArcGIS Online Viewer. However, I have the problem that only a small amount of buildings is visible in ArcGIS Online. When I zoom, existing buildings vanish and new buildings appear. I guess, the generation of different level of details at various zoom levels causes problems here. This problem does not occur in ArcGIS Pro.

Reply