Skip to main content

Data

Using readers and writers to extract and load data.

  • 7,006 Posts
  • 25,815 Replies
7006 Posts
sar_copp
Contributor
sar_coppContributor
asked in Data
Help with creating a nested JSON from an Excel table

Hello,Is it possible to transform a table (ie from Postgresql, or Excel) into this complicated Json format within FME please? I have attached the json file and also a small dummy sample excel file (which doesnt quite have the correct column headers as its a work in progress).I am completely new to JSON and I have done the JSON Templater tutorial but no idea how to actually apply it to my needs!!Help wanted!!

662
sar_copp
Contributor
marta.podsiad
Supporter
marta.podsiadSupporter
asked in Data
Read Share Group information from Esri Enterpise

Hi,I am having trouble reading the names of the groups with which the layer is shared in Esri Enterprise. I am using an HTTPCaller with the following settings, but the Share info does not appear in the JSON response.The only information I get is:   "access": "shared". Could you please advise on how to retrieve this information?

355
marta.podsiad
Supporter
alexlynch3450
Contributor
alexlynch3450Contributor
asked in Data
SchemaMapper Advance Mapping Create Whitespace before evaluating data value

Hello All,I’m having an issue where SchemaMapper prefix whitespace (space) when using fmeexpression=.I attached a sample workspace and would appreciate if anyone could test before submitting ticket. This issue came apparent after upgrading to FME 2023. for example: col0=’2’, col1=’3’-->using “fmeexpression=@Evaluate(@Value(col0)+@Value(col1))” returns value “ 5” or “fmeexpression=@Evaluate(@Value(col0)@Value(col1))” returns value “ 23”Previously, using “@Evaluate([return “@Value(col0)@Value(col1)])” returns value “23”. I know 2023 introduced new functionality to SchemaMapper, per transformer help doc and https://support.safe.com/hc/en-us/articles/25407523665165-SchemaMapper-Advanced-Schema-Mapping.  Would appreciated if someone could test before submitting ticket. Output SchemaMapperDoc 

683
debbiatsafe
Safer
smithgk
Contributor
smithgkContributor
asked in Data
Overwriting an AGOL Feature Layer that has editing DISABLED

Hello! I am having some trouble overwriting a hosted feature layer on ArcGIS Online that is public and has editing disabled. I have a geojson file that is updated fairly regularly and I would like to create a workflow that reads the geojson and writes it to an AGOL feature service. I have that initial-publish workflow down, but after I log onto AGOL and change the feature service to public (sharing) and disable editing, subsequent runs of my FME tool fail and produce an error stating,  "Feature Service 'xxx_layer' does not support writer mode ''. Its owner may have disabled inserting or editing features. Writes may fail if the current user is not the owner or administrator". That error makes sense since I've disabled editing to the feature service, but I'm wondering if there is a way for FME to overwrite the feature service without me having to enable edits on the AGOL feature service. Because the data is public facing, I cannot allow edits to be enabl

2404
carmijo
Supporter
mgryko
Contributor
mgrykoContributor
asked in Data
USD Reader

Hi all,FME supports writing to USD file, but there is no option in the reader for USD.  How would I go about reading a USD file in FME? Thanks,Michael

332
mgryko
Contributor
bruceharold
Contributor
bruceharoldContributor
asked in Data
Partioned parquet writing - partition attribute order

HiQuick question before I fall back on trial and error, because my dataset is quite big, but does anyone know if changing the partitioning attribute order on a partitioned parquet writer determines the partition folder hierarchy?  The doc says its the schema order which determines the partition hierarchy, I’m hoping to override the reader’s attribute order.  I’m hoping manually editing the writer attribute order will determine partition hierarchy.  Thanks.

473
bruceharold
Contributor
redgeographics
Celebrity
redgeographicsCelebrity
asked in Data
Does FME support the new ArcGIS Online data types

Quick and easy question I hope: does FME 2024.2 support the new ArcGIS Online data types (Date Only, Time Only, Timestap Offset and Big Integer) as described here: https://www.esri.com/arcgis-blog/products/arcgis-online/mapping/time-is-on-your-side-with-new-field-types-in-arcgis-online/

896
jobvdnoort
Supporter
pkno
Contributor
pknoContributor
asked in Data
fme_feature_type missing (sometimes)

Hello everyone,I have a question regarding the fme_feature_type. I encountered a problem in my workspace regarding the SchemaMapper which (after working properly before) gave me the following error:SchemaMapper_Mapper (SchemaMappingFactory): A required column was not found in the table. Row fme_feature_type will be skipped. After doing some testing I realized the attribute fme_feature_type was <missing> across the board in this particular conversion which I believe is required for the SchemaMapper to work. I’m somewhat stumped because sometimes the workspace functions properly and the attribute has proper entries. I noticed especially with larger datasets that fme_feature_type just goes missing between transformers. Some culprints that I found were for example the SubstringExtractor and the AttributeKeeper. The data goes in with fme_feature_type properly filled and comes out with the data missing.I don’t think this is expected behaviour but maybe someone can shine some light on w

427
pkno
Contributor
sanshukla
Contributor
sanshuklaContributor
asked in Data
Latest Version Detail for upgradeable transformer

Hi All, I am trying to gather information on upgradeable transformers and see that current version is available as part of fmw file extension. I want to know where does it get the information on latest version, Is this information saved somewhere. In this example, AttributeManager is currently at ver 2 but latest version is ver 4. I am not sure where this version information stored. Thanks   

523
redgeographics
Celebrity
juliarozema
Contributor
juliarozemaContributor
asked in Data
ESRI Pro Annotation has different location when viewed in FME

Ultimate Goal: to use FME to test if there are ESRI annotations that fall on top of each other.  Issue being experienced: When the ESRI annotation is displayed in FME Inspector, the annotation has been shifted.  My theory is the “shift” is something to do with the annotation is left justified in ArcGIS Pro and its placement of the annotation is based on a point in the bottom left corner.  In FME the annotation point seems to be placed along the bottom centre.  Question: Is there a way to have FME place the ESRI Annotation based on the same placement point?ORHow can we shift the annotation when being viewed in FME so that it is placed half of the width of the annotations to the left along the annotation axes?   ArcGIS Pro: 3.2.3ArcGIS SDE: SQL 2022FME Form: FME(R) 2024.1.3.0 (20240927 - Build 24627 - WIN64)FME Annotation Reader Format: Esri Geodatabase (ArcSDE Geodb) See attached imagery for example of annotation shift.  

522
juliarozema
Contributor
lifalin2016
Contributor
lifalin2016Contributor
asked in Data
GeoDatabase writer works on desktop, but fails on server ?

Y’all.I have a workspace, that exports a number of database tables to a File GeoDatabase.The workspace uses a FeatureReader and a Writer with three dynamic featuretypes (for point, line and polygon features). The writer is “Esri Geodatabase (File Geodb Open API)”.When I run it in Desktop(Form) 2020.2, it works perfectly.When I run it as a job on Server(Flow) 2022.2, all features issues a warning, e.g.FileGDB Writer: A feature with feature type `HOERSHOLM_Afløbskomponent_point' could not be written I’ve tried running it in Desktop 2022.2, and this apparently doesn’t work either.I’ve tried to upload the workspace from here, but this still fails the same way.The output path is a UNC path to a ZIP file.Does the FileGDB writer have an issue with 2022.2 or any of the above details ?

242
lifalin2016
Contributor
kateaew
Contributor
kateaewContributor
asked in Data
ESRI Geodatabase domains - no ESRI licence

A client of ours is changing their spatial data format from MITAB to ESRI GDB. They use domains in their GDB and we don’t have an ESRI licence so am unable to read these in correctly with the domains exposed. I have requested the XML to go with it as per what I found in other threads.Is the best way to give it geometry to pop on a map a FeatureMerger with the XML and GDB, matching on an ID they have?I can’t help but think this is a poor way of doing it (it took forever as it’s a large dataset when I tested it), but without the ESRI licence I think I’m a bit stuck for options. Wondering if someone has a better solution.Using FME Form 2024.1Thanks!

713
kimo
Contributor
nea
Contributor
neaContributor
asked in Data
Coordinate system is not setting for Revit to GIS conversion

I’m converting 135 Revit models to GIS but the coordinate system is not setting (location is Philidelphia but data is appearing in Ohio).  I’ve tried setting the coordinate system (NAD 83, Pennsylvania state plane south, US foot) in the writer and using CoordinateSystemSetter.  Individual models appear properly in ArcGIS Pro.  I’m really baffled by this one; probably something simple I’m missing.

633
saraatsafe
Safer
benwu
Participant
benwuParticipant
asked in Data
Problems encountered using Cesium 3DTiles Writer

Excuse me, I am a master student. During one of my curriculum module, I used the Cesium 3D Tiles Writer in my project. And the result is not good: As we can see, some of the lines of point cloud is missing.And during the procedure of translation, I ahve warnings like this:What’s more, is that the result put all points inside the same pnts file:So when I try to visualize it on the website, it becomes stuck and process in a very low speed.May I ask how should I manage the parameters inside the Cesium 3DTiles Writer? If anyone could help me, I will appreciate it.

311
AliAtSafe
Safer
loicgueninrande
Contributor
loicgueninrandeContributor
asked in Data
Geoparquet Reader on FME Form 2024

Hi @Safe !I have a probleme with Geoparquet reader on FME Form 2024. It does’t work with the file in attachment but work fine with Apache parquet on FME Form 2023. The error is :Thanks for your help.Loïc

765
AliAtSafe
Safer
P
proudgis
asked in Data
How to set extents on DWG ouput file

I am writing AutoCAD DWG with changing areas/extents. My template file defines the styling. When opening the file in CAD it zooms to a blank area (possibly from the extents of the template file) and I need to zoom to extents to find my current area. Is it possible to either remove any extent information from the template file, or force the output DWG file to zoom in on the current area extents?

1366
scottchase
Participant
piercephi
Contributor
piercephiContributor
asked in Data
Cloned list between two sets of values based on a rank

I need to generate a list between two numbers and repeat a value the number of times between the two numbers. The values need to generate the number of rows between the higher ranked value and the lower ranked value. Below is the data I have now, and the second screenshot is what I need generated. Thank you for the help.What I have now:What I need: 

282
piercephi
Contributor
G
gisadamgis
asked in Data
Issues connecting with Portal !!!

Hello,I'm having issues trying to connect to the my Portal that is using OKTA - Auth2 to login.I've seen a fee threads on here regarding this issue but does not really explain clearly oon what to do.  I get this error "OAuth2 Approval Please copy this code, switch to your application and paste it there: " I followed the steps to register FME in Portal. Filled in all the correct info in the Manage Web Services. I put in my crednitals and get that error. I don't even know if it;s an error. I just dont know what to do at this point ? OAuth2 Approval Please copy this code, switch to your application and paste it there: "  I've seen a couple threads, but didn't really give an answer just said to submit a ticket.https://knowledge.safe.com/questions/109164/failing-to-connect-to-esri-arcgis-portal.html  https://knowledge.safe.com/questions/88927/how-do-i-reliably-connect-fme-server-201819-to-arc.html Can someone please help me figure this out . Thank you

785
salmayarista
Participant
g.bizournet
Contributor
g.bizournetContributor
asked in Data
Oracle Update sysdate problems

Hello,  Using a SQl executersyntax: update <table name>  set date_start= to_char(sysdate -1,'dd/mm/yyyy')        where  DATE_START is nullcommitNo updates of the tables.Does FME have a problem with using Oracle “to_char(sysdate -1,'dd/mm/yyyy')”?Because the syntax works under ORACLE SQLPLUS  

1038
g.bizournet
Contributor
marta.podsiad
Supporter
marta.podsiadSupporter
asked in Data
Create Hyperlinks in Excel

I am struggling to save text as a hyperlink in Excel. I tried using this formula:=HYPERLINK("http://www.1spatial.com/fme","http://www.1spatial.com/fme")However, the outcome in Excel didn’t work as expected.  Does anyone have any idea why it behaves this way?

501
david_r
Celebrity
mariaka
Contributor
mariakaContributor
asked in Data
3D object appears transparent

IssueI have several 3D objects (coming from IFC) with variable appearance styles. In one case the style seems transparent. Both in FME inspector and in the final writer (Cesium3DTiles) these objects are kind of transparent.(see feature informations in annex)the dark-red “wall” seems transparant by contrast with the two parts ontopWhat I did so farChanged/Reset appearance (AppearanceSetter) → no effect checked on Geometry (GeometryValidator, VertexNormalGenerator, GeometryRefiner, etc) → no effectQuestionsHow can I make the transparent objects opaque? What’s the difference to the other objects?

553
mariaka
Contributor
sworkman
Contributor
sworkmanContributor
asked in Data
Split list by keyword to individual lists.Or something??

Hello, I have a pdf report that I need to parse out. I can read it and create a line by line list. It has a line with employee code/name, then subsequent lines with codes and their values associated to the employee. Here is a sample of the report and then what it looks like when it is read in. I can get it split nicely into rows/columns/attributes no problem. My issue is that I dont know how to break it up in each employees portion and add the employee number to each row.There would be a few hundred employees. So in the end I want to be able to spit out a table that has the columns:Employee num, Code, Description, units, extensions,etc I am very new to FME and just struggling to get it figured out. I dont know if I should be using some kind of loop, python code, aggreator etc. Any help would be greatly appreciated!  

542
sworkman
Contributor
mmccart
Supporter
mmccartSupporter
asked in Data
How to Set Update Key Column on ArcGIS Online Writer 2017.1

I need to update an ArcGIS Online feature service using something other than 'OBJECTID' field. In previous versions, I was able to set the "Update Key Field" in the 'Properties' dialog box of the AGOL Writer. In 2017.1 that option is gone. I can't find any documentation or examples on how to set the Update Key Field to something else.Any suggestions?

1008
izabreton
Contributor
adnanyazdani
Contributor
adnanyazdaniContributor
asked in Data
Import data from Excel spreadsheet using file path provided in excel

HiI need guidance / SupportI have an excel where FiletypeName: .MDBPath of the MDB is mentionedWorkspace should read the excel and go to tha path mentioned in the excel and read the mdb for each and it should individual for each   Workspace should go to that path and read the MDB file in which Different FC are mentioned and then write it back with same Name of MDB and Fc in GDB formatKindly Guide me how can i proceedI have attached sample data with excel and MDB

1525
evieatsafe
Safer
nordpil
Supporter
nordpilSupporter
asked in Data
Reading/Writing SDE Archived featureclasses

I have MSSQL ESRI Entrerprise Geodatabases that are non-versioned but have archiving enabled. As far as I have figured out, there are no ESRI tools that easily copy data with the full archive of the feature class.If it was versioned I could use the ESRI tools for replication…What is the current best methods for an operation like this?

562
nordpil
Supporter

Badge Winners

Show all badges

Community Stats

31,334
Posts
118,864
Replies
38,957
Members

Latest FME

Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.