Skip to main content

Data

Using readers and writers to extract and load data.

  • 6,970 Posts
  • 25,684 Replies
6970 Posts
R
raghunaren
asked in Data
Does writing to a MapInfo Extended TAB format take longer than to a MapInfo TAB ?

Does writing to a MapInfo Extended TAB format take longer than to a MapInfo TAB ? Just wanted to check with the community if this is a known/expected behaviour with writing data to MapInfo Extended TAB and also if there are workarounds to quicken the writing process. Although I have experienced FME 2017 take considerably longer time to write a MapInfo Extended TAB for polygon data that is about 1.2 million in count, I failed to take note of the actual time it took, from the log. I do have a stat for a much smaller dataset, that of 1500 polygons and it took about 14 minutes to write to a MapInfo Extended TAB

406
sigher
Contributor
Jonathan Slope
Contributor
Jonathan SlopeContributor
asked in Data
Create centerline on highways and intersections

Hi! I have a new question today. I want to create centerlines on highways. Now, long lines, without intersections, work fine. However, i get wrong results when fme encounters a intersection, like in the pictures below. I expected this and tried to solve it, but it only became worse. Does anyone know a right way to do it? Thanks in advance!

524
s.jager
Influencer
A
andrewflaws
asked in Data
Microsoft SQL server Spatial reader failing

Microsoft SQL Server Spatial Reader now fails to connect post upgrade to 2020.1. It worked with 2020.0. We are getting this error: Microsoft SQL Server Spatial Reader: Failed to connect using Microsoft OLE DB Driver for SQL Server. Provider error '(-2146824582) Provider cannot be found. It may not be properly installed.'. - is there something we need to install?

6657
braggken
Supporter
michielschram
michielschram
asked in Data
Can FME write to an Excel table?

I have got some beautiful Excel templates containing tables (not normal ranges). It's quite easy to write to a named range within Excel but I haven't figured out how to write to a Table. Does anyone know how to do so?

1675
ekkischeffler
Contributor
denverinpagosa
Contributor
denverinpagosaContributor
asked in Data
Join excel file to geodatabase to match addresses

I have an excel file that has a list of addresses, and their associated license number. I want to join this to a geodatabase file to output any parcels that have any of the addresses in the excel file. I can’t seem to make that work. Can anybody point me in the right direction? I’ve tried FeatureJoiner and FeatureMerger and can’t seem to get any output without failure.

392
denverinpagosa
Contributor
denverinpagosa
Contributor
denverinpagosaContributor
asked in Data
Can't add Excel reader

I’m using the Data Interoperability extension in ArcGIS Pro. I can’t seem to get a excel reader to add in. I’ll set the parameters, it’s not open anywhere else, it’s not password protected or anything like that, and it’s on my local system. When I click ok to add it, fme acts like it’s trying to add it, but then just sits there and spins until it eventually times out. What am I doing wrong? I haven’t used FME in about 7 years or so, but I used it extensively prior to that and never had this sort of issue.

372
denverinpagosa
Contributor
sar_copp
Contributor
sar_coppContributor
asked in Data
Help with creating a nested JSON from an Excel table

Hello, Is it possible to transform a table (ie from Postgresql, or Excel) into this complicated Json format within FME please? I have attached the json file and also a small dummy sample excel file (which doesnt quite have the correct column headers as its a work in progress).I am completely new to JSON and I have done the JSON Templater tutorial but no idea how to actually apply it to my needs!! Help wanted!!

612
sar_copp
Contributor
marta.podsiad
Supporter
marta.podsiadSupporter
asked in Data
Read Share Group information from Esri Enterpise

Hi, I am having trouble reading the names of the groups with which the layer is shared in Esri Enterprise. I am using an HTTPCaller with the following settings, but the Share info does not appear in the JSON response. The only information I get is: "access": "shared". Could you please advise on how to retrieve this information?

355
marta.podsiad
Supporter
alexlynch3450
Contributor
alexlynch3450Contributor
asked in Data
SchemaMapper Advance Mapping Create Whitespace before evaluating data value

Hello All, I’m having an issue where SchemaMapper prefix whitespace (space) when using fmeexpression=. I attached a sample workspace and would appreciate if anyone could test before submitting ticket. This issue came apparent after upgrading to FME 2023. for example: col0=’2’, col1=’3’-->using “fmeexpression=@Evaluate(@Value(col0)+@Value(col1))” returns value “ 5” or “fmeexpression=@Evaluate(@Value(col0)@Value(col1))” returns value “ 23” Previously, using “@Evaluate([return “@Value(col0)@Value(col1)])” returns value “23”. I know 2023 introduced new functionality to SchemaMapper, per transformer help doc and https://support.safe.com/hc/en-us/articles/25407523665165-SchemaMapper-Advanced-Schema-Mapping. Would appreciated if someone could test before submitting ticket. Output SchemaMapperDoc

653
debbiatsafe
Safer
smithgk
Contributor
smithgkContributor
asked in Data
Overwriting an AGOL Feature Layer that has editing DISABLED

Hello! I am having some trouble overwriting a hosted feature layer on ArcGIS Online that is public and has editing disabled. I have a geojson file that is updated fairly regularly and I would like to create a workflow that reads the geojson and writes it to an AGOL feature service. I have that initial-publish workflow down, but after I log onto AGOL and change the feature service to public (sharing) and disable editing, subsequent runs of my FME tool fail and produce an error stating, "Feature Service 'xxx_layer' does not support writer mode ''. Its owner may have disabled inserting or editing features. Writes may fail if the current user is not the owner or administrator". That error makes sense since I've disabled editing to the feature service, but I'm wondering if there is a way for FME to overwrite the feature service without me having to enable edits on the AGOL feature service. Because the data is public facing, I cannot allow edits to be enabled. Any help is greatly appreciated

2214
carmijo
Supporter
mgryko
Contributor
mgrykoContributor
asked in Data
USD Reader

Hi all, FME supports writing to USD file, but there is no option in the reader for USD. How would I go about reading a USD file in FME? Thanks, Michael

332
mgryko
Contributor
bruceharold
Contributor
bruceharoldContributor
asked in Data
Partioned parquet writing - partition attribute order

Hi Quick question before I fall back on trial and error, because my dataset is quite big, but does anyone know if changing the partitioning attribute order on a partitioned parquet writer determines the partition folder hierarchy? The doc says its the schema order which determines the partition hierarchy, I’m hoping to override the reader’s attribute order. I’m hoping manually editing the writer attribute order will determine partition hierarchy. Thanks.

473
bruceharold
Contributor
redgeographics
Celebrity
redgeographicsCelebrity
asked in Data
Does FME support the new ArcGIS Online data types

Quick and easy question I hope: does FME 2024.2 support the new ArcGIS Online data types (Date Only, Time Only, Timestap Offset and Big Integer) as described here: https://www.esri.com/arcgis-blog/products/arcgis-online/mapping/time-is-on-your-side-with-new-field-types-in-arcgis-online/

876
jobvdnoort
Supporter
pkno
Contributor
pknoContributor
asked in Data
fme_feature_type missing (sometimes)

Hello everyone, I have a question regarding the fme_feature_type. I encountered a problem in my workspace regarding the SchemaMapper which (after working properly before) gave me the following error: SchemaMapper_Mapper (SchemaMappingFactory): A required column was not found in the table. Row fme_feature_type will be skipped. After doing some testing I realized the attribute fme_feature_type was <missing> across the board in this particular conversion which I believe is required for the SchemaMapper to work. I’m somewhat stumped because sometimes the workspace functions properly and the attribute has proper entries. I noticed especially with larger datasets that fme_feature_type just goes missing between transformers. Some culprints that I found were for example the SubstringExtractor and the AttributeKeeper. The data goes in with fme_feature_type properly filled and comes out with the data missing. I don’t think this is expected behaviour but maybe someone can shine some light on what

417
pkno
Contributor
sanshukla
Contributor
sanshuklaContributor
asked in Data
Latest Version Detail for upgradeable transformer

Hi All, I am trying to gather information on upgradeable transformers and see that current version is available as part of fmw file extension. I want to know where does it get the information on latest version, Is this information saved somewhere. In this example, AttributeManager is currently at ver 2 but latest version is ver 4. I am not sure where this version information stored. Thanks

523
redgeographics
Celebrity
juliarozema
Contributor
juliarozemaContributor
asked in Data
ESRI Pro Annotation has different location when viewed in FME

Ultimate Goal: to use FME to test if there are ESRI annotations that fall on top of each other. Issue being experienced: When the ESRI annotation is displayed in FME Inspector, the annotation has been shifted. My theory is the “shift” is something to do with the annotation is left justified in ArcGIS Pro and its placement of the annotation is based on a point in the bottom left corner. In FME the annotation point seems to be placed along the bottom centre. Question: Is there a way to have FME place the ESRI Annotation based on the same placement point?ORHow can we shift the annotation when being viewed in FME so that it is placed half of the width of the annotations to the left along the annotation axes? ArcGIS Pro: 3.2.3ArcGIS SDE: SQL 2022FME Form: FME(R) 2024.1.3.0 (20240927 - Build 24627 - WIN64)FME Annotation Reader Format: Esri Geodatabase (ArcSDE Geodb) See attached imagery for example of annotation shift.

442
juliarozema
Contributor
lifalin2016
Contributor
lifalin2016Contributor
asked in Data
GeoDatabase writer works on desktop, but fails on server ?

Y’all. I have a workspace, that exports a number of database tables to a File GeoDatabase. The workspace uses a FeatureReader and a Writer with three dynamic featuretypes (for point, line and polygon features). The writer is “Esri Geodatabase (File Geodb Open API)”. When I run it in Desktop(Form) 2020.2, it works perfectly.When I run it as a job on Server(Flow) 2022.2, all features issues a warning, e.g. FileGDB Writer: A feature with feature type `HOERSHOLM_Afløbskomponent_point' could not be written I’ve tried running it in Desktop 2022.2, and this apparently doesn’t work either.I’ve tried to upload the workspace from here, but this still fails the same way. The output path is a UNC path to a ZIP file.Does the FileGDB writer have an issue with 2022.2 or any of the above details ?

242
lifalin2016
Contributor
kateaew
Contributor
kateaewContributor
asked in Data
ESRI Geodatabase domains - no ESRI licence

A client of ours is changing their spatial data format from MITAB to ESRI GDB. They use domains in their GDB and we don’t have an ESRI licence so am unable to read these in correctly with the domains exposed. I have requested the XML to go with it as per what I found in other threads. Is the best way to give it geometry to pop on a map a FeatureMerger with the XML and GDB, matching on an ID they have? I can’t help but think this is a poor way of doing it (it took forever as it’s a large dataset when I tested it), but without the ESRI licence I think I’m a bit stuck for options. Wondering if someone has a better solution. Using FME Form 2024.1 Thanks!

713
kimo
Contributor
nea
Contributor
neaContributor
asked in Data
Coordinate system is not setting for Revit to GIS conversion

I’m converting 135 Revit models to GIS but the coordinate system is not setting (location is Philidelphia but data is appearing in Ohio). I’ve tried setting the coordinate system (NAD 83, Pennsylvania state plane south, US foot) in the writer and using CoordinateSystemSetter. Individual models appear properly in ArcGIS Pro. I’m really baffled by this one; probably something simple I’m missing.

583
saraatsafe
Safer
benwu
Participant
benwuParticipant
asked in Data
Problems encountered using Cesium 3DTiles Writer

Excuse me, I am a master student. During one of my curriculum module, I used the Cesium 3D Tiles Writer in my project. And the result is not good: As we can see, some of the lines of point cloud is missing. And during the procedure of translation, I ahve warnings like this: What’s more, is that the result put all points inside the same pnts file: So when I try to visualize it on the website, it becomes stuck and process in a very low speed. May I ask how should I manage the parameters inside the Cesium 3DTiles Writer? If anyone could help me, I will appreciate it.

301
AliAtSafe
Safer
loicgueninrande
Contributor
loicgueninrandeContributor
asked in Data
Geoparquet Reader on FME Form 2024

Hi @Safe ! I have a probleme with Geoparquet reader on FME Form 2024. It does’t work with the file in attachment but work fine with Apache parquet on FME Form 2023. The error is : Thanks for your help. Loïc

655
AliAtSafe
Safer
P
proudgis
asked in Data
How to set extents on DWG ouput file

I am writing AutoCAD DWG with changing areas/extents. My template file defines the styling. When opening the file in CAD it zooms to a blank area (possibly from the extents of the template file) and I need to zoom to extents to find my current area. Is it possible to either remove any extent information from the template file, or force the output DWG file to zoom in on the current area extents?

1286
scottchase
Participant
piercephi
Contributor
piercephiContributor
asked in Data
Cloned list between two sets of values based on a rank

I need to generate a list between two numbers and repeat a value the number of times between the two numbers. The values need to generate the number of rows between the higher ranked value and the lower ranked value. Below is the data I have now, and the second screenshot is what I need generated. Thank you for the help. What I have now: What I need:

272
piercephi
Contributor
G
gisadamgis
asked in Data
Issues connecting with Portal !!!

Hello, I'm having issues trying to connect to the my Portal that is using OKTA - Auth2 to login. I've seen a fee threads on here regarding this issue but does not really explain clearly oon what to do. I get this error "OAuth2 Approval Please copy this code, switch to your application and paste it there: " I followed the steps to register FME in Portal. Filled in all the correct info in the Manage Web Services. I put in my crednitals and get that error. I don't even know if it;s an error. I just dont know what to do at this point ? OAuth2 Approval Please copy this code, switch to your application and paste it there: " I've seen a couple threads, but didn't really give an answer just said to submit a ticket. https://knowledge.safe.com/questions/109164/failing-to-connect-to-esri-arcgis-portal.html https://knowledge.safe.com/questions/88927/how-do-i-reliably-connect-fme-server-201819-to-arc.html Can someone please help me figure this out . Thank you

745
salmayarista
Participant
g.bizournet
Contributor
g.bizournetContributor
asked in Data
Oracle Update sysdate problems

Hello, Using a SQl executer syntax: update <table name> set date_start= to_char(sysdate -1,'dd/mm/yyyy') where DATE_START is nullcommit No updates of the tables. Does FME have a problem with using Oracle “to_char(sysdate -1,'dd/mm/yyyy')”? Because the syntax works under ORACLE SQLPLUS

888
g.bizournet
Contributor

Helpful Members This Week

Badge Winners

Show all badges

Popular Tags

Community Stats

31,142
Posts
118,068
Replies
38,761
Members

Latest FME

Terms & ConditionsCookie settings

Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.