Skip to main content
  • EN

Data

Using readers and writers to extract and load data.

  • 7,092 Posts
  • 26,104 Replies
7092 Posts
caroline_pnw
Contributor
caroline_pnwContributor
asked in Data
Updating a Feature Service through ArcGIS Portal - Using Dynamic Schema

I am trying to see if I can update a feature layer in our ESRI Portal from a FGDB. We know we can insert, update, and delete features and have success on those fronts. However, we will like to also have dynamic schema available as we may change the data in our FGDB by removing/adding fields. I have tried several times to get a dynamic writer working to account for a schema change but without luck. I have tried to delete data in the feature layer to then insert but that had errors. Does anyone have advice or has done this? The FGDB is a testing stage as eventually we would want to update directly from SDE. 

4577
philippeb
Enthusiast
youssef
Contributor
youssefContributor
asked in Data
How do i write Geometry to AIXM RunwayTimeSlice?

Hey guys, i really struggle to write my geometry to the RunwayTimeSlice Element of the AIXM 5 writer of FME. After the GeometryExtractor i got a string looking like this:  <gml:Surface srsName="EPSG:31467" srsDimension="2"><gml:patches><gml:PolygonPatch><gml:exterior><gml:LinearRing><gml:posList>5544321.76358795 3466140.39319303 5544321.79681692 3466111.70099084 5544321.79831432 3466110.40802792 5544321.8330376 3466080.42553198 5544021.7716845 3466080.40979097 5543896.905 3466084.247 5542896.762 3466080.481 5540345.41046032 3466080.53414733 5540345.41261875 3466140.53829729 5542896.8 3466140.46 5543901.789 3466140.512 5544021.76302164 3466140.39707783 5544321.76358795 3466140.39319303</gml:posList></gml:LinearRing></gml:exterior></gml:PolygonPatch></gml:patches></gml:Surface> which should be the right format for the gml geometry. But when i try to write it in the surfaceProperties Attribute of the writer nothing ha

1687
youssef
Contributor
sigbono
Contributor
sigbonoContributor
asked in Data
ArcGIS Online Feature Service Reader Error Code '400' Deatils '24214'

We published a feature layer in ArcGIS Online and when we try to read the features in FME Form 2024.1 (20240628 - Build 24612 - WIN64), we get this error. ArcGIS Online Feature Service Reader: Encountered an unexpected error. The error code from the server was '400' and the message was: 'Cannot perform query. Invalid query parameters.'. Details: '24214: Circular arc segments with Z values must have equal Z value for all 3 points., Unable to perform query. Please check your parameters.'Python Exception <ArcGISServerError>: FMEException: 928221: ArcGIS Online Feature Service Reader: Encountered an unexpected error. The error code from the server was '400' and the message was: 'Cannot perform query. Invalid query parameters.'. Details: '24214: Circular arc segments with Z values must have equal Z value for all 3 points., Unable to perform query. Please check your parameters.'We have published two more feature layers and it all works fine. Does this have something to do with our data

6492
spatialexjames
Contributor
mr_fme
Enthusiast
mr_fmeEnthusiast
asked in Data
Filter in the Feature Reader parameters

Hi,I’m using the Feature Reader to read Sentinel images.I would like to filter only the files (in a folder) that contain "B08" as part of the filename. There can be many files with "B08" in some part of the name.How can I do that? Thank´s  

421
hkingsbury
Celebrity
leestamm
Contributor
leestammContributor
asked in Data
BPF format

Hi can FME deal with BPF files (point cloud data storage), or is it planning on supporting this..

704
leestamm
Contributor
paulmcm601
Participant
paulmcm601Participant
asked in Data
Symbology issue when creating shapefile - ListExploder/Aggregator issue?

[Software FME Workbench 2021.2.3]Hi,Trying to generate an Esri Shapefile Polygon output of areas with data merged from multiple datasets to the Area dataset.The addition of Location Names (at least one per area) seems to be throwing up an issue.After doing the steps below the Transformer Views and Shapefile Symbology changes and goes a bit weird, and I’m not exactly sure how to sort it.Below polygon symbology is one area, but there are probably 30% of 1000 Polygons that have similar symbology issues.Any help would be greatly appreciated - Many thanks.Step One:Feature MergerRequestor: Area DatasetSupplier: Location Name DatasetMerge on AreaIDGenerate List:List Name: _listAdd to List: Selected AttributesSelected Attributes: Location_Name Feature Merger: Merged OutputStep Two:List ExploderList Attribute: _listAccumulation Mode: Merge List AttributesConflict Resolution: Use List Attribute Values List Exploder: Elements OutputStep Three:Aggregator to update the Location_Name to have multipl

472
paulmcm601
Participant
markrmunoz
Contributor
markrmunozContributor
asked in Data
Writing date time from an xlsx

I'm trying to overwrite a date/time field with two columns from an excel file. I'd like to use attribute manager to concatenate the two columns into (Date) MM/dd/yyyy and (Time) HH:mm:ss AM/PM however I'm getting null values. The datetimeconverter led to an error saying my format wasn't FME standard. Any help is appreciated, thank you.

751
bwn
Evangelist
nbeekers
Contributor
nbeekersContributor
asked in Data
Group records in range of Values and time

Hello,  Im searching for a solution to give records a group value based on multiple values statements. Data needs to be grouped on a Vehicle in time asc.So far this is not the problem.The grouping also needs to Start with a value Status = "Start”and all the other records after Start.The grouping needs to be closed after Status End comes as a record.It looks like a range or a ranking. If someone has some idea thanks. Kind regards, Nick 

423
nbeekers
Contributor
iberzins
Observer
iberzinsObserver
asked in Data
Reflectors in AMDB

Hello!I am importing data from Aerodrome data (gdb) to AMDB. Aerodrome has reflectors on the edges of taxyway. What should these reflectors be stored as in the AMDB?Maybe AerodromeSurfaceLighting or some kind of Marking or AerodromeSign or something else? The Reflectors 

160
iberzins
Observer
chris.schulz
Contributor
chris.schulzContributor
asked in Data
CouchDB deprecation

Hi, Ive noted that CouchDB has been deprecated. Of course, this is the exact thing I need right now :( Is there any alternative way to access this database using the latest (2024) version of FME? 

923
bwn
Evangelist
niksands
Contributor
niksandsContributor
asked in Data
AGOL Feature Service Writer resets visibility when using 'Truncate Existing'

I’m using the ESRI ArcGIS Online Feature Service Writer to update a layer in AGOL.  I have it configured as per the screenshot below.After it runs, the layer in AGOL is visible when zoomed all the way out!So, I then republish the layer in ArcGIS Pro and it fixes the layer so that it now disappears when zoomed out (I could also do this directly in AGOL).I then change the FME writer to ‘Use Existing’ (and not ‘Truncate Existing’) and re-run the translation.This time it works without affecting the AGOL layer visibility.So I change the FME writer back to ‘Truncate Existing’ (and not ‘Use Existing’) and re-run the translation.It breaks the AGOL layer visibility again!I plan to use something like this to update many layers, and I cannot have them all visible when zoomed out.  It would be a complete mess.Am I doing something wrong? How can I update the data (including deleting features) without affecting the AGOL layer’s visibility?(The issue occurs for both 'Insert' and 'Upsert'.  I’m using

891
niksands
Contributor
philippeb
Enthusiast
philippebEnthusiast
asked in Data
Create new field to ArcGIS Portal Feature

Hello,Is it possible to write new fields in ArcGIS Portal Feature Service with a writer ?I’m getting that message : ArcGIS Portal Feature Service Writer: Attribute 'NAME' on feature type 'Roads' does not match any field on the corresponding layer/table. The attribute will be ignored I can’t believe I have to create those fields with ArcGIS Pro, publish the layer and than run my workbench. Update:OK I just found out that I need my fields to be in lowercase to make it work.But for some reasons some field type are not working.Is there any existing list of all the constraints we need to follow for the fields in Portal? That would save me time! Thanks!

892
philippeb
Enthusiast
emerson_styles
Contributor
emerson_stylesContributor
asked in Data
Strange characters appearing instead of date

I have a script that pulls numerous datasets together, formats them into a consistent set of attributes and then writes them to .TAB files.I’m having difficulty with date field (STARTD) in the final output because although appearing to be correct in the .TAB files, ArcPy is not able to detect the dates in the .TAB files and instead shows the data as odd characters. Please see the image below for more information. Has anyone seen this before or aware of how to fix the issue? I’m not sure if there is a setting in my FME script that may need to be amended. 

201
redgeographics
Celebrity
femo
Contributor
femoContributor
asked in Data
Reading Excel files with different columns

Hello,i need to process excel files, currently by means of FME Form 2023.1, perspectively using FME Flow. These files, however, can differ in columns available. See attachments. The files are processed individually, only one file at a time.If i run the process with file “tk_template_v0.0.xlsx” I want the featurereader to display only the columns, containing in this file. If I run “tk_template_v0.1.xlsx”, which has got 3 more columns, i want these columns to be available in the process as well.I would like to run the script with these files without clicking on the “Feature Types to Read” button any time I process a new file! The sheet name always have the name “Tabelle1”.Unfortunately  I do not find a set of parameter that results in the desired output.The existing posts have not yet enlighten me! Any ideas?    

1895
bwn
Evangelist
pehrs
Contributor
pehrsContributor
asked in Data
IFMEFeature with aggregated/multiple geometry

We have a reader plugin (c++) that should be able to set aggregated/ multiple geometry to an IFMEFeature.For example:We have only several IFMEGeometry objects. These IFMEGeometry objects should be set to an IFMEFeature. So that we have finally an IFMEFeature that consists of several different geometry objects/ parts.                                                         IFMEFeatureIFMEGeometry_Part_1    IFMEGeometry_Part_2    ….      IFMEGeometry_Part_nfme_type = fme_areafme_geometry = fme_aggregatefme_feature_type = DepthArea How can we solve this? 

594
pehrs
Contributor
philippeb
Enthusiast
philippebEnthusiast
asked in Data
Read multiple Excel file with different starting line

Hi,I have thousand of Excel files I need to read but all of them have different starting attributes line. Sometimes line 15 or 16 or 17.(not me who developed that method of working….)The attributes are all the same for all the files.Any idea how I could have the Excel Reader to detect the starting line?Thanks !

683
philippeb
Enthusiast
albjoh
Contributor
albjohContributor
asked in Data
Colorize polygons based on attribute value for writing/visualizing HTML

Hi, I’m looknig for a way to colorize polygons based on attribute value for writing to HTML and further use in data streaming service with workspace app. Each polygon have attributes with unique name and color.I want to apply these colors to based on the values in the “name” attribute. I know it’s possible to replace text in the “html_content” using the string replacer but how can I make use of my existing attributes when adding eg. a javaScript function? I guess I need to populate the feature properties dynamically from name attribute:However my edits seem to be omitted when written to final html…Any examples/ideas on this

480
albjoh
Contributor
scarter
Contributor
scarterContributor
asked in Data
Issues converting tab to geojson due to NativeX in file

Hello Forum, I need to convert a tab file to geojson, which is ordinarily a simple task. However I am getting this error message…“MITAB Reader: Unsupported table type 'NativeX' in file #######################.  This type of .TAB file cannot be read by this library.”Note that the # symbols are masking the UNC path and file name.Does anyone know a way around this? Thanks,Stuart 

363
scarter
Contributor
fmesafe.podpora
Contributor
fmesafe.podporaContributor
asked in Data
GeoJSON entry according to the defined schema

Hey. Hey. I have a problem with GeoJson. GeoJson automatically writes the geometry and I would need to write an extra "layer" block to this geometry block, see attachment layer. I tried to create a template on json templater see fmwt, but it doesn't work. The idea of this template was that I would define the structure as json including the "layer" block and then write it as text with all parameters as geoJson, but I don't know if this is a good way to go. Would you recommend me any other options? I also have the schemas available, but I'm not sure on which transformer and in what combination with other transformers to use them to achieve the result. Thank you very much

1538
fmesafe.podpora
Contributor
P
phallot
asked in Data
SHP with ZM dimensions to Postgis linestringZ

Hi all,  I did not found the way to remove the M dimension from a shapefile to include it into a postgis geometry defined as linestringZ.  Do you know the transforer to be used ?  Thanks,  Pierre 

335
evieatsafe
Safer
R
robkromwijk
asked in Data
Fitview when writing to microstation dgn

I am using FME to write to microstation DGN. The FME workbench file is using the fanout option to write different areas to the microstation DGN file, based on the same seed file. At the moment I have to use a microstation macro to "fitview" of each produced file in order to have the microstation file centered at each specific area. I rather use only FME to avoid this additonal step.Is there a way / transformer in FME to save the microstation file centered in the written data?

455
emma.at.safe
Safer
nicholas
Contributor
nicholasContributor
asked in Data
Can Writer Esri Geodatabase (File Geodb Open API) write metadata?

Hello all,I have succeeded in building a workspace that Reads from Oracle and Writes a fileGDB that includes metadata for the feature layer in the fileGDB. I used an XMLTemplater to achieve this.I am successful when I use the Writer Esri Geodatabase (File Geodb)However, I would prefer to use the Writer Esri Geodatabase (File Geodb Open API)But when I substitute this Writer, it fails to include the metadata for the feature layer.Is there some setting that I am missing?Or is this functionality just missing from the Open API Writer?

371
emma.at.safe
Safer
gbd
Contributor
gbdContributor
asked in Data
Neighborfinder 3d, calculate 3d distance.

I´m in need to calculate the shortest distance in 3d between a set of basepoints and a line.Something similar to a Neighborfinder in 3d mode.Do any of you know a way to do this? 

1524
gbd
Contributor
ddecoene
Contributor
ddecoeneContributor
asked in Data
Reading API into attributes

Hello, i have an api which i need to read on a daily basis.I use an httpcaller, this is what i get.{   "values" : [      {         "id" : "66d9f33fa4766fe73ac59fb9",         "start_time" : "2024-09-05T20:06:55+02:00",         "end_time" : "2024-09-05T20:07:54+02:00",         "vehicle" : {            "id" : "xxxxx",            "country_code" : "BE",            "id_type" : "licenseplate"         },         "price" : {            "amount" : 0,            "vat" : 0         },         "right_type" : {            "code" : "PAID_PARKING"         },         "area" : {            "code" : "xxxx",            "area_manager_id" : "611e58c434e393575bd3a064"         },         "registration_time" : "2024-09-05T20:06:55.593+02:00",         "registered_by_organization" : {            "code" : "xxxxx",            "name" : "xxxxx"         },         "external_user_reference" : "31358053",         "channel" : {            "medium" : "api"         },         "tariff" : {            "id" : "653ba7cd9c4bdbc

512
ebygomm
Influencer
andybarker
Participant
andybarkerParticipant
asked in Data
Unable to select Google BigQuery as a writer

I’m working through a test case to pull a table data from one of our MS SQL databases into Google BigQuery. I’ve downloaded and installed the BigQuery package from FME Hub and setup the needed Clientid and secret to have it connect to our Google Cloud project. It tests successfully in the services setup, and if I use the GoogleBigQueryConnector, I’m able to select our cloud project and dataset for reading, so I know the connectivity part of it is working. The issue is trying to write something to BigQuery.If I go to create a writer and choose “Google BigQuery” as the format, the Format selection stays red, like it’s an invalid selection. What am I missing?Unable to select Google BigQuery as a Writer format 

462
andybarker
Participant

Badge Winners

Show all badges

Community Stats

31,807
Posts
120,824
Replies
39,436
Members

Latest FME

Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.