- - Community Resources
Ideas
Product Updates
Events
Blog

Home
Forums
FME Form
Data

Data

Using readers and writers to extract and load data.

6,784 Posts
24,952 Replies

When you subscribe we will email you when there is a new topic in this category

6784 Posts

Recently active

Most replies Most views Newest first

mr_fmeEnthusiast

asked in Data

Extract pixel

Hi,I need to extract pixel with value 3 ,8 and 9. Have some transformer that filter the pixel value? Thank´s

20225

1 month ago

dustinInfluencer

asked in Data

Help with understading OSM donuts

I could use some help in understanding how donuts are stored in OSM XML. Or more specifically, why FME Data Inspector is not constructing the donut from the XML. Here is the simple shapefile test data I’ve created.In FME, I’m writing out the OSM using the normal writer:When I read that OSM back into Data Inspector, it doesn’t recognize the shape as a multipolygon. The geometry type is fme_no_geom:The XML code is below. I’m seeing the relation present, so I’m not sure what else is required.<?xml version="1.0" encoding="UTF-8"?><osm version="0.6"> <node lat="34.6596384" lon="-87.0320968" id="-7"/> <node lat="34.5631538" lon="-86.8849822" id="-10"/> <node lat="34.7427899" lon="-86.9769288" id="-8"/> <node lat="34.4741373" lon="-86.9217608" id="-4"/> <node lat="34.6293808" lon="-86.7401662" id="-2"/> <node lat="34.5063457" lon="-87.0619795" id="-5"/> <node lat="34.5536885" lon="-86.9194622" id="-16"/> <node lat="34.5669396" lon="-87.

771

1 month ago

smithgkContributor

asked in Data

Converting Overpass Turbo (OSM) data from xml

Hello,There have been some previous posts about converting OSM data in FME, but I’m hoping to breath some new life into the conversation. I have an Overpass Turbo query here meant to query ski area boundaries, lifts, and runs within Colorado. The query works just fine in Overpass Turbo, and I’m able to read the raw XML data into FME with the HTTP Caller no problem. The issue I’m running into is visualizing the data beyond that. I’m aware of the example workflow that is similar to my process, but I don’t understand how the XSLT Processor and XML Fragmenter transformers work. If I use those transformers in my workflow, all the coordinate information seems to disappear when the data is fragmented.Does anyone know how to properly display the XML data in FME? I might need a geometry filter to split out points, lines, and polygons, but I don’t think that’s what is holding me back currently. Any help would be greatly appreciated!

Apply Python shutdown script on file before FME Flow Download service

Hi,I’m trying to modify the output file of a workspace using a Python shutdown script before it’s caught by the download service (or after, but before downloading). Is there any way to do that?The configuration is:workspace creating an output text file (interlis) workspace added to a Flow app with a download serviceI’ve tried to use the destination dataset parameter from the writer but without success. The script is :import fmedest_dataset = fme.macroValues['DestDataset_Interlis']with open(dest_dataset, 'r') as file: filedata = file.read()filedata = filedata.replace(',', '.')with open(dest_dataset, 'w') as file: file.write(filedata)Or if you have any clue on how to force an Interlis writer to use dots instead of commas as separator without changing the server locale, I’ll take that info as well. FME versions:Flow 2023.1.1.1 Form 2023.1.1.1 Best regards,Julien

161

1 month ago

scarecrowContributor

asked in Data

Raster renaming in Esri SDE

Hi All,I’m creating an archiving workbench for our Esri Enterprise SDE databases. I’m trying to automate checking for an existing version of a raster in either our shared or archived databases and then assigning a version suffix according to if the data already exists in one of the databases. I have the majority of the workbench sorted but struggling to automate renaming the raster to include the version suffix when writing to archive.In my example below I’ve identified that there are two existing versions of the test data “gen_aaa_aaa_test_ras” so I’ve created a name field “archhiveddataname”, which contains the the name I want to give the raster when I write it to Archived “gen_aaa_aaa_test_ras_2”. I can access this information from the schema port but not from the generic. If I include the information from the schema port I get an error saying that I’m trying to write non-raster geometry and the workbench fails. I’m using FME 2023.1.2.0 (20231110 - Build 23641 - WIN64) Thanks in adv

191

1 month ago

caroline_pnwContributor

asked in Data

Updating a Feature Service through ArcGIS Portal - Using Dynamic Schema

I am trying to see if I can update a feature layer in our ESRI Portal from a FGDB. We know we can insert, update, and delete features and have success on those fronts. However, we will like to also have dynamic schema available as we may change the data in our FGDB by removing/adding fields. I have tried several times to get a dynamic writer working to account for a schema change but without luck. I have tried to delete data in the feature layer to then insert but that had errors. Does anyone have advice or has done this? The FGDB is a testing stage as eventually we would want to update directly from SDE.

1627

1 month ago

youssefContributor

asked in Data

How do i write Geometry to AIXM RunwayTimeSlice?

Hey guys, i really struggle to write my geometry to the RunwayTimeSlice Element of the AIXM 5 writer of FME. After the GeometryExtractor i got a string looking like this: <gml:Surface srsName="EPSG:31467" srsDimension="2"><gml:patches><gml:PolygonPatch><gml:exterior><gml:LinearRing><gml:posList>5544321.76358795 3466140.39319303 5544321.79681692 3466111.70099084 5544321.79831432 3466110.40802792 5544321.8330376 3466080.42553198 5544021.7716845 3466080.40979097 5543896.905 3466084.247 5542896.762 3466080.481 5540345.41046032 3466080.53414733 5540345.41261875 3466140.53829729 5542896.8 3466140.46 5543901.789 3466140.512 5544021.76302164 3466140.39707783 5544321.76358795 3466140.39319303</gml:posList></gml:LinearRing></gml:exterior></gml:PolygonPatch></gml:patches></gml:Surface> which should be the right format for the gml geometry. But when i try to write it in the surfaceProperties Attribute of the writer nothing ha

1557

1 month ago

sigbonoContributor

asked in Data

ArcGIS Online Feature Service Reader Error Code '400' Deatils '24214'

We published a feature layer in ArcGIS Online and when we try to read the features in FME Form 2024.1 (20240628 - Build 24612 - WIN64), we get this error. ArcGIS Online Feature Service Reader: Encountered an unexpected error. The error code from the server was '400' and the message was: 'Cannot perform query. Invalid query parameters.'. Details: '24214: Circular arc segments with Z values must have equal Z value for all 3 points., Unable to perform query. Please check your parameters.'Python Exception <ArcGISServerError>: FMEException: 928221: ArcGIS Online Feature Service Reader: Encountered an unexpected error. The error code from the server was '400' and the message was: 'Cannot perform query. Invalid query parameters.'. Details: '24214: Circular arc segments with Z values must have equal Z value for all 3 points., Unable to perform query. Please check your parameters.'We have published two more feature layers and it all works fine. Does this have something to do with our data

1172

1 month ago

mr_fmeEnthusiast

asked in Data

Filter in the Feature Reader parameters

Hi,I’m using the Feature Reader to read Sentinel images.I would like to filter only the files (in a folder) that contain "B08" as part of the filename. There can be many files with "B08" in some part of the name.How can I do that? Thank´s

leestammContributor

Hi can FME deal with BPF files (point cloud data storage), or is it planning on supporting this..

514

1 month ago

paulmcm601Participant

asked in Data

Symbology issue when creating shapefile - ListExploder/Aggregator issue?

[Software FME Workbench 2021.2.3]Hi,Trying to generate an Esri Shapefile Polygon output of areas with data merged from multiple datasets to the Area dataset.The addition of Location Names (at least one per area) seems to be throwing up an issue.After doing the steps below the Transformer Views and Shapefile Symbology changes and goes a bit weird, and I’m not exactly sure how to sort it.Below polygon symbology is one area, but there are probably 30% of 1000 Polygons that have similar symbology issues.Any help would be greatly appreciated - Many thanks.Step One:Feature MergerRequestor: Area DatasetSupplier: Location Name DatasetMerge on AreaIDGenerate List:List Name: _listAdd to List: Selected AttributesSelected Attributes: Location_Name Feature Merger: Merged OutputStep Two:List ExploderList Attribute: _listAccumulation Mode: Merge List AttributesConflict Resolution: Use List Attribute Values List Exploder: Elements OutputStep Three:Aggregator to update the Location_Name to have multipl

412

1 month ago

markrmunozContributor

asked in Data

Writing date time from an xlsx

I'm trying to overwrite a date/time field with two columns from an excel file. I'd like to use attribute manager to concatenate the two columns into (Date) MM/dd/yyyy and (Time) HH:mm:ss AM/PM however I'm getting null values. The datetimeconverter led to an error saying my format wasn't FME standard. Any help is appreciated, thank you.

381

2 months ago

nbeekersContributor

asked in Data

Group records in range of Values and time

Hello, Im searching for a solution to give records a group value based on multiple values statements. Data needs to be grouped on a Vehicle in time asc.So far this is not the problem.The grouping also needs to Start with a value Status = "Start”and all the other records after Start.The grouping needs to be closed after Status End comes as a record.It looks like a range or a ranking. If someone has some idea thanks. Kind regards, Nick

iberzinsObserver

Hello!I am importing data from Aerodrome data (gdb) to AMDB. Aerodrome has reflectors on the edges of taxyway. What should these reflectors be stored as in the AMDB?Maybe AerodromeSurfaceLighting or some kind of Marking or AerodromeSign or something else? The Reflectors

150

2 months ago

chris.schulzContributor

asked in Data

CouchDB deprecation

Hi, Ive noted that CouchDB has been deprecated. Of course, this is the exact thing I need right now :( Is there any alternative way to access this database using the latest (2024) version of FME?

523

2 months ago

niksandsContributor

asked in Data

AGOL Feature Service Writer resets visibility when using 'Truncate Existing'

I’m using the ESRI ArcGIS Online Feature Service Writer to update a layer in AGOL. I have it configured as per the screenshot below.After it runs, the layer in AGOL is visible when zoomed all the way out!So, I then republish the layer in ArcGIS Pro and it fixes the layer so that it now disappears when zoomed out (I could also do this directly in AGOL).I then change the FME writer to ‘Use Existing’ (and not ‘Truncate Existing’) and re-run the translation.This time it works without affecting the AGOL layer visibility.So I change the FME writer back to ‘Truncate Existing’ (and not ‘Use Existing’) and re-run the translation.It breaks the AGOL layer visibility again!I plan to use something like this to update many layers, and I cannot have them all visible when zoomed out. It would be a complete mess.Am I doing something wrong? How can I update the data (including deleting features) without affecting the AGOL layer’s visibility?(The issue occurs for both 'Insert' and 'Upsert'. I’m using

441

2 months ago

philippebEnthusiast

asked in Data

Create new field to ArcGIS Portal Feature

Hello,Is it possible to write new fields in ArcGIS Portal Feature Service with a writer ?I’m getting that message : ArcGIS Portal Feature Service Writer: Attribute 'NAME' on feature type 'Roads' does not match any field on the corresponding layer/table. The attribute will be ignored I can’t believe I have to create those fields with ArcGIS Pro, publish the layer and than run my workbench. Update:OK I just found out that I need my fields to be in lowercase to make it work.But for some reasons some field type are not working.Is there any existing list of all the constraints we need to follow for the fields in Portal? That would save me time! Thanks!

442

2 months ago

emerson_stylesContributor

asked in Data

Strange characters appearing instead of date

I have a script that pulls numerous datasets together, formats them into a consistent set of attributes and then writes them to .TAB files.I’m having difficulty with date field (STARTD) in the final output because although appearing to be correct in the .TAB files, ArcPy is not able to detect the dates in the .TAB files and instead shows the data as odd characters. Please see the image below for more information. Has anyone seen this before or aware of how to fix the issue? I’m not sure if there is a setting in my FME script that may need to be amended.

181

2 months ago

femoContributor

asked in Data

Reading Excel files with different columns

Hello,i need to process excel files, currently by means of FME Form 2023.1, perspectively using FME Flow. These files, however, can differ in columns available. See attachments. The files are processed individually, only one file at a time.If i run the process with file “tk_template_v0.0.xlsx” I want the featurereader to display only the columns, containing in this file. If I run “tk_template_v0.1.xlsx”, which has got 3 more columns, i want these columns to be available in the process as well.I would like to run the script with these files without clicking on the “Feature Types to Read” button any time I process a new file! The sheet name always have the name “Tabelle1”.Unfortunately I do not find a set of parameter that results in the desired output.The existing posts have not yet enlighten me! Any ideas?

635

2 months ago

pehrsContributor

asked in Data

IFMEFeature with aggregated/multiple geometry

We have a reader plugin (c++) that should be able to set aggregated/ multiple geometry to an IFMEFeature.For example:We have only several IFMEGeometry objects. These IFMEGeometry objects should be set to an IFMEFeature. So that we have finally an IFMEFeature that consists of several different geometry objects/ parts. IFMEFeatureIFMEGeometry_Part_1 IFMEGeometry_Part_2 …. IFMEGeometry_Part_nfme_type = fme_areafme_geometry = fme_aggregatefme_feature_type = DepthArea How can we solve this?

494

2 months ago

philippebEnthusiast

asked in Data

Read multiple Excel file with different starting line

Hi,I have thousand of Excel files I need to read but all of them have different starting attributes line. Sometimes line 15 or 16 or 17.(not me who developed that method of working….)The attributes are all the same for all the files.Any idea how I could have the Excel Reader to detect the starting line?Thanks !

503

2 months ago

albjohContributor

asked in Data

Colorize polygons based on attribute value for writing/visualizing HTML

Hi, I’m looknig for a way to colorize polygons based on attribute value for writing to HTML and further use in data streaming service with workspace app. Each polygon have attributes with unique name and color.I want to apply these colors to based on the values in the “name” attribute. I know it’s possible to replace text in the “html_content” using the string replacer but how can I make use of my existing attributes when adding eg. a javaScript function? I guess I need to populate the feature properties dynamically from name attribute:However my edits seem to be omitted when written to final html…Any examples/ideas on this

430

2 months ago

scarterContributor

asked in Data

Issues converting tab to geojson due to NativeX in file

Hello Forum, I need to convert a tab file to geojson, which is ordinarily a simple task. However I am getting this error message…“MITAB Reader: Unsupported table type 'NativeX' in file #######################. This type of .TAB file cannot be read by this library.”Note that the # symbols are masking the UNC path and file name.Does anyone know a way around this? Thanks,Stuart

263

2 months ago

fmesafe.podporaContributor

asked in Data

GeoJSON entry according to the defined schema

Hey. Hey. I have a problem with GeoJson. GeoJson automatically writes the geometry and I would need to write an extra "layer" block to this geometry block, see attachment layer. I tried to create a template on json templater see fmwt, but it doesn't work. The idea of this template was that I would define the structure as json including the "layer" block and then write it as text with all parameters as geoJson, but I don't know if this is a good way to go. Would you recommend me any other options? I also have the schemas available, but I'm not sure on which transformer and in what combination with other transformers to use them to achieve the result. Thank you very much

SHP with ZM dimensions to Postgis linestringZ

Hi all, I did not found the way to remove the M dimension from a shapefile to include it into a postgis geometry defined as linestringZ. Do you know the transforer to be used ? Thanks, Pierre

295

2 months ago

Page 5 / 272

Badge winners

arunkumarkchas earned the badge FME Form Basic Training
malthernhas earned the badge Student (Grey)
sgsolingenhas earned the badge Teacher (Grey)
changeofmotionhas earned the badge Teacher (Grey)
User15923979821343989192has earned the badge Welcome Back

Show all badges

30,146: Posts

Terms & Conditions Cookie settings

Sign up

Already have an account? Login

An FME Account is required to contribute

Login to the community

An FME Account is required to contribute

Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.

Enter your e-mail address

Back to overview

Scanning file for viruses.

Sorry, we're still checking this file's contents to make sure it's safe to download. Please try again in a few minutes.

This file cannot be downloaded

Sorry, our virus scanner detected that this file isn't safe to download.

A product of

Company Products Giving Back Careers Contact Privacy Feedback

Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.