- - Community Resources
Ideas
Product Updates
Events
Blog

Home
Forums
FME Form
Data

Data

Using readers and writers to extract and load data.

6,784 Posts
24,952 Replies

When you subscribe we will email you when there is a new topic in this category

6784 Posts

Recently active

Most replies Most views Newest first

skotnicamParticipant

asked in Data

FME 2021.0.3.2 crashes on macOS 14.6.1

Hello,I am running on macOS 14.6.1 and I use FME 2021.0.3.2 which stopped working from today. It crashes after “Initializing Main Window”. I am attaching the relevant part (I hope) of the crash report. Has anyone run into the same problem?(Data inspector and Quick translator work, only Workbench crushes)Thank you! Notes:PC register does not match crashing frame (0x0 vs 0x7FF89317AA78)Crashed Thread: 0 CrBrowserMain Dispatch queue: com.apple.main-threadException Type: EXC_CRASH (SIGABRT)Exception Codes: 0x0000000000000000, 0x0000000000000000Termination Reason: Namespace SIGNAL, Code 6 Abort trap: 6Terminating Process: FME Workbench [1810]Application Specific Information:abort() calledError Formulating Crash Report:PC register does not match crashing frame (0x0 vs 0x7FF89317AA78)Thread 0 Crashed:: CrBrowserMain Dispatch queue: com.apple.main-thread0 ??? 0x7ff89317aa78 ???1 libsystem_kernel.dylib 0x7ff802e92db6 __pt

471

2 months ago

axcel.fmeContributor

asked in Data

Substring operation incorrect in several cases

Dear all,I’m spending a lot of time trying to sort out an issue with a substring operation but I cannot see my results are not correct in several places.In FME workbench 2024.0, I use the the AttributeManager to read a substring from the attribute _response_body and to write it to the attribute info. The response body is the result from a reverse geo-coding with OpenStreetMap and I want to extract the location for the given coordinates. To extract the relevant part, I use the following sequence of String operators:1) Take everything left of </result>2) From this, take everything right of address_rank3) From this, take everything right of >The actual (unfortunately complicated) condition statement which I’m using is:@Right(@Right(@Left(@Value(_response_body),@FindString(@Value(_response_body),</result>)-1),@StringLength(@Left(@Value(_response_body),@FindString(@Value(_response_body),</result>)-1))-@FindString(@Value(_response_body),address_rank)),@StringLength(@Righ

556

2 months ago

joku041199Participant

asked in Data

Only loading in Raster data based on Area of Interest (from a GeometryReplacer and User Parameters)

Hello, I am using FME 2022.2 and I’m trying to load in Raster data, based on an Area of Interest (AOI), specified by the end-user in FME Server (yes my organization is not using FME Flow yet, but the old FME Server). We have a few big Raster datasets on a local drive (of the entire Netherlands), which are loaded in as regular Readers. Then, using a Creator and GeometryReplacer, the end-user can specify an AOI (using a User Parameter), and the Rasters are then clipped in the Workspace using a Clipper. However, this means that the big Raster datasets need to be loaded/processed every single time the Workspace is ran, which is bad for performance. What I want, is that the Raster data is loaded into the Workspace, but only the data which intersects the AOI, specified by the end-user. I tried using a FeatureReader and use the AOI from the end-user as trigger to load in the Raster data. However, for some reason, the entire Raster dataset is loaded every time, instead of only the Raster data

392

2 months ago

raabsterContributor

asked in Data

Can't get workspacerunner to run as expected

Hi!So, I have a workbench that takes a shapefile and an excel file as input, does some magic and outputs an excelfile. The workbench works as expected. So far so good. Se image below. Now, I have a number of shapefiles that I want to process in the same way, so I figured a workspacerunner would be a good solution, instead of replaceing the input shapefile in the image above.So in a second workbech, I take a PATH as reader, filters out my shapefiles and into the workspacerunner (see img below). Everything seem to be working - but only the shapefile that has the same filename (test.shp in the images) gets treated properly.So in this example, test.shp get an output excel file that seems fine, but the other two does not. Any ideas of why that is? My guess is that I’m doing something wrong 😅 I can’t share the workbenches I’m afraid. Cheers!

333

2 months ago

mr_fmeEnthusiast

asked in Data

Get attribute value in Python Caller

Hi,I'm programming in Python using FME Desktop, specifically using the Python Caller transformer. I'm trying to use a variable called test in my script, where test should have the value assigned to the attribute _creation_instance. I'm declaring it as follows: test = feature.getAttribute('_creation_instance'), but I'm getting the following error:Python Exception <NameError>: name 'feature' is not defined How can I properly get the value from _creation_instance? Thank´s

773

2 months ago

lambertusEnthusiast

asked in Data

Looping a set of varying user parameters

Hi, I have a workbench which is fed by a set of user parameters. It produces a dataset of 10.000+ buildings for a chosen municipality and extraction date. Is it possible to create an input which loops through these user parameters:Userparameter 1: CityUserparameter 2: Extraction dateRun City Date 1 A 2020-01-01 2 B 2020-01-01 3 C 2020-01-01 Workbench should behave like first run City A with first date, than City B with 2nd date, than City C with third date

mr_fmeEnthusiast

Hi, Sometimes after running some workspaces, the process freezes at the point shown below: The workspace doesn't run anymore. It can stay like this for hours and not progress... Has this happened to anyone else?

572

2 months ago

lambertusEnthusiast

asked in Data

Assign street codes - spatial order

I have a dataset with 3000 polygons represting streets. Each street is divided into subparts and need a unique code to be assgined. The first part refers to the streetname and is the same for each subpart, 2nd part is a unique counter.Image below shows my current (top) and wanted situation (bottom part).Any support is appreciated!

PostGIS dynamic writer forces lowercase attribute (column) names

As is stated in the title. However I need column names to be created exactly as they are, for example I have incoming attribute "OGC_ANGLE.uom" and even though I have Lower Case Attribute Names unchecked,tables in database have column names are created lowercase ("OGC_ANGLE.uom" becomes ogc_angle_uom) - this is a problem, if I read those tables and try to use xsd to create gml again, as these attributes are unknown to the xsd, therefore failing creation of the gml. I could manually rename those columns, however I would need to rename those columns every time xsd file changes, which is why I need to do this dynamically in the first place, as those xsd files are going to be updated quite often... And it seems that the dynamic PostGIS writer ignores Table Qualifier, it always creates tables in default schema, unless I specify schema in Table Name (schema.@Value(fme_feature_type))

Can FME write to an Excel table?

I have got some beautiful Excel templates containing tables (not normal ranges). It's quite easy to write to a named range within Excel but I haven't figured out how to write to a Table. Does anyone know how to do so?

964

2 months ago

helmoetzContributor

asked in Data

CS:MAP GRID Fallback used on non existing coordinate

Hi, my workspace reads WFS and gives me the warning“CS-MAP: Coordinate (approx) 3E:47N outside grid file coverage; fallback used”It seems to run fine any further, just I like to know where this message comes from, since this coordinate nowhere pops up on neither GetFeature nor GetCapabilities request. Indeed the approx coordinate does not fit within the dutch Grid, but it does not seem to come from the WFS source? My FME version is 2022.2.4

gbdContributor

I’m converting geotagged mobile photos to dwg format. Inserting them at correct photo location.All works fine except that FME automatically converts my jpeg images to png files.This results in image files (png) which are many timmes bigger then the jpeg files. Is there a way to make FME write jpeg images instead of png with the dwg writer?

792

2 months ago

tb09114Contributor

asked in Data

Add custom coordinate system to Form

I need to incorporate a custom coordinate system definition into FME Form, but can’t find the right parameters in the online form to fulfil the definition.The following parameters define the map projection, but e.g. ITRF does not exist anywhere in the online tool.projection type: Gauss Kruger (Transverse Mercator)reference ellipsoid. GRS80datum / reference frame. ITRF2005central meridian. 11° 20’scale factor at central meridian. 1.0false northing. 0.0 meterfalse easting. 1000000.0 meter How can I create a proper fme file with these parameters?

472

2 months ago

hawaiialexContributor

asked in Data

Writing one field from GIS Feature Class to API Table PUT Request

Hi everyone,Curious if I could pick your brain on the workflow below.I would like to just write the asset_id field values from the GIS feature class into the corresponding assetID field in the API table. I’d want to write the asset_id from the GIS FC to the assetID field in the API table only if the assetID doesn’t exist in the API table. They have different field names.Curious where I would place the HttpCaller transformer with the PUT request in the workspace.The GIS EGDB reader would be at the start of the workspace exposing the relevant fields. Once I rename the asset_id field from the GIS feature class to match that of assetID for the API table, what would I do? Could I hook up the output from an AttributeManager directly to an HTTPCaller without a Creator?

646

2 months ago

carlasouzaContributor

asked in Data

Update Block Attribute from CSV file

Hi everyone,I need to update the block attributes with the csv data, but I don’t know what I’m doing wrong. Can someone check my script and with this issues? I’ll really appreciate it.

392

2 months ago

fmencoContributor

asked in Data

Flattening JSONS that contain lists

Hi, I have a JSON that I want to "flatten”, Currently, it has this structure. {"A":{"id":"1","Name":"Aa","IsTrue":"True"},"1stList":[{"Type":"Yes","Number":"100"},{"Type":"No","Number":"100"},{"Type":"Maybe","Number":"1000"},{"Type":"Unknown","Number":"11"},{"Type":"Hh"}],"2ndList":[{"X":"10","Y":"100","Z":"Abc","Q":"Def"}]}{"A":{"id":"2","Name":"Bb","IsTrue":"True"},"2ndList":[{"X":"10","Y":"11","R":"100000","Z":"Efg","Q":"Klmo"}]}{"A":{"id":"3","Name":"Cc","IsTrue":"True"},"2ndList":[{"X":"10","Y":"12","Z":"Lmno","Q":"Stuv"}]}Just reading it with the following parameter settings doesn't work:Flatten Nested JSON Values into Attributes: YES I've tried with 2 listexploders after reading, but that also doesn't give the correct result.I just want the endresult to be a flat table. How can I best achieve this in FME?

707

2 months ago

lazarlubomirContributor

asked in Data

Problem with exporting to Excel

Hello, unfortunately I have repeatedly problem with exporting to Excel in Workspace. I checked log, there is following message: 2016-11-24 10:05:57| 67.6| 0.2|ERROR |Excel Writer: Failed to save changes to the spreadsheet 'D:\\XXX\\XXX\\XXX\\new.xlsx'. Check that you have write permission to the file, it has no password protection and is not opened by another application. Error message is 'Permission denied.'It looks, that there is error on Excel side, right? I just want to create new Excel file...

14920

2 months ago

juliarozemaContributor

asked in Data

How to Find Unicode Characters in Text String

We have some text fields that shouldn’t have any UNICODE characters. Turns out at least one has snuck into the data. Is there a way to Test to find UNICODE characters and then remove them?FME 2022.2.3

903

2 months ago

nordpilContributor

asked in Data

Shapefile with cst encoding spec (from GeoServer)

When exporting to shapefile from GeoServer/WFS one can specify encoding in the request, for my case I specified UTF-8.This shapefile has a cst-file that establishes the encoding (similar to cpg from ArcGIS).When I read this in Workbench/Server 2022.2 the fields have encoding cp-1252 and some garbled characters. If I change the filename to from cst to cpg the attributes come in with the correct encoding.Is this fixed in later versions, or should it be reported as a bug… ?

Writing Autocad Dynamic Blocks

I saw that in this idea, writing dynamic autocad blocks was implemented in build 22098. Is there any documentation available for this yet? I understand this is still a beta build so it might not be available. The reason for this is im trying to write out FME aggregates as Autocad blocks with different colours stored as appearances for individual geometries within the aggregate and i cant find a way to preserve these when writing to Autocad.

663

3 months ago

roshContributor

asked in Data

retrieving data list from many data tables

Hello everyone, I have a small problem I am just starting on FME so I need your help. I have at my disposal a database in which there are many schemas and many tables inside the schemas. I would like to be able to retrieve a list of each last update date of each table and a list of each first creation date of each table. Here I would like not to do it table by table but all at once! Thank you for your help and excuse me for my approximate English! (For information I am on FME 2021)to be a little clearer I want to create a loop that separates me and sends me the tables one by one to sort the attributes then writes the result in an excel file the problem is not the sorting table by table but the loop and how to automate the sending of a table by a table, Thank you!

Creating new CSV files to upload to Azure Blob Storage, via FME Server

Hello,I was hoping to be able to create an FME workspace to be uploaded to FME server, that would run an SQL script (via SQL Creator) and create a CSV file that would then be uploaded directly to Azure Blob storage.I tried the 'AzureBlobStorageConnector' (using the 'File' upload option), but it appears that the file needs to be stored locally to use this option, which seems to be no good if I want to publish and automate this script via server. I also tried 'FeatureWriter', but couldn't see an option to write to a web service.Any help would be greatly appreciated!ThanksChris

1966

3 months ago

mushailehObserver

asked in Data

Destination Esri Shapefile Folder not found

Hi all , i hope you can advise on this issue. i’m using the cloud FME Flow , i uploaded my fmw file but when i run the workspace i don’t find the shape files at the Destination Esri Shapefile Folder. any advise ?

171

3 months ago

dmbowmanContributor

asked in Data

ArcGIS Feature Service Reader Limiting Endpoints

We are moving to short-lived token authentication for ArcGIS Feature Service access. I am able to use the FeatureReader transformer and pass the token and read all endpoints (132 in total). However, I only want the transformer to retrieve a subset of endpoints. I attempted to create individual readers for each endpoint similar to what we had when using long tokens, but layers are now passing to the Initiator port. I thought adding a GeometryExtractor transformer would work, but it did not.

PostGIS writer updates column while not in attribute definition.

Heya FME specialists, can you please give a thought on the next issue:Created a new table in PostGIS with FME from shape, using the PostGIS writer. (see 00_initiate.fmw) Point feature, attributes pkey and a, b, c with values. Insert, drop and create.Updated this table with the PostGIS writer. (see 01_update.fmw) Point feature, pkey, and only a, b with values. Update based on pkey, use existing.The problem is: We decided to maintain attribute c with QGIS directly in PostGIS but kept editing the geometry and attributes a and b in the shape. I deleted the attribute c in the writer assuming it wouldn't write it anymore to the database. That assumption appeared to be wrong and to prevent updating attribute c I had to use an AttributeKeeper in front of the PostGIS writer. My problem solved but curious if this is a DB writer syntax or a bug?Using FME Desktop 2016.1.0.1 on Windows 7. Uploaded workbenches in zip for reproduction.Thanks!

287

3 months ago

Page 7 / 272

Badge winners

jakes3has earned the badge Student (Grey)
arunkumarkchas earned the badge FME Form Basic Training
malthernhas earned the badge Student (Grey)
sgsolingenhas earned the badge Teacher (Grey)
changeofmotionhas earned the badge Teacher (Grey)

Show all badges

30,147: Posts

Terms & Conditions Cookie settings

Sign up

Already have an account? Login

An FME Account is required to contribute

Login to the community

An FME Account is required to contribute

Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.

Enter your e-mail address

Back to overview

Scanning file for viruses.

Sorry, we're still checking this file's contents to make sure it's safe to download. Please try again in a few minutes.

This file cannot be downloaded

Sorry, our virus scanner detected that this file isn't safe to download.

A product of

Company Products Giving Back Careers Contact Privacy Feedback

Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.