Using transformers to modify data.
Recently active
Good morning,I have a folder in an S3 bucket that contains a folder for each day of the year and each of those folders contain about 24 files. Since I need to download the data in order to process it, is there a way to only download the folders that contain the word “2025”? I have not been able to format a wildcard that works for a folder. I could also potentially use a wildcard to select the files, if there is a way to do that.Thank you,Peter
Hi, We use the transformer HTML2PDFConvertor, but for some reason it has stoped working and reject the input. Anyone who has an idea?
Just want to check I’m not missing something obvious.I want to dissolve some polygons and then aggregate distinct areas into multipolygons. One of the attributes of the original polygons needs to be concatenated into a single line with no duplicates. Fairly simple with a list builder/listduplicateremover/listconcatenator if i was only using the dissolver. But if i want to get combine attributes from both processes it seems unnecessarily complicated without resorting to python (which only requires a couple of lines). Am I missing an easier way to do this?StartEnd
I’m connecting a Sharepoint dataset to a portal dataset, with the intention of creating geospatial data in portal, each record having an object-id along with geometry. The Sharepoint record would be manually linked via this object-id, and then FME would sync changes of related columns.My question is which transform (there are so many!) makes the most sense, and how can I limit the updates to only those records that have a matching object-id?In my writer, the upsert action exposes a match column config, but the update action doesn’t. I feel like I’m missing something, as an update statement still needs something to match on.
Hi all,I’m trying to create clean cutouts (like donut shapes) from overlapping polygons in FME.For example, lets say I have four geometries A, B, C, and D, with each progressively larger and fully containing/overlapping the smaller ones. I want to subtract the inner shapes from the outer ones (e.g. D minus C, B, A; C minus B, A; etc.) so that only the ring-like areas remain. I’ve previously used the AreaOnAreaOverlayer with overlap counts to determine hierarchy (e.g. a shape with 1 overlap is the largest, and one with 3 overlaps is the smallest, so I can then figure out what is what).However, this approach breaks when some of the smaller shapes are missing. I still need the output features to retain their original attributes (e.g. A, B, C, etc.), regardless of which shapes are present. For example, shape A might be missing, or both A and B but you’d never have A and C without B. So I need a method that can still produce correct results even when some of the smaller shapes aren’t ther
I am merging incoming attributes in Aggregator. I concatenate the values from a field but I am getting duplicate values. How do I create a cleaner attribute without duplicates?
Hi community, does anyone know how I can convert a LAS file into an RCP point cloud file using FME?
Hello, I have a field consisting of two lines:Line 1: (empty space) String1 (empty space) String2 (empty space) String3 (empty space)Line 2 : (empty space) String5 (empty space) String6 (empty space) String7 (empty space) String8Both lines exist within the same field.My objective:If I choose to keep, for example, **String1** from Line 1 and **String5** from Line 2, then all other strings in their respective lines should be replaced with empty spaces.Alternatively, if I choose to keep **String3** from Line 1 and **String7** from Line 2, the rest of the strings in each line should become empty spaces.In short, I want the ability to retain specific strings from each line while replacing all others with empty spaces.example : Thanks in advance
Hello,I have recently upgraded from fme flow 2024 to 2025.0.1. - (still using fme form 2024 for now)(EDIT: I have since aligned the versions between form and flow, and still getting the same error.)I have a workbench which uses the SharePoint online connector transformer I am currently using python version 3.12+.I can run this workbench without any errors on fme form 2024, but when uploaded to fme flow 2025 I am getting the following issue: Any tips on how to resolve this?Thanks.
We upgraded our migration chain last week from FME 2025.0.1 to FME 2025.0.2 and our Excel writers started to mess up xlsx files. Some column names are renamed and hundreds of them added all with the same name: "Created by LibXL trial version 4.5.0. Please buy the LibXL full version for removing this message."Strange thing is that not all workbenches are messing xlsx files like in the screenshot. Some of them behave as before in FME 2025.0.1. I indeed find libxl.dll library in the plugins folder of FME installation and version is 4.5.0.0.During upgrade to FME 2025.0.2 all workbenches, transformers, readers and writes are upgraded. Recreating Excel writer in FME 2025.0.2 does not help.Can someone please give me a hint if its possible to overcome this issue in FME 2025.0.2?
I have a HTTPCaller to access an API from the Dutch road service Melvin. It returns an attribute with the json including geometry (response_body - here renamed to ‘gegevens’). I am quite new to JSON.In order to extract the features I use 2 JSONFragmenters: one for the attributes and one for the geometry. I've tried many other setups with JSONfragmenters,-flatteners,-extractors etc, but I can't seem to find an answer as to how to get geometry and attributes from a json nicely joined back together.Current setup below. First I extract the geometry from the attribute (gegevens) and then from the same attribute (as the resulting attribute gets changed, thus I created another with the same information named gegevens_basis) by the fragmenter for the attributes. However it multiplies the number of attributes (700+ to 500,000+). Another setup (also below) sent all geometry to rejected. I also tried to do it from the same attribute from a junction but then I cant join then as I cannot get the id
Hi,i have an problem with the Oracle Non-Spatial (and Spatial) Reader / Writer.FME seems to have an issue when the password contains a section sign (§).Other special characters don't seem to cause any problems - at least dollar signs ($) and slashes (/) work.As soon as a § is included, the login fails.Version: FME Form 2024.2.1.0 (Build 24801)I tried with Named-Connection and with Embedded Connection Parameters.Maybe it is a similar problem as this post:Error Message:Creating reader for format: Oracle Non-SpatialCreating reader for format: Oracle Non-SpatialEnvironment variable ORACLE_HOME is not set.Connecting to ORACLE database `ABC' with user name of `CARLO_TEST' and password of `***'Error connecting to Oracle database: message was `ORA-01017: Benutzername/Kennwort ungültig; Anmeldung abgelehnt'. (serverType=`ORACLE8i', serverName=`ABC', userName=`CARLO_TEST', password=`***', dbname=`')Failed to obtain any schemas from reader 'ORACLE_NONSPATIAL' from 1 datasets. This may be due to
Hi all,I seem to have an issue with writing to the Temp folder location (E:\\Temp, in this case). I will get an error like the following -Failed to write feature data to `E:\\temp\\wb-cache-NSPL_V3.2_ShortName-rUhMFp\\Main_FeatureJoiner_20 -1 24 fo 0 JOINED 0 6641bd2679d46e05f371fbb029766a8cafbc270f.ffsupdating' I've tried creating a new workspace from scratch, renaming it and I am able to copy files to that folder without any issue. I also ran FME Workbench with elevated privileges and as a different user, even with a different Temp location but no success with any of them. Has anyone here had anything like this occur? And if so, is there a known way to resolve it? I've deleted all Temp files and again, no change. All a bit odd out of nowhere!Thank you
I’m migrating to ArcGIS Pro and working on a workflow to convert geodb_pro_annotations to geodb_annotations so that I can edit/create annotations in ArcGIS Pro and downgrade them so that they can be used in ArcMap and read with a legacy tool (that currently has no ArcGIS Pro upgrade path).I seem to be having an issue writing some of the features to a new local FGDB where the FeatureWriter errors out with the following message:“An error occurred while attempting to create a feature using the feature class 'StreetName_Rural_Smaller_LEVEL_63'. The error number from ArcObjects is: '-2147220947'. The error message from ArcObjects is: {Cannot acquire a lock.}”The data is created/edited in an SDE and exported to a local FGDB for this downgrade. There are 25 annotation features containing 1 to 2,000,000 annotation records. This error seems to only occur on annotation features with more than 500,000 annotation records. The others are able to be written out to the FGDB with no issues.I’ve tried
Hi,I have a CSV file containing ObjectID and associated AttachmentNumber values.I want to group the ObjectIDs so that the total number of attachments in each group is as close to 1,000 as possible.How can I achieve this?
Hi all,I have .shp data sets (100+) which I read with feature reader. The attributes’ values are saved as list ,attribute “ID” for shopA.shp has values as {0, 2, 4, 8} and another attribute “Desc_”: {cake, bread, drink}, my aim is to save them as json file so the json will look:{ "uuid" : "1b2299d1-b97a-48d9-9a99-9994491ebb34", "path_directory_windows" : "O:\\M\\A\\B\\", "members" : [ { "base_name" : "shopA.shp", "created" : "20240225", “attributes” : [“ID”: “long”, “0, 2, 4, 8”, min = 0, max = 8, mean = 3.5, median = 3, mode = None,“Desc” : char(256), “cake, bread, drink” ] }, { "uuid" : "322299d1-b94a-48d9-9a99-99944912bb34", "path_directory_windows" : "O:\\M\\A\\C\\", "members" : [ { "base_name" : "shopB.shp", "created" : "20240225", “attributes” : [“ID”: “long”, “1, 2, 3, 4”, min = 1, max = 4, mean = 2.5, median = 2.5, mode = None,“Desc” : char(256), “cake, orange, apple” ] }, Thank you so
TLDR: sys.path does not contain FME_MF_DIR, so I can’t directly import a local Python module. This contradicts FME’s documentation. What am I missing? Hi all,I’m having problems importing a local Python module located in the same directory as my FME workspace. Could you help me find out what the problem is or am I missing something?According to the PythonCaller documentation,FME will search both the standard Python module locations and the workspace location to find the module to be imported.However, when I try to import my module in <workspace directory>/feature_processor/__init__.py with the following code:from feature_processor import FeatureProcessorI get the following exception:Message Type: fme::internal::_v0::py::ExceptionPython Exception <ModuleNotFoundError>: No module named 'feature_processor'Error executing string `from feature_processor import FeatureProcessor'Factory proxy not initializedFeatureProcessor (PythonFactory): PythonFactory failed to process featu
I'm working with FME 2021 and have run into a challenge regarding dynamic GeoJSON handling.I use the HTTPCaller to retrieve GeoJSON from several hundred different services. Each service returns features with different attribute schemas. My goal is to dynamically expose all attributes and the geometry from each GeoJSON response and write them to various formats (e.g., GDB, Shapefile, etc.).I've tested workflows using JSONFragmenter and manually selected attributes, but this approach doesn't scale—it’s not feasible to manually configure the schema for hundreds of unique GeoJSON sources.Is there a robust and dynamic way in FME 2021 to automatically expose all properties (attributes) in each feature, regardless of schema, along with the geometry?Appreciate any guidance or best practices for handling this kind of dynamic schema in FME 2021.
Hello, I want to replace the list of attributes of the ChangeDetector with a user parameter. The attributes of the original and revised features are not exposed. The user parameter is in Python: it's a string derived from another parameter (example: 'field1 field2 field3'). I don't get any error but I guess values are not considered.I have the same problem with other transformers based on the same attribute list type (AttributeExposer).It is an issue or a problem with the string? Thank you
Hi,I am facing the problem mentioned in the title, especially when I want to detect Self-Intersections in 2D. FME detects a lot more Self-Intersections while PostGIS is fine with the geometry.What are the reasons?As we want to build up a standard quality gate for our data, I need to be certain that both ways - either using PostGIS or FME - act in the same way. What is the configuration then for FME to match the behaviour of PostGIS’s ST_IsValid function?Thanks in advance! :-)
Hi,I split a line at two points using PointOnLineOverlayer, which generated three segments. I extracted the coordinates before breaking the line and again after the splits. I was expecting the start coordinates of the second and third segments to be different from the original line's start coordinates, but all three segments appear to have the same start x and y in both split and original coordinates. Is it something with the split that went wrong because i had only one case in which coordinates were different but rest of the examples are showing this, which is not helping. Any feedback would be greatly appreciated! Thanks.
I have a workflow that:Takes a feature class from an ESRI Geodatabase (ArcSDE) Removes some attributes via AttributeRemover Writes to several different formatsI want to change Step 2 to use an external list (for example, a TXT file or CSV - the format is flexible), because other users will be regularly updating the list of attributes to remove.It seems like AttributeRemover only takes a single string or manual selection of attributes, so I decided to try BulkAttributeRemover, since it can take an attribute value. However, I ran into this problem:Let’s say I want to remove all attributes that contain the string PRIVATE.When I type the string PRIVATE into BulkAttributeRemover: Expression to Remove, the transformer functions as expected. It removes all attributes that contain the string PRIVATE.However, if I create an attribute called “regex” and set its value to PRIVATE, and then use the attribute “regex” as the Expression to Remove in BulkAttributeRemover, the transformer removes all va
I have an attribute called “month” from an imported CSV. I’ve formatted the value to be in the format “September 1 2024”.Now I want to convert the “month” attribute to a usable date format such as ISO YYYY-MM-DD.If I use the dropdown to select the attribute, the value of the DateTime Attributes becomes “@Value(month)”.When run, the workspace sends all rows to the Rejected port with fme_rejection_code. = MISSING_ATTRIBUTE_DATETIME and fme_rejection_message = Null, empty, or missing datetime was found. Enable the passthrough parameter to disable this However, if I manually type in “month” into Datetime Attributes:I get the expected results: Version: FME(R) 2024.2.1.0 (20241202 - Build 24801 - macosx-aarch64)I’ve tested it on FME(R) 2025.0.2.0 (20250512 - Build 25237 - macosx-aarch64) with the same result.
I am trying to concatenate 6-7 fields to create single field with spaces in-between different field values to create a address like string, no commas needed. Can I do it in text editor? Field1 Field2 Field3 original: 123 ABC Street Filed Combined Target: 123 ABC Street
I just returned to work from the Peak Of Data and AI in Seattle and am trying new things. In existing workspaces, I have added two creators to my workspace where I am adding a DateTimeStamp >DateTimeConverter - then writing to a new tab (Information) in my excel worksheetthe Second creator I am utilizing a FilePathExtractor and writing to the same new tab in excel. Essentially I am creating metadata to make my job easier to rerun projects year after year. The question I have is : is there an easier way to extract the File Path of where I have the .fmw saved without needing to navigate to it within the FilePathExtractor? My ultimate goal is to create a custom transformer that I can share with co-workers that can help them add the date created and the fmw’s file path to their excel writers.