Using transformers to modify data.
Recently active
Hi FME Community We have a dynamic FME workbench which iterates through 89 datasets in an Oracle Database and inserts all records into datasets in a SQL Database. Some of the datasets are large (over 4 million records) and the whole process for truncate & load can take over 24 hours to run. We are looking for ways to improve the speed of this process. We are currently using a feature reader to get the Oracle features, then using a few transformers to clean up the data (i.e. in case there are multiple geometries), then using a feature writer to insert the records into the SQL database. How can we speed up and/or optimize this process? Is there a way to use a SQL Executor to improve the process?
Hi Community,I’ve developed a workflow that enables me to connect to an external SFTP server and download all contents from a specified folder. I followed the guidance provided in the following post:The workflow functions well for a single folder. However, the SFTP directory structure includes multiple nested subfolders. I now need to enhance the automation to recursively traverse these subfolders. Please see the structure example below. Additionally, the SFTP contains files in various formats. To handle this, I plan to use multiple StringSearcher transformers—one for each file type (e.g., .tiff, .asci, .las). My challenge is ensuring that FME iterates through all subfolders to identify and process the relevant files.Could anyone advise on how to configure FME to recursively search through subfolders and apply the appropriate filters?Many thanks in advance!
What is the best way to extract the path_modified_date, path_created_date, and path_accessed_date for files in a folder (these are shapefiles and zip file)? I ran the Directory and File Pathnames Reader, and it shows these attributes as missing. Is there a transformer I need to use after this Reader to extract this information?
I have this line (the green one): And I want to calculate the shortest path from the start of the line to its end across the network (the violet one).How can I do it?What is it the right way to do it? How can I set the parameters of the “ShortestPathFinder”?attached files!Tnks
Hi everyone,Brand new to FME (ArcGIS Interoperability) - like 6 hours into it. I have a process in ArcGIS Pro that I’m trying to replicate in FME. It starts with a one to many spatial join where I also keep all target features. I have parcel data (Target) that I want to join to address points. Multple address points can sit on a single parcel (think apartments and duplexes), also 1 address point can be snapped to multple parcels (think of a property owner that also owns the adjoining lot. I am trying to create a layer : that will duplicate the parcel for every time an address intersects with it - 5 apartments on one parcel will produce 5 stacked parcels If an address is snapped on the edge of two parcels, both output parcels will inherit the attributes of the address point If a parcel has no address, it will still be included in the output.This is very simple in ArcPro with a one to many spatial join set to keep all target features But I’ve been struggling for hours trying to get
Hello everyone,I have a column with output that changes depending on different input.Ideally, I would like to populate a published parameter (Choice) based on the output of that column, so then a user can click on a value of that choice list on FME Server in let's say a workspace app.Does anyone know how to dynamically fill in a choice list based on the output of a column?
Hi community, does anyone know how to find these points using NeighborFinder/VertexCreator and then create a line with LineBuilder? My goal is to find the shortest route between the "common" points of these two road networks (existing vs planned).
Hello together (and thanks for your help)I’m testing my 2022.2 workbenches in 2024.2Every single workbench generates an error when having a testfilter in it:Cannot have multiple factories with the same name. Multiple factories were assigned the name 'BufferRuler_4_TestFilter'The testfilter itself is not filled anymore.Like that it looks in 2022.2: This is 2024.2: Any suggestion anyone? Thanks in advance, Hannes
Can FME create folders in an on-prem Sharepoint Library? I’m successfully connecting to the library using the Sharepoint List type (Reader and Writer), and looking at written data I can see an existing folder, but I can’t use the writer to create folders.
Hello,I am currently working with some database PostGIS rasters and wanted to import some into FME 2024.1, using a filter to only select 2 out of 11030.To do so, I tried both a SQL Executor and a FeatureReader (as both enables PostGIS raster reading).The SQL Executor filters the rasters correctly but doesn’t create the geometry and returns an error in the log file.The FeatureReader loads the 11030 rasters without filtering, suggesting a problem with the “WHERE clause”.I tried the same thing using FME 2023.1 and the SQL Executor is working under this version. I am wondering if there is a bug in the FME 2024.1 version.The problem was the same for the Feature Reader.A colleague said that the Feature Reader outputs everything as well in previous versions of FME (2022.2 and 2021.1).Do you know when the FME 2024.1 bug will be corrected?We are linking our work with ArcPro and we know we will have to use FME 2024.I am enclosing a share folder with the different logs (as txt) and the 2 workbenc
I’ve tried multiple times to read gml file with feature reader or import that into the workspace using reader. I tried with multiple settings but failed.The gml file structure seems ok as it is loaded into old QGIS without any problems. Any advises or tips how to handle such files are welcome.
Error:I have a workflow using a generic reader and it give me below warning:2022-03-10 22:07:41| 135.3| 0.3|WARN |Detected 2600000 unique Attribute Names strings in the translation. Several unique names may result from operations like list attribute creation or feature type fanout. Reducing the number of unique names may improve translation performance and reduce memory usageany idea what the reason is?
Hey all,After upgrading from the 2024 to 2025 version, I noticed my workbench break. After doing some investigating I noticed that the PointOnAreaOverlayer transformer handles attributes differently.On the 2024 version the attributes of joined datasets would come through as ‘exposed’ attributes.In the 2025 version, these attributes were pushed into the unexposed attributes:Is this by design? Why the change?The properties of the transformer are exactly the same between the two versions. It isn’t the end of the world as I can use the attribute exposer to make it work. I was just also wondering is it normal that workbenches may break after major release changes? As this has now occurred after each last 2 major changes.
Hello,I'm working on an electrical network with substations and feeders.I want to assign a number to each feeder using NetworkTopologyCalculator, grouped by substation name. What I get What I want to achieveMany thanks in advance
Dear All,We need to convert TAB files that contain annotations similar to the following into a SQL Server spatial table with point geometry and an attribute to represent the annotation I have created the following workbench but it does not update the “dimention” attribute with the annotation text available in the MapInfo TAB fileOutput
In FME version 2023.1. We can keep the attribute field order after schema scanner for most of dataset. But for the attached dataset, we can't keep the attribute field order after schema scanner.Is any problem in my dataset?The fmw is attached alsoThanks
Good day Safers,I’m cutting my teeth on the new DocumentPDF Writer in FME 2025.1 Trying to set up a “Ledger” (17x11 inches) page in the stylerhttps://docs.safe.com/fme/html/FME-Form-Documentation/FME-Transformers/Transformers/documentpdfstyler-pkg.htmNot really clear how to do this with the Page Properties.I know this is a work in progress so hopefully this can be worked out. Please advise.Best, Reed
Hi All,Has anyone tried to follow the steps for ‘Using the OpenAICompletionsConnector’ tutorial? Link here: https://support.safe.com/hc/en-us/articles/25407431621517-Using-the-OpenAICompletionsConnectorI have used the FMWT attached to the tutorial but I have the following error when trying to run it: A few considerations below:I’m using FME(R) 2024.2.4.0 (20250318 - Build 24831 - WIN64) I have created my own API Key Thanks
Hi everyone,I’m working on an FME workspace where I have two input ports: One port brings in the main data (features). The second port provides attribute values from an Excel file, which I want to use to filter the data from port one. And here you can see how the data from port one looks:Here is what the Excel file looks like: I’m basically trying to filter the data (output of AttributeRemover_6, see first image) where:destination_class_source = StructureBoundary AND assetgroup.description = InstallationBoundary AND assettype.description = CabinetBoundary and cus_cstype_id = 2 destination_class_source = StructureBoundary AND assetgroup.description = InstallationBoundary AND assettype.description = CabinetBoundary and cus_cstype_id = 9 destination_class_source = StructureBoundary AND assetgroup.description = InstallationBoundary AND assettype.description = CabinetBoundary and cus_cstype_id = 3 The filter information comes from the Excel file (second image). What would be the best
Hello, does anyone know how to create a solid from a list of intersecting 3D planes, the result of the intersection of which creates a closed shape? The input 3D planes are not logically numbered with each other in any way.
Hi,As per my current workspace I use feature reader to read layers from AWS and this after some translation is written to Feature writer with file format as gdb. I want the Alias also to be written to the file gdb. Tried by using attribute manager with similar attribute name followed by _alias (same case as existing attribute). This dint work. Any Ideas?
Hi I have a point cloud that I want to simplify. I need 1 point in every 1cmx1cmx1cm space/voxel. I already did something similar, but that was in 2D with point clouds containing only ground points and with the resulting point cloud as some sort of DTM.In short, I used the 2dgridaccumulator to make a grid. Then I calculated the mean Z value’s of al points in a grid tile and added that value to the centerpoint of that grid. Than I used the vertexcreator to make the new point cloud. Now I need to do ‘kind of’ the same thing but in a 3D grid…I was looking at some transformers: pointcloudduplicateremover, pointcloudsimplifier, pointcloudspatialthinner, 3dboundingboxreplacer… I haven’t found a really good way to do this. Anyone with ideas?
Good morning,I have a folder in an S3 bucket that contains a folder for each day of the year and each of those folders contain about 24 files. Since I need to download the data in order to process it, is there a way to only download the folders that contain the word “2025”? I have not been able to format a wildcard that works for a folder. I could also potentially use a wildcard to select the files, if there is a way to do that.Thank you,Peter
Hi, We use the transformer HTML2PDFConvertor, but for some reason it has stoped working and reject the input. Anyone who has an idea?
Just want to check I’m not missing something obvious.I want to dissolve some polygons and then aggregate distinct areas into multipolygons. One of the attributes of the original polygons needs to be concatenated into a single line with no duplicates. Fairly simple with a list builder/listduplicateremover/listconcatenator if i was only using the dissolver. But if i want to get combine attributes from both processes it seems unnecessarily complicated without resorting to python (which only requires a couple of lines). Am I missing an easier way to do this?StartEnd