Using transformers to modify data.
Recently active
I have an HTTP caller using a GET Request, to access an API site using a Web Connection for authentication. I am trying to find a way for it to return (either within the JSON response body or otherwise) the access token which is generated when the process runsI have Accept and Content Type as my headers. I have tried: Storing the cookies doing grant_type = client_credentials in the Query String Parameters. Creating a list for the Headers (no token content within that)In the setting for my web connection, I have this:But I am not returning anything called ‘access_token’ in the response body. To be clear, my process works as intended, but due to new constraints we need to store the access token that is generated every time to store it to use downstream
I have a workbench that takes data from Oracle, merges it with a few other tables, runs it through a schema scanner and then tries to output to a file geodatabase.The file geodatabase has the merged table created but it only contains the schema definition not the data. The output from the schema scanner shows the points in the graphics display. These are the transformers that scan and output Here is the Feature Writer The summary is showing the feature count There is some disconnect between the schema scanner and my output and was hoping someone might be able to see what I am doing wrong. The workbench is attached. Any assistance would be greatly appreciated.
I am not sure why I am struggling with this one… I have tried quite a few options. What I would like to do is:Pull Alignment file from an XML (this has the starting value as an attribute, end value can be calculated by start value + line length) Set Start and End Measures Kick out an m-enabled polyline that is geometrically calibratedI have the workbench set up to pull this in, using a geometry coercer to make sure it is a line, then I have been trying the measure generator and measure setters to get these values to stick with no luck.
HiI am encountering a critical issue when using the ArcGIS Online Feature Service Writer (within a FeatureWriter transformer) in a dynamic workspace.When a specific column contains data that AGOL deems "invalid HTML" (e.g., strings containing <, >, or specific URL parameters), the translation fails with a Fatal Error. Instead of rejecting only the problematic feature or failing just that specific layer, the entire FME Engine stops, preventing the rest of the workspace from processing other layers. I have a custom Transformer that read, process and write individual layer. this custom transformer has a group by set to unique layer id.Current Configuration:Writer: ESRI Feature Service Writer (Dynamic).Transformer: FeatureWriterError Message: ArcGIS Online Feature Service Writer: 'column_name' has invalid HTML content.What I Need Help With:Error Handling: How can I configure the FeatureWriter or the AGOL ESRI Feature Service writer to "ignore" or "skip" features with invalid HTML con
I would like to use gdalinfo to read the metadata/header for a folder of rasterfiles to get the extent and of each file. Ggalinfo can read this without loading the raster itself. This greatly speeds up the process.Can we use the gdal implementation of FME to do this and if so how? The goal is to also have this working on FME Flow.Example of gdalinfo output:gdalinfo "C:\path\to\file\1.img"Driver: HFA/Erdas Imagine Images (.img)Files: C:\path\to\file\1.imgSize is 200, 130Origin = (209000.000000000000000,529000.000000000000000)Pixel Size = (100.000000000000000,-100.000000000000000)Metadata: DataType=GenericCorner Coordinates:Upper Left ( 209000.000, 529000.000)Lower Left ( 209000.000, 516000.000)Upper Right ( 229000.000, 529000.000)Lower Right ( 229000.000, 516000.000)Center ( 219000.000, 522500.000)Band 1 Block=64x64 Type=Float32, ColorInterp=Undefined Description = Layer_1 Min=-54.770 Max=-21.170 Minimum=-54.770, Maximum=-21.170, Mean=-40.290, StdDev=6.935 NoData V
Hi, I have many LAS tiles with gps_time component. Gps time values are in standard GPS Time : ex 363439432.1942503 I would like to display the minimum and maximum times in DD/MM/YYYY HH:MM:SS format. I can use the PointCloudStatisticsCalculator to get the minimum and maximum values, but I can't work out how to convert 363439432.1942503 to the required format. Any idea? Thanks u !
Hi everyone,I am working in FME and would like to orient the flow direction of a gas network.Data: main supply lines service lines connected to gas connections Destination points – representing the gas service connection locations Objective:My goal is to ensure that all Service lines are oriented towards the Gas Service connection points.If a line is digitized in the opposite direction, it should be automatically reversed.Additionally, some lines connected to Service Connection points are split into multiple segments. I need all connected segments to have a consistent direction towards the same Service Connection points .What I tried: I used the NetworkFlowOrientor transformer I applied a Anchoredsnapper beforehand to ensure that each Service connections point lies exactly on the endpoint of the corresponding Service lines Problem: The NetworkFlowOrientor returns no output Questions: What is the correct workflow to ensure that all Service line segments are consistently orien
Good day,I have a table with a couple of records.I would like to keep one record per category and this record should be the record which has the less attributes with a null (missing) value (I highlighted them in green on the screenshot.). How can I automatically calculate which record per category I should keep?
Hi all.I just experimented with a small workflow, that scans a folder for files, and attempts to read those files using the generic format.I tried with CSV and DGN, and both worked well.I then tried with Geopackage, and then it blew up.First because FME’s guess using the extension was wrong, so I had to parametricize the format used.Then it blew up with an internal error message (“Too many recursive macro substitutions”):2026-03-03 15:14:24| 2.6| 0.0|WARN |UniversalReader -- readSchema resulted in 0 schema features being returned2026-03-03 15:14:24| 2.6| 0.0|ERROR |Too many recursive macro substitutions -- line currently is: `SOURCE_READER GENERIC REAL_FORMAT "$(REAL_FORMAT)" EXPOSED_ATTRS "$($(FORMAT_SHORT_NAME)_EXPOSE_FORMAT_ATTRS)"'2026-03-03 15:14:24| 2.6| 0.0|ERROR |class stk::ex::UString<0> __cdecl tabrdr::ClassicTableReader::macroSubstitute(const class stk::ex::UString<0> &,int)(C:\code\fme\foundation\framework\engine\tabrdr\tabrd
Hi everybody, I have a similar issue to the one the creator of this thread has: The difference is that I'm dealing with donuts. The workaround from Takashi (buffer issue location → dissolve → aggregate) doesn’t work with donuts. The reason why this doesn't work is explained here: The GeometryValidator reports self-intersections in the wrong place for donuts. If I buffer the issue location, part of the donut that doesn’t need to be buffered gets buffered, and the actual issue location remains a problem. I also tried the GeometryValidator repair function, but this didn’t work either. Attached is an example workflow which uses some public data that I use in the real workflow. You can download this data here (to big to attach): https://daten.gdz.bkg.bund.de/produkte/vg/vg250_ebenen_0101/aktuell/vg250_01-01.utm32s.shape.ebenen.zipThe workflow shows one example of the problem and my current workaround. Unfortunately, I can’t share the real workflow. In the real workflow, however, many geome
I have a donut geometry (see WKT below) which I am trying to validate. Using the GeometryValidator only looking for self intersections I get an error on the start/end point of the outer ring: Error report:Coordinates from the inspector: I would expect this to not be reported as a self intersection since polygons normally require intersection at exactly that point. Why is FME reporting this? I have tried using both FME Desktop 2021.2 and 2022 beta. Geometry WKT: POLYGON Z ((558000 6142955.0838 0,558000 6142917.0404 0,557997.11 6142919.37 0,557989.1699999999 6142921.13 0,557985.12 6142922 0,557980.8300000001 6142923.23 0,557976.28 6142926.07 0,557969.0900000001 6142930.65 0,557964.65 6142933.45 0,557957.52 6142934.96 0,557954.37 6142937.05 0,557952.6699999999 6142938.1 0,557951.45 6142938.28 0,557950.3 6142937.8 0,557949 6142936.3 0,557948.13 6142930.34 0,557949.23 6142924.47 0,557953.26 6142917.84 0,557961.4 6142913.23 0,557968.81 6142907.49 0,557974.75 6142903.11 0,557980.95 6142900.5
Hello everyone,I am seeking confirmation on whether GeometryValidator supports 3D geometry validation (such as solids, meshes, point clouds, etc.) in version 2025.0I noticed that GeometryValidator includes an issue type called “Self-Intersection in 2D”, which suggests that this one specifically built for 2D geometry. However, I would like to understand whether the other issue types - such as Missing Vertex Normals, Missing Texture Coordinates, Invalid Solid Boundaries, Invalid Solid Voids, Contains Non-Planar Surfaces, Invalid Area Orientation, Surface Orientation, and others - are applicable to and capable of validating 3D data.Could you please clarify which of these validations support 3D geometries?Thanks in advanced!
Hello I have FME 2022.2 version and i also could upgrade it to 2025 if it helps but i would like to extract data Input data : *.pts or *.e57 file format (about 2 to 5 room point cloud data in One *.pts file/*.e57file)Desired output data : *.pts file format (ONLY 1 room to be extracted in *.pts file format)Other supporting data : Sitewide room model in *.ifc format (federated model having about 20+ rooms in 1 IFC format) TASK : Using IFC room model for any give Room for example (Room1), i would like to clip the input data set i.e. in *.pts/*.e57 file format and out only that particular room (Room1) *.pts format data. Am not sure how to set / which reader and writer / transformer to use to get it right . I understand i will have to use Clipper to clip *.pts file using IFC file somewhere. Highly appreciate and obliged with workflow for your support and know-how.
Hi all, I regularly setup workspaces with quite a few feature readers & writers that need a lot of parameters changed when moving between environments. I make use of scripted parameters to make this easy based on an initial Environment selection.The one parameter I can’t get to update correctly with a scripted parameter is the web connection, does anyone know if this is possible?
Has anyone else experienced this? I'm using FME 2022.2. If I pick an attribute from the arrow drop down menu, it inserts it in the DateTime Attributes field as @Value(CREATETIMESTAMP). When the transformer runs, it doesn't convert the value. But if I select the attribute from the 3 dots menu, it's fineIn the 3 dots menu chooser, there are sort of two sets of the attributes....
Hi all,I am experiencing an issue with dataset fanout in my workbench after upgrading to 2025.2.0.0 with Build 25810 (WIN64). The workbench worked perfectly in 2024.2.The workbench reads in a FGDB file 30,000,000 features using a reader, then performs spatial filtering with coordinate grid tiles and fanout into geopackage files according to the tile names using a writer. The workbench always stopped while outputting the first geopackage file with the following error message. Insufficient memory available -- error code was 2 - please read the FME Help section 'Adjusting Memory Resources' for workarounds.The workbench worked fine when I set the max features to read in the reader as 100,000, but the writer failed once the reader read all 30,000,000 features. I tried lowering the number of features per transaction in the writer but it did not work. Is anyone else experiencing a similar memory issue with dataset fanout in 2025.2?Thank you for your help in advance.
Hello everyone,I have a workspace app that converts a .dgn file to a .gpkg An user tied to run the tool with a dgn file, but he received a failure message. I looked at the job and I saw the following errors 464 OGCGEOPACKAGE writer: Invalid curve: not enough points 754 OGCGEOPACKAGE writer: Invalid curve: not enough points 872 OGCGEOPACKAGE writer: Invalid curve: not enough points 957 OGCGEOPACKAGE writer: Invalid curve: not enough points 1051 OGCGEOPACKAGE writer: Invalid curve: not enough points Did anyone have similar errors and what can I do to solve these errors?
I believe there’s an issue with the tolerance values greater than 0.5 not being respected when using the NumericRasterizer transformer.With tolerances like 0 and 0.5, I can see a difference in output. However, a tolerance value of 1.0 has no change in the output. Output from NumericRasterizer with tolerance set to 0.0, with the input vector overlayed Output from NumericRasterizer with tolerance set to 0.5, with the input vector overlayed Output from NumericRasterizer with tolerance set to 1.0, with the input vector overlayed Going by the documentation, it explicitly states that a tolerance of 1.0 will draw all pixels touched by the input vector line, which can been seen above as not being seen in the output. Work flow and NumericRasterizer parameters Documentation of the tolerance parameter from the corresponding version of FME Flow I have test values such as 0.7 0.8, 0.9, and 0.999 with no luck. Does anyone know if this is intentional or a bu
How can one rasterize a vector where all cells touched by a polygon/vector get drawn during rasterization?I have tried the NumericRasterizer, but I believe there’s an issue with tolerance values greater than 0.5 not being respected when using the NumericRasterizer transformer.With tolerances like 0 and 0.5, I can see a difference in output. However, a tolerance value of 1.0 has no change in the output (see below). Output from NumericRasterizer with tolerance set to 0.0, with the input vector overlayed Output from NumericRasterizer with tolerance set to 0.5, with the input vector overlayed Output from NumericRasterizer with tolerance set to 1.0, with the input vector overlayed Going by the documentation, it explicitly states that a tolerance of 1.0 will draw all pixels touched by the input vector line, which can been seen above as not being seen in the output. Work flow and NumericRasterizer parameters Documentation of the tolerance parameter from th
I have opened an old workspace and all of the transformers in that workspace look like the first AttributeCreate in my screenshot. The Input port is hidden.When I copy the AttributeCreator, it looks different. The Input port becomes visible. It looks like the second AttributeCreator in my screenshot. When I add new transformers, they all have a visible Input port. When I open a completely new workspace and add transformers, they also have a visible Input port.How can I hide the Input port like on the first AttributeCreator in my screenshot?It seems like this cannot be achieved for any newly created transformers and workspaces. It only looks like this in this one old workspace of mine.There is an option under View that makes the Input ports visible on all of my Transformers. But it’s impossible to reverse that action. The option to make the Input ports visible disappears, once they are visible on all transformers. There is no option to reverse that. It can be reversed only by clicking t
I need to connect to a PostgreSQL database via an SSH connection. How can I do that ?Thank you
I had a hard time with the integration of GDAL and FME raster transformers. So, for the record, I posted a short tutorial:https://blog.fiorino.fr/index.php/2022/05/11/fme-python-api-working-with-rasters-and-gdal/
I need to update environmental data in our system. The input data comes from an ESRI Shapefile, and I am using FME 2021.2. The output is stored in Oracle Spatial.However, the system that reads the data from Oracle only accepts simple polygons. When I try to insert multipolygons, part of the geometry is ignored by that system.Is there a way in FME to convert multipolygons into single polygons while maintaining a 1:1 relationship between input and output features?If I use the Deaggregator, the number of records grows exponentially, which is not viable for our database.I also started working with FME recently, so I may be missing a simpler or recommended approach.What would be the recommended way in FME to transform multipolygons into valid simple polygons without increasing the number of features?
My long time transformer friend FMEFlowLogFileRetriever is not avaliable in FME 2025. The error asked me to use HTTPCaller to retrieve the job log file. So I used this rest point in the HTTPCaller’s Request URL with GET - https://my-fme-flow-hostname/fmerest/v3/transformations/jobs/@Value(jobid)/log. I tried the Basic and Web Connection authentications, but both gives the error “HTTPCaller (HTTPFactory): An error occurred while accessing the URL 'https://my-fme-flow-hostname/fmerest/v3/transformations/jobs/98/log'”. I have no problem publishing using the same Web Connection for the authentication, so don’t understand what the issue is. The jobid = 98 was run successfully on the FME Flow.
Hi,In a workbench, I created a Custom Transformer to loop batch of features.It is a workaround to prevent machine reboot on large feature datasets processing. At least, writing-along features in the Output to .csv. file help to recover features processed in PythonCaller and it is easier to resume for the remaining features… Here is the workbench, with Inputs:batch = 10counter = 54000 (total of features)loop = 0 (to keep track of iteration)offset = 0The Tester_counter transformer filter out batches 10 features at each iteration, sent to PythonCaller. At the end of processing the Tester_Loop tests if there is remaining features to process. Anything processed is sent to the Output and written in a .csv Writer transformer in the ‘Main’ workbench, whereas ‘failed’ remaining features from Tester_counter have their attributes incremented for variables loop and offset for the next iteration.However, when running the workbench, the output .csv file is created but kept empty. Also, each iteratio