Skip to main content

Transformers

Using transformers to modify data.

  • 12,962 Posts
  • 51,736 Replies
10000 Posts
deanhowell
Influencer
deanhowellInfluencer
asked in Transformers
JSON Templator for Parameters

I am reading in a very poorly formatted JSON file which has hundreds of lines but I just want to extract some key attributes that will then be used to create a JSON file that can be passed to FME Flow to start a work bench using the extracted attributes. I am reading the JSON file in and running it through an attribute manager to get the attributes I need. If I then run through JSON writer I just get a list like this [ { "json_featuretype" : "JSONFeature", "Catalogue Key" : "Data Catalogue", "Layer Name" : "Testville Roads", "Data From" : "Import type from CSV file", "Data Access" : "Restricted", "Update User" : "Alex", "Description" : "Roads data for Testville COuncil", "Dataset Dimension" : "3D", "Data Update Frequency" : "As needed", "Geometry Type" : "Points", "Spatial Presentation Type" : "Vector", "Data Currency" : "2025-11-09", "Initial Publication Date" : "2025-04-15", "Contact" : "Fred Flintstone", "Data Classification" : "Business Impact Levels (BIL)", "Dataset Aggregator" :

172
deanhowell
Influencer
albinepro
Supporter
albineproSupporter
asked in Transformers
Debug Python Scripted Selection

Im trying to implement some dynamic selection based on an API call response. So far I can build a package for scripted selection according to Dynamic choices using Scripted Selection - FME Packages SDK Guide and debug in VS Code attaching to Workbench (Build a Hello World package - FME Packages SDK Guide).The problem is that debugging is bound to running the translation and that is already late for the script I need to debug. The selection scripts are run when filling in the transformer parameters. I need to add breakpoints and inspect the execution of selection script (ScriptedSelectionCallback) during the API calls and browsing the object structure. Is there some setup to make this possible?The final purpose is to get this into User parameter scripted selection in FME Flow. I understand it would work this way in scripted parameter selection as it is in the transformers.

454
albinepro
Supporter
mgrupp
Participant
mgruppParticipant
asked in Transformers
ArcGIS portal feature writer fails to fill certain fields when run on FME Flow

Hi everyone, I have a problem that I can't seem to solve. I have a workspace that writes data from a PostGIS database to a feature layer on our portal using the ArcGIS portal writer. When this workspace is run locally on FME form, it works perfectly fine. When it is uploaded to portal and run there, it seems to work fine (the amount of features written is identical to the amount when run locally), but some important fields are not filled. Flow and Form both use the same web connection with the same authorization parameters. Why is it that a identical workspace that runs locally can fill all fields fine, but when run on portal it seems to not be able to fill these fields??? I have put testfilters and loggers in between to check if the fields actually have values and they do, so it must be a writer quirk. Field names between the features and writer are identical as well (all lowercase) so I am at a loss... I am using fme form/flow 2023.2.2 and ArcGIS portal 11.1

563
artw
Participant
fme_can_do_it
Enthusiast
fme_can_do_itEnthusiast
asked in Transformers
adding points to the line

Hi, I have a task in which I have station values attached to the line. I need to select station values and split the line at the station points. Some station points are easy to split; for example, 1789+00, but others, like 1201+50, are not that simple. I am trying to create points between two station points. I want to place points 10 ft apart and calculate station values so that I can split the line at those values. I have attempted to generate points between two station values using the Densifier transformer, but I am not achieving the desired outcome. If someone can assist me in this situation, I would greatly appreciate it.

645
fme_can_do_it
Enthusiast
3d_ff
Contributor
3d_ffContributor
asked in Transformers
FME Python Script for Automated DDH Desurveying (Minimum Curvature, Flexible Intervals)

​@daveatsafe Dear FME Community, I am seeking assistance in developing a Python script for use within an FME workspace to automate the desurveying of diamond drill holes (DDH) using the minimum curvature method. I aim to create a flexible solution that allows users to specify a custom desurvey interval (e.g., 1m, 3m, etc.). Background: Desurveying a drill hole involves calculating the actual XYZ coordinates along its length based on the collar location and survey data (azimuth and dip at regular downhole distances). I've researched existing methods and found the following resources helpful: Understanding Drillhole De-surveying: https://seequent.zendesk.com/hc/en-us/articles/360001547955-Understanding-drillhole-de-surveying Desurveying Methods and Interactive Tool: https://endarthur.github.io/writting/desurvey.html Drillhole Desurveying Python Code (GitHub): https://github.com/WilliamsB39/Drillhole-Desurveying QGIS Geoscience Plugin: https://www.spatialintegration.com/processing-drill-h

364
3d_ff
Contributor
fikusas
Contributor
fikusasContributor
asked in Transformers
Compress FGDB with ArcGIS Pro tool in FME

I’m trying to compress FGDB with ArcToolbox tool with PythonCaller. This gives no error, but nothing happens after running it. What I am missing? Using FME 2024 with ArcGIS Pro 3.4. Compatability is set to ArcGIS Python 3.11. import fmeimport arcpy def compress_fgdb(fme_feature): fgdb_path = fme.macroValues['FGDB_PATH'] arcpy.management.CompressFileGeodatabaseData(fgdb_path, "Lossless compression") return fme_feature

171
hkingsbury
Celebrity
gennine
Contributor
gennineContributor
asked in Transformers
How to automatically reverse digitizing direction of an electric network

I would like to automatically number an electrical network as shown in the diagram below.If the digitization direction of the features is correct, then I get the correct numbering. I used the NetworkTopologyCalculator and TopologyBuilder transformers to perform the automatic numbering.If the direction is reversed, then the numbering is incorrect.Is there a solution to this problem?Thanks in advance.

242
gennine
Contributor
galigis
Enthusiast
galigisEnthusiast
asked in Transformers
AzureBlobStorageConnector / Clip LAS file without Download

Hi All, I’m working on a workflow that reads a LAS file storage as blob in a Azure database. I need to clip the LAS file with a polygon (AOI) and then exports the result as LAS file (or other format). After doing some research, the community suggests to set the My AzureBlobStorageConnector settings to download the LAS file and then process the data. My current LAS file is around 10GB and downloading it every time that needs to be clipped is not very good! The total time of downloading the LAS, clip it and re-export it can be crazy! here are my settings: Is there any way to read the LAS file as blob and clip it directly without download it?

493
kailinatsafe
Safer
tb09114
Supporter
tb09114Supporter
asked in Transformers
Challenges with Python in FME 2024.

I have an issue with some Python code in Form 2024.1. I have to make a connection to a PostgreSQL database using psycogp2. When I run the PythonCaller in my workspace then it fails with an error message that psycogp2 is missing (Python Exception <ModuleNotFoundError>: No module named 'psycopg2') I tried to single out the problem in a minimal workspace (attached), but here it works without a problem. Hence, I am wondering how and what is causing the issue. The actual workspace is rather big and I will share a screenshot of the essential part that is failing. The AttributeManager10 receives a single feature from the summary port of a FeatureWriter and in AttributeManager10 all other exposed attributes are deleted, except the site_id (integer) is kept. Unexposed attributes are there only two: fme_geometry -> fme_undefined fme_type -> fme_no_geom Running to the Attributemanager10 works fine. The next partial run of PythonCaller_4 fails with: 2025-04-30 13:46:19| 3.6| 3.4|ERROR |Message Typ

222
david_r
Celebrity
nicholas
Contributor
nicholasContributor
asked in Transformers
How do I relate the version number of a Transformer to the Version number of the software?

Hi All, I have set up my tool tip to tell me the version number of a particular Transformer in my workbench. Is there a way of relating the version number of a Transformer to the version number of my software? Can I find out which version of the software that a particular Transformer has an upgrade? For any particular software version, is there a list of the version numbers of the Transformers? Thanks, Nick

240
nicholas
Contributor
bromp
Contributor
brompContributor
asked in Transformers
Esri Portal Connection for Survey123ReportGetter

Hi there, I am trying to generate a survey123 report using FME. I have downloaded and added the custom Survey123ReportGeter transformer however I am using a portal connection instead of an AGOL connection: After this step i have the Tester that checks for the submit_response_body beginning with {"jobId" however the flow always fails here. I get an invalid token message in my submit_response_body: {"error":{"message":"498: Invalid token.","code":498},"success":false} I am not sure if the request URL needs to be different for portal? or if the token i am providing in my query string parameters is not being read in properly… Any help would be greatly appreciated. This is what my transformer looks like:

342
bromp
Contributor
jomaura
Observer
jomauraObserver
asked in Transformers
Sharepoint Online Connector : Create subfolders

I am encountering difficulties when using the SharePointOnlineConnector transformer. The flow is as follows: I have an Excel file listing a series of files to download from the BOX service. If the file is not yet downloaded locally, I download it to G:\workspace\tmp. I then modify the relative path by removing the filename: Example: /xxx/xxx/xxx.zip becomes /xxx/xxx I upload the locally downloaded file to the target SharePoint site, using the modified path as the destination. I successfully managed to upload around 700 files out of 3600 because facertains numbers of folders already exist into the sharpeoint site.However, I am now consistently getting errors such as: UPLOAD_ERROR Could not find the folder. It seems that I might need to create the subfolders first. What would be the best way to proceed from now? Thank you in advance for your help!

141
evieatsafe
Safer
cg-wcc
Contributor
cg-wccContributor
asked in Transformers
String Search to match feature sets

Hi, Pretty new to FME, so hopefully I have all the terminology right and I’m asking the right sort of questions. I have a feature set from Freshservice API of the available products in their asset system.It’s mostly the Latitude and OptiPlex products I need to work with so the solution can work just for them. I need to match the freshservice_product_name to a feature set that I don’t control and isn’t consistent. As you can see the tesma-description is a bit of a mess but it’s all I have to try match up the freshservice_product_name to. Can someone point me in the right track of how they would tackle this? Thanks,

464
cg-wcc
Contributor
smishsmosh
Participant
smishsmoshParticipant
asked in Transformers
Workflow for transposing attributes into rows

Hello all, Complete FME newbie here I need to transform the data compiled by a different department as an excel spreadsheet to look like this: From Initial structure To Final structure Basically the idea is to pair each of the projects with each of the areas in order to then join the final table to a different spatial table on the ‘AREA’ attribute, ideally without manual input into transformers. The project attribute will have new entries added to it, with corresponding counts, the areas will remain the same. Thank you.

322
smishsmosh
Participant
mohamedalsobh
Contributor
mohamedalsobhContributor
asked in Transformers
FeatureJoiner MISSING_LEFT_KEY

I'm trying to join two Excel files that share a common column called "ID". File (A), which I consider the main file, has 7 columns. File (B) has 16 columns, but I only need one column from it, called "NON", to be added to file (A). I tried using the FeatureJoiner, but it didn’t work. I also used the AttributeKeeper to filter the data, but the result is always null. Note: The column titles in both Excel files are written in Arabic. is this could be the reason?

524
mohamedalsobh
Contributor
gogopotter90
Contributor
gogopotter90Contributor
asked in Transformers
How to split strings then concatenate them ?

Hello , I have four tables. Each table includes an attribute called result, which contains fields value string string number. (Hello Community 30 ) ,all tables share a common attribute id I would like to combine the output from all four tables into a single attribute called result String String String String String number String number String number. String no. I would like the first string from each table to always appear above and the second string and number written on the next line and All values should be combined into the same field. Thanks on advance

553
takashi
Influencer
norris
Contributor
norrisContributor
asked in Transformers
How to serve folder/files created by SystemCaller (python) in FME Flow?

I am using SystemCaller to run a pyton script (uses conda env) that creates a bunch of files and I am trying to figure out how I can get FME to then read the folder location and serve that to the user as a .zip file. The folder location is an attribute that I use to pass to SystemCaller but I cant figure out how to pass this attribute value to say a FileCopy or even a FeatureReader. The output folder is a concatonated path $(FME_WORKSPACE_DIR) + _output that I made as a new attribute value (_output_folder). In the pic below, I tried to connect the Logger to FileCopy but that doesnt seem to do anything here is the FileCopy settings i am using. Am I doing this correctly or any other suggestions?

423
norris
Contributor
gogopotter90
Contributor
gogopotter90Contributor
asked in Transformers
How to avoid Manual X/Y Adjustments in PDF page formatter

Hello , Is it possible to control the position of Upper left x and Upper left y in a PDF formatter by setting their values in an attribute beforehand, so that the PDF formatter reads and uses them directly? I don’t want to keep adjusting the x and y positions directly inside the PDF page formatter every time I find that the output is not properly aligned. Do you have any tricks or suggestions to help me arrange my PDF layout more quickly?It takes a long time to design the PDF layout . Thanks in advance .

501
debbiatsafe
Safer
rickarhayd
Participant
rickarhaydParticipant
asked in Transformers
Dynamic Attribute Renaming using a Mapping Spreadsheet

Hi all, I am querying an API to download a large number of tables (50+) as CSVs that will be written into a DB. I have automated the process of downloading the tables and dynamically writing them to the DB, but the default/source attributes the tables come with when downloaded are messy/poor quality. One attempt at this I’ve tried is to set up a mapping spreadsheet for each table with an example shown below. I have been able to use a SchemaMapper and dynamic CSV writer to write a single table with the updated attribute names, but I am wanting to do this dynamically using a WorkspaceRunner to automate updating the attribute names for all the tables, while also reading and writing records to the DB. Does anyone have any advice for this? The SchemaMapper is not able to take parameters/attributes to read the mapping spreadsheets so it does not seem like an option.

251
takashi
Influencer
chavdar
Contributor
chavdarContributor
asked in Transformers
Read specific table from Microsoft Database

I would like to open and read a .accdb file which contains multiple named tables. Instead of reading all tables I just want to select the ones which names match the ID criteria. The example below works fine by using a Microsoft Access reader which compiles all tables into one single merged feature type. After that I compare the source CSV file name against one of the listed attributes and pass forward only the matched lines. That method worked well and served its purpose, but the downside is that it is way too slow since it is reading the whole accdb and this file is constantly growing because we keep on adding tables over time. I have tried to use the Feature Reader with an initiator which provides the route ID as a key to search an open table if name contains the ID but with no luck. I have tried to add a SQL WHERE statement but still no luck. Once again, my simple aim is to pull out tables from a accdb which names meet the ID criteria.

408
takashi
Influencer
tb09114
Supporter
tb09114Supporter
asked in Transformers
How to transpose an attribute table?

I need to transpose an attribute table and tried so far the StatisticsCalculator as well as the AttributePivoter, but do not get the result I need. In addition I was following the advice of Mark (@mark2atsafe​ ) in this posting at StackExchange. While the advice from mark looks 'straight' forward, I struggle with the reader-as-resource since the data are produce through analysis within the workspace where I need to transpose the attributes. After sorting my results I get the following attribute table and what I need is a structure like So, the attribute values from the 'label' column have the provide colum names, while 'Area_perc' has to provide the corresponding values for these columns, and everything is grouped by 'Sectorname'. How can I achieve that within FME?

5154
pgammie
Contributor
sijmensp
Contributor
sijmenspContributor
asked in Transformers
stringcasechanger upgrade

Currently I am testing the upgrade of FME Form 2024 (from FME form 2022) for my electrical company. When doing auto-update I found out that my stringcasechanger set on lowercase were updated to UPPERCASE. The same behaviour occurs when you update the transformer manually and choosing the default setting. It would be better if it can take over the old setting.

563
jovitaatsafe
Safer
agelfert
Contributor
agelfertContributor
asked in Transformers
VertexExtractor not working in FME 2024.2

As mentioned in another post, I'm currently upgrading a bunch of workbenches to run on FME Flow 2024, some of them many years old. Typically, for anupgrade, I will go through each workbench, upgrade all transformers, readers, and writers, then run it in test desktop (workbench) environment, then publish to and test in a test server (flow) environment. In, FME 2024 the Vertex Extractor gives me bogus results, even after upgrading all the constituent transformers in this custom transformer. It looks like X and Y, or Lat and Lon, are reversed. When I open the CoordinateConcatenator (arrow above), I see this: Default setting While I haven’t dug through the rest of the custom transformer, this - frankly - looks counter-intuitive. When I change that to “XYZ”, everything runs just fine and results are as expected. My tweak I have tried to compare my version (I’m assuming that upgrading all components makes it uptodate) to older versions to see whether upgrading might have introduced this prob

857
DanAtSafe
Safer
ryan_k
Contributor
ryan_kContributor
asked in Transformers
Updating M measures

I have been able to extract polylines from esri fgdb, combine the lines into one record but I cannot seem to figure out how to now recalculate all the M-measures in that line. In arcmap I would manually use the Route Measure editing tools, Set As Distance to 0 and Apply Factor of 0.00018939 to recalculate the line M-measures in miles for the line. I couldn’t seem to get the measure transformers to get the result I was looking for. This was probably user error. Thanks, Ryan

292
ryan_k
Contributor
hnahoulder
Contributor
hnahoulderContributor
asked in Transformers
Attributes key and value to list

Hello, I need your help. I'm looking for a workflow to transform my keys and values into a list. Here is an example: I have this table: Key Value a 1 b 2 c 3 a 4 b 5 c 6 a 7 b 8 c 9 and I wanna transform it to a b c 1 2 3 4 5 6 7 8 9 Thank you

543
hnahoulder
Contributor

Helpful Members This Week

Badge Winners

Show all badges

Popular Tags

Community Stats

31,140
Posts
118,066
Replies
38,760
Members

Latest FME

Terms & ConditionsCookie settings

Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.