Skip to main content
  • EN

Transformers

Using transformers to modify data.

  • 10,000 Posts
  • 52,686 Replies
10000 Posts
cs_vt_psd
Contributor
cs_vt_psdContributor
asked in Transformers
Generic source file name confusion? Or bad workflow?

I’m working through a data cleaning process in a workspace. I have multiple files that are different from each other in schema and attributes, but are essentially all saying similar things. Hence the cleaning. I’m working in one workspace to make re-use of transformers a little easier, but when I’m using the FirenameParkExtractor transformer I need to identify a user parameter source for it to gather up a name to chop up. This is what that dropdown looks like: The names here do not align with how I’ve named the readers and associated files so I’m a little confused about how to make this situation easier to work with. I’d like to be able to rename the source datasets, but I’m not sure where I’d do that. Can you tell me? As an aside, is it a bad idea to process many things in one workspace? Should each file get it’s own? Is there some helpful documentation out there about designing workflows? (I’ll be searching on that just now, but I thought of the question as I was writing the post!) H

112
cs_vt_psd
Contributor
ebygomm
Influencer
ebygommInfluencer
asked in Transformers
Dissolver - Attributes to Sum and Multi Polygons:1+2 = 5

Not sure if it’s an issue as such, although I struggle to think of a scenario where you would want this functionality, and it might be obvious to some but just as an FYI If you send multipolygons into a Dissolver with Aggregate Handling set to Deaggregate and choose attributes to sum, the polygons will be deaggregated first, this means that each constituent part inherits the attributes of the original multipolygon feature and this will then be summed to leading to unexpected values. So from thisYou get this 

304
DanAtSafe
Safer
S
sanashaikh
asked in Transformers
Python Exception <AttributeError>: 'NoneType' object has no attribute 'name' I am facing this error, while reading the feature from dataset. This is coming in Feature Reader. Using latest FME version 2023.0.

5146
davechoi
Safer
kevinkies
Contributor
kevinkiesContributor
asked in Transformers
Create date segments of two table with overlap of times

Hello,I have two tables with the same field names. Both tables have 2 date fields, with a start date and a end date and other fields which has the same name in both files. The only thing is that the date segmenst (time between start and end) has a overlap.Example:TABLE 1:DATUM_INGANG DATUM_EINDE ID REPLACER 19921101 20040119 0502010000027183 OWNER 20040119 20160930 0502010000027183 OWNER 20160930 99991231 0502010000027183 OWNER  TABLE 2: DATE_START DATE_END ID REPLACER 19921101 20060716 0502010000027183 USER 20060716 20061008 0502010000027183 USER 20061008 20070220 0502010000027183 USER 20070220 20100616 0502010000027183 USER 20100616 20110709 0502010000027183 USER 20110709 20111001 0502010000027183 USER 20111001 20190325 0502010000027183 USER 20190325 20190617 0502010000027183 USER 20190617 99991231 0502010000027183 USER  I want to combine them to one table, this should be the outcome: DA

17625
takashi
Celebrity
ronmak
Contributor
ronmakContributor
asked in Transformers
How do I test/filter attributes using a list with wildcards?

I am processing a large amount of simple data: Points with attributes. I want to filter the data by a specific set of attributes, some of which have wild cards. e.g., "T", "T BASE", "T BENT" should all pass through. For each run, a user will be specifying the valid attributes by putting them into a file (probably text), which will be read in at the start of the process. I mention using a list, but they could go into a table as well if that would work better.

3369
aquee
Contributor
davidlee
Supporter
davidleeSupporter
asked in Transformers
Creating a new feature class in a GEODATABASE from an ARCGISFEATURES source fails

This issue seems to appear again in FME 2025.1! 

470
davidlee
Supporter
simonlr
Contributor
simonlrContributor
asked in Transformers
How to see which features have invalid source datasets when using a FeatureWrite?

Hi, i am copying files using the FeatureWriter with FileCopy and but some of them are missing but i dont get an attribute telling me which files are missing. All i get is this in the log:File Copy Writer: Skipped 401 features that contained invalid source datasetsBut what i would like is to route those with invalid source data set and have them write to an excel ark or similar so i know which files cant be found.But right now the features all come out of the same port with no distinction of wheter or not the copy was actually succesful. Which is not very useful for me. Any help is very much appriciated :)

536
takashi
Celebrity
bkgg
Contributor
bkggContributor
asked in Transformers
Simple arithmatic problem

Hey guys, So I ran into a problem I shouldn't have. For my workspace I'm trying to subtract two integers from one another. The problem that I'm facing is that the arithmatic expression is not giving me any usable data. This is also the case when slightly change the expression. Below is the most recent attempt. Can anybody point out how I might be able to make this work?I have performed similar expressions in older versions of FME with no problems whatsoeverI I'm currently running FME 2024.2.4.0. The data type is originally Real64 but I converted it to Int32. Any help would appreciated. 

603
bkgg
Contributor
georgiec
Contributor
georgiecContributor
asked in Transformers
Aggregator grouping by attribute removes overlapped areas

I have land parcels that are currently separate features in ArcGIS SDE.  Below highlights 2 parcels that that belong to the same property 1st and 2nd floors.  (2nd floor is the smaller shape)I want to aggregate these into a multi part single feature grouped by an ID that they share called LANDNO.After using the aggregator transformer, it looks like the below where the second floor has been totally cut out leaving nothing.  I haven’t tried a Disslove yet but I reckon that would remove all vectors of the second floor which I still want to keep.Here are the settings in Aggregator.   Any suggestions to achieve my objective ? 

748
georgiec
Contributor
georgiec
Contributor
georgiecContributor
asked in Transformers
Retain trailing zero on coordinate value in text editor

I’m building a value for the Longitude of a point feature. The field type I have defined is number (20,6) I create the value using this in the text editor:   @Format(%.5f,@XValue())So I want 5 decimal places.   eg.     15436225.30510However, what I see in my result table viewer is 15436225.3051   (no trailing zero)But when I double click on that value, it displays the value “with” the zero.  15436225.30510 The value that gets written to the writer (to AGOL) is the one without trailing zero.Works fine until I use a change detector, then it decides that the trailing zero is causing a difference and wants to update the value to have the trailing zero.So my question is,  can I enforce the trailing zero in the table result?  Then I will rewrite my destination records one time again and hopefully my change detector will realise they are the same from that point on.   My original format statement again is :  @Format(%.5f,@XValue())

302
hkingsbury
Celebrity
coopersykes
Contributor
coopersykesContributor
asked in Transformers
How to get a list of Asana tasks with their corresponding custom field values?

I have been able to successfully connect to Asana by using two resources (attached at the bottom) and extract a list of task names and custom field names, but I have been unsuccessful in using the JSONFormatter, JSONFlattener, and the JSONExtractor tools to create a list of tasks and their corresponding custom field values for a specific Asana project. If anyone is experienced in working with the Asana API to collect this kind of data from a particular Asana project, I’d be really appreciative of any helpful tips to create a final output of a project’s tasks and their custom fields. Thank you so much!https://hub.safe.com/publishers/safe-sandbox/web-connections/asana)

802
coopersykes
Contributor
markcoopersdc
Contributor
markcoopersdcContributor
asked in Transformers
Problem loading multiple AGOL feature services with workspace runner

HiI am doing a comparison of multiple AGOL feature services and am using a workspace runner to iterated through the feature services. I have created a published user parameter for the feature service name, which is populated by the workbench with the workspace runner. This appears to work fine but the problem is I cannot seem to make the layer name a parameter so it keeps reading the layer name from the feature service that the agol reader was set up with originally.Does anyone know a way round this please? Or perhaps it is resolved in a newer version of FME desktop (Im currently using 2023.1)?

260
markcoopersdc
Contributor
stadimarmiladi
Contributor
stadimarmiladiContributor
asked in Transformers
Using one AttributeRounder for different accuracies

Hello all,is it possible to use one AttributeRounder to round different attributes to different accuracies?Let’s assume i have two attributes, A and B. I want to round A to 2 decimal places, whereas i want to round B to three decimal places. Is this possible with one AttributeRounder or do I need to use the AttributeManager.Cheers!

351
max_h
Supporter
aron
Supporter
aronSupporter
asked in Transformers
How to pass User Parameters to WorkspaceRunner?

 I have a parent WS that runs a load of WorkSpaceRunners. I want to pass on some User Parameters from the parent to the “kids”, see settings below. The child workspace is already setup with these User Parameters and run fine with them when run on their own. I have not been able to find any documentation on this, so the settings below are my best guesstimate. My steps are User Parameters in parent -› User Parameters in workspacerunner settings -› User Parameters set up in child workspace settings -› use them in child workspace.So, how do I fix this?Does the child WS need to be set up in a special way in order to pick up the incoming parameters? Is there a misstake in the syntax? Anything else I can look into?  

33912
bill7875
Contributor
markcoopersdc
Contributor
markcoopersdcContributor
asked in Transformers
How to compare multiple AGOL Feature Services

HiWe are migrating from one AGOL portal to another (with regional hosting). I have used the AGOL connector to list all service definitions of hosted feature services, download/upload the .sd files and publish the service on the new portal. This appears to have been successful.However, I am now trying to compare the feature services in each AGOL account and am struggling. I want to iterate through each feature service and compare it with the the one on the other account using the Change Detector. I have used the AGOL connector to list all the feature services and fed that into a feature reader. Unfortunately, all the features just come out the generic port without any schema so it seems impossible to compare the feature services. Does anyone know a way this could be done please?Also, I need it to finish with one feature service before it moves on to the next one. Is this possible?Many thanks 

311
crutledge
Influencer
mwoods
Contributor
mwoodsContributor
asked in Transformers
1019 error with change detector and polygons

Hi all. I am just trying to do something very basic but keep getting a “1019 error. The specified feature could not be updated or does not exist” when using the update feature writer.This only happens with polygons and lines/polylines but works fine with points.The workbench below is taking a polygon boundary dataset from portal, taking a polygon from AGOL putting it through a change detector - whose unique identifier is a name in both features- then updating/inserting/deleting when necessary to essentially make the AGOL layer the same as the portal layer….Or in theory this is what i want it to do. Any help would be much appreciated.  

364
mwoods
Contributor
don_weaver
Contributor
don_weaverContributor
asked in Transformers
AttributeRenamer leaving missing fields in

When ‘Remove Output Attribute’ is selected for ‘If Input Attribute is Missing’  The missing attributes (the attributes with no data) are still present in the output.  Any clues? 

1068
crutledge
Influencer
A
alan123_
asked in Transformers
Error: Geodatabase: In Feature Type 'd_y_n', must specify 'fme_match_columns' when 'fme_selection_method' is set to 'MATCH_COLUMN

Hi there, I got a weird error when I attempted to use featurewriter transformer to write feature into SDE dynamically (see the error screenshot below). Could someone please explain what is the error is talking about? Is there any solution?Thanks in advance for your answerAlan

962
jobvdnoort
Supporter
mst
Contributor
mstContributor
asked in Transformers
How to write scientific e notation number as a decimal real number to .CSV

I’m using an FME FeatureReader to read a content of an Excel file containing various values, including decimal numbers, some of them small with more then 5 decimal zeroes.The FME, apparently, handles these small numbers using scientific e notation.After processing the data, I write it into a CSV file using a FeatureWriter. The problem is, that the small numbers are written in their scientific notation form, and not as real decimal numbers. This is causing an issue further down the line.I tried various things to force the FME to write the numbers as decimals:forcing datatype of the affected attributes in the FeatureWriter to Decimal or Real using AttributeManager to change the datatype of the affected attributes to Real, Decimal and even String forcing the datatype of the affected attributes in the FeatureWriter to String changing the datatype of the affected cells in the Excel file to TextNothing helped.I tried to find a String or Math function that would achieve that, but I’m not sure

5004
christian_dan
Contributor
connecter
Contributor
connecterContributor
asked in Transformers
NeighborFinder loop necessary?

Hello,unfortunately I do not know how to solve the following problem. I have many points and lines and would like to create a connection to the line layer for the points. The lines should not be drawn directly to the starting lines as in the second case, but an iteration should take place. Here is the process as I would describe it:1. a point is taken and the closest point to the line layer is determined and this line is also created2. the new line is added to the existing line layer3. the next point is taken and the closest point to the line layer is determined. The new line layer may result in a shorter route than the original layer.4. the new line is added back to the original data set, etc.So far I have tried various workflows with a loop, but unfortunately I have not gotten very far. I would be very grateful for any ideas and suggestions.

705
hkingsbury
Celebrity
kyp4810
Contributor
kyp4810Contributor
asked in Transformers
How to apply geoid when converting ifc to cesium 3d tiles

 How to apply geoid when converting ifc to cesium 3d tilesHello, I am converting ifc to cesium 3d tiles. The photos below are the photos that were converted. If I do not apply geoid, the conversion is done well, but if I apply geoid, the following error occurs. In the Vertical option, Geoid Height Grid, .gdc, .gtx, etc., the same error occurs no matter what file I put in. Is there any solution? The operating system is mac, and the corresponding fme version is 2023.2.  

543
daveatsafe
Safer
virtualcitymatt
Celebrity
virtualcitymattCelebrity
posted in Transformers
CustomTransfomers: A word of warning (Parameter Usage)

Likely known by many here, however, I can’t see it documented officially so I’m going to spell it out.If you have created a CustomTranformer or are working with one you need to be very careful how you use/reference parameters.There are many gotchas when working with CustomTransformers but this is the one that trips me up the most. If you define a parameter in a Custom Transform with an attribute value this will not always be treated as a Global Parameter inside the transformer. I.e., depending on what happens in the Transformer the value will not always be accessible.I saw a great comment by ​@ctredinnick in this thread from a couple of years ago: where they mention that Parameter Values defined by an attribute value will be treated in a similar way to unexposed attributes.I had always assumed that Parameter values were somehow ‘locked in’ of defined for every feature when they went in but that’s not true. Anytime the parameter is referenced inside a CustomTransformer it will just reso

593
alexbiz
Influencer
jeff.butler
Contributor
jeff.butlerContributor
asked in Transformers
NeighborFinder output with multiple candidate have same Measure value

Greetings all!I searched for similar questions but wasn’t able to find any pertaining to this, so apologies if I missed them. I’m using the NeighborFinder transformer to locate points along measure-aware polylines to acquire the Measure value of all lines within 5 meters and merging all attributes and generating a list when multiple candidates are found. When there are multiple candidates, it’s assigning the Measure value of the point from one of the lines at random (possible the closest?). However, when I use the ListExploder transformer to create features from all elements in the list, while all other attributes appear to be specific per feature, the Measure values are the same for all features spawning from the same feature. Hopefully this make sense. I assume I have a parameter set incorrectly or something, but how do I ensure each feature has the measure value specific to the line it was matched to?

663
jeff.butler
Contributor
karnil
Contributor
karnilContributor
asked in Transformers
Are there best practices for shrinking LoD 3 exports to CityGML in FME

I’m exporting LoD 3 models from FME to CityGML and the files blow up in size. FME writes every Appearance on each triangle and also makes a separate position list for each triangle, everything becomes separate XML elements, so the markup absolutely dominates the file. I think CityGML should be able to store things more compactly at the object level, but I don’t see a way to make FME do that automatically. Every triangle whould not need its own position list right? I have thome different thoughts on the matter: Can FME group or merge geometries/appearances to cut down the markup? Can ImplicitGeometry be used effectively without creating a complex monster workspace? Is CityJSON fundamentally a better route? (I’ve heard numbers around ≈ 1/6 of the file size.) Are there external tools that post-optimize a CityGML without losing semantics? Concrete tips, transformer setups, or example workspaces are very welcome.

572
kailinatsafe
Safer
ayoub.salama
Contributor
ayoub.salamaContributor
asked in Transformers
3D points with attributes (shapefile) to Cesium 3D Tiles

Hi everyone,I'm working with a shapefile that contains 3D point geometries, each with several attribute fields. I would like to convert this dataset into Cesium 3D Tiles format while preserving both the 3D coordinates and the associated attributes. Has anyone successfully done this using FME? If so, could you share your workflow or recommend which transformers and settings are best suited for this task? Any tips or guidance would be greatly appreciated!

542
ayoub.salama
Contributor

Badge Winners

Show all badges

Community Stats

31,773
Posts
120,717
Replies
39,398
Members

Latest FME

Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.