Using transformers to modify data.
Recently active
Hi, I am using the Change Detector transformer and had planned on using the Match all Except Selected Attributes parameter for the Attribute Matching Strategy since my list of excluded fields is shorter than the list of included fields. I'm wondering if this parameter is working as it should? Everything appears to have changed when I used this parameter (which is not correct). However, when I reverse it and use the Match Selected Attributes (and select all the appropriate fields), it works as expected. I can use the transformer the way that I have it working but I was wondering if I am missing something? Thanks, fiona P.S. I'm using FME 2013 Build 13264 WIN 32 with SDE 10.1 data (from SQL Server).
Hi, I have a sample R script. While the output is correct, FME is logging incorrect information. The Code: library(sf)shp_path <- Output$shapefile_pathif (is.null(shp_path) || shp_path == "") { stop("location empty or no exists.")}shp_path <- normalizePath(shp_path, winslash = "/")if (!file.exists(shp_path)) { stop("location not found: ", shp_path)}print(paste("read shapefile:", shp_path))shape_data <- st_read(shp_path)if (is.null(shape_data)) { stop("Error in shapefile:", shp_path)}shape_data$ano <- 2025output_path <- shp_pathprint(paste("Writing:", output_path))st_write(shape_data, output_path, delete_layer = TRUE)print("Success.") Log below: Error in utils::read.table(value, sep = sep, header = header, skip = skip, : no lines available in inputCalls: dbWriteTable -> dbWriteTable -> .local -> <Anonymous>In addition: Warning message:In file(file, "rt") : file("") only supports open = "w+" and open = "w+b": using the formerExecution haltedFailed to
Any idea how I could cut my 3D surfaces pipes where the 3D surface wall stands ?Thanks
Hi everyone,I’m trying to set up FME to geocode addresses using a custom locator hosted in ArcGIS Portal, but I’ve hit a roadblock.In the Geocoder transformer, one of the parameters is the Authentication Type, where I selected Web Service (see attached image). From there, I’m required to create a web connection. This is where I’m having trouble. I’ve been following this guide, which explains how to connect FME to an ArcGIS Portal locator. The guide seems to focus on locators tied to an "Application" type item, but our locator is a hosted locator, not an application. Here’s where I’m stuck: I’m using the **Geocoder** transformer in FME and trying to configure it to connect to our hosted locator. The guide asks for information like a client ID and client secret, which seem to apply to an Application type item in Portal. Since our locator isn’t an application, I’m not sure what to provide for those fields. Can someone confirm: Is it even possible to use a hosted locator from ArcGIS
I’m trying to retrieve a WebConnection using the FMENamedConnectionManager in a PythonCreator as described here.Content of the PythonCreator:import fmeobjectsimport fmewebservicesclass FeatureCreator(object): def __init__(self): fmewebservices.FMENamedConnectionManager.getNamedConnection('SAP S4') def input(self, feature: fmeobjects.FMEFeature): pass def close(self): passThe execution fails with the ErrorPython Exception <TypeError>: descriptor 'getNamedConnection' for 'fmewebservices.FMENamedConnectionManager' objects doesn't apply to a 'str' objecteven though the documentation clearly states that the first parameter must be a string object (connectionName (str))What am I doing wrong?
Hi, Can someone help me with the Chartgenerator transformer. As a result I plot a pdf that looks like this: The only thing that I would like to do is to plot the values of the points as text labels next to the points. Like what was done before in EXCEL. Can anyone here help me to get labels next to my points? Thanks in advance.
Hi all,Currently I have a process that gets an excel file from an open data site, saves it into a Network File share location, and then i read it back into my workbench to continue my ETL process.(Download I am using from open.canada.ca: https://data.ontario.ca/dataset/7a049187-cf29-4ffe-9028-235b95c61fa3/resource/6545c5ec-a5ce-411c-8ad5-d66363da8891/download/private_school_contact_information_eng.xlsx)I would like to stream line the process by avoiding the need to download the file and then reading it into my workbench. I’d rather save it to an attribute and go from there. When I use a BinaryDecoder with default settings, it’s able to read the attributes (see attached image) from the attached HTTPCaller, however, when I run it, It just fails with this huge hexadecimal error log. FYI, I am running FME Workbench 2022.2, and I shared a portion of my workbench in the attached zip. Anyone have a similar process that can provide some guidance? Am I limited to only downloading? Any and all h
Hi everyone,I'm trying to replicate the "Explode" functionality of ArcGIS Pro in FME. The Explode tool in ArcGIS Pro breaks multipart geometries into individual single-part features. For example, if I have a multipart polygon, it will split it into multiple single-part polygons, each becoming a separate feature in the output.I've looked into the GeometryPartExtractor, but I'm not entirely sure if this is the correct transformer or how to configure it properly for this task. Is there a specific transformer or method in FME that is directly equivalent to ArcGIS Pro's Explode?Any advice or examples would be greatly appreciated!Thanks in advance for your help.
Folks, I have upgraded from 2024.1.x to 2024.2.1.0Clearly the Transformer Suggestor where you start typing a name and it suggests transformers to you is not working properly (I dont know what the function is officially called)But in the previous version - i would start typing and it suggested everything with that name. Now it hardly suggests anything - and when it does its decidedly odd. For instance i type the word “String” and get nothing.. Is there anyway to change the behaviour of this feature? its been useful for years and i really liked it returned a longer list.
Hello everyone,I am trying to use the change detector to compare changes from the current input data compared with previous runs to filter out Features that have not changed in order to save some processing time. In order to do that I create a hash using the CRCCalculator and save that away. With each new run those hashes are compared agains the previously saved hashes. The ChangeDetector gives me pause though.For some tables the ChangeDetector identifies changes where there are none. Whats more is, when I rerun the ChangeDetector, second time around the results are as expected without changing anything in the workspace. Maybe anyone has some insight into what could be going on here?1st time running identifies updated features that arent updated at all2nd time running without changing anything: different (this time correct) results
HiI need to extract the distinct attribute values of each field/column. An example is provided below: Input Owner Building_Type Building_ AgeRow 1: Steve Office 1970-79Row 2: Adam Office 1980-89Row 3: Steve House 2001-5 Output: Owner Building_Type Building_ AgeRow 1: Steve Office 1970-79Row 2: Adam House 1980-89Row 3: NULL NULL 2001-5 Any help will be appreciated.
Hello everyone,My colleague is trying to read a zip file in FME Form that contains 3 .xlsx files. My colleague read in this these three files and exploded the data, so it looks like image 1 and we were wondering how we would be able to transpose it to image 2. A criteria is that A1, B1,C1 should be in the same row as shown as in image2. I was thinking of using the xlsx_row_id variable and maybe the name of excel file as attributes for sorting that out later. However, to go from image 1 to image 2 beforehand is unfortunately still a bit of a mystery to us. I looked up into the attributetransposer, but that didn't yield the result I wanted. image 1image 2
Bonjour,Je bataille depuis quelques temps avec l’appel de l’API SIRENE v3.11https://api.insee.fr/catalogue/site/themes/wso2/subthemes/insee/pages/item-info.jag?name=Sirene&version=V3.11&provider=insee En utilisant un HTTPCaller j’arrive bien à faire fonctionner l’API mais je ne comprend pas comment appliquer les variables de la recherche multicritère de l’API POST /siret.Avez vous déjà fait appel à cette API et des conseils à me donner ?Merci d’avance :) Voici l’appel à l’API que je souhaite : Et voici les paramétrages que j’ai essayer dans mon transformer :Fonctionnel mais sans filtresNon fonctionnel
Hello everyoneI am trying to do the following: My workspace writes into a database. I have set a user parameter to give the user an option “delete Database and start a fresh database OR continue adding data onto existing data”. In case of deleting the Database the creation of IDs (which are created in the Process) should start at 1. In case of adding data to the existing database the creation of IDs should continue from whatever the max value already existing in the database is.I believe this should work as follows using a YES/NO User Parameter.Test Filter on that User Parameter:YES → Sampler (create single initiator) → SQL Executor (delete all data with “delete from tables”)NO → Sampler (create single initiator) → Feature Reader on Database → Statistics Calculator (extract max ID) Now in a different path within the workspace I have a Counter which I use to create the IDs. I thought I could simply use a conditional Value set on my YES/NO User Parameter withYES → Start at 1NO → Start at
I am reviewing some of the Transformers I either rarely use or have never used and came across the Feature Holder. Can someone provide me an example or two of why/when it would be a good idea to use the Feature Holder Transformer?
Hi All,I would require your help dealing with empty values in a columnI have a file with "n" features. In one of the column (Example: Column A) certain values are null/empty. I would like to replace it with non-empty value. Also, Column A has only 2 distinct values - either null or date.Example:Feature Column A1 Jan 20252 Jan 20253 Jan 20254 null5 null6 null7 Jan 20258 Jan 2025I would like to change Feature 4,5,6 to "Jan 2025" instead of null.In Java world, I would store them in local variable and run column A in loop and replace with value if null.Not sure how to perform similar scenario in FME. Would really appreciate your help in achieving the above expected output.Thanks in advance
Hi all.I'm having a hard time figuring out why lists with one entry, especially if it's read from XML, is mishandled in FME.A list may have any number of entries, and if reading such from XML, they often have any number including zero entries.When flattening the output, cycling thru the list, one normally uses ListExploder. However, if the list in question has one entry, it's auto-exploded, which makes ListExploder reject it. So I have to check the rejected features to see, if the list had been auto-exploded or if it had no entries at all. E.g. two or more transformers instead of just one.Why?Wouldn't it be more logical to _always_ create a list, if there are 1 or more entries ?It works as such in 2021.0Is this behaviour specific to XML (or JSON?) flattening ?Cheers.
I have Excel sheet with 2 fields, and I need to create folders with the values of the field
Which version of FME Desktop/Flow supports “Microsoft SQL Server Spatial Reader/Writer”? I want to re-project MapInfo Tables in the GIS database. Currently I’m using FME Desktop 2022.1. Is there any alternative transformer I can use to re project the objects in the live database?
Hello,I’ve had a workbench running on FME Server and FME Desktop 2022.2 perfectly fine through the last 2 years. Suddenly, it’s throwing an error HTTP Transfer error: Transferred a partial file. I tested it with FME Server and FME Desktop 2022.2 and get the same error. When I access the url through a browser, it perfectly downloads the zip file. But the HTTPCaller fails. Request Url: https://static.aer.ca/prd/data/wells/PLA_RTF_SHP.zipAny insights would be greatly appreciated.Thank you. Sample below for testing purposes, attempting to download the file in my local.HTTPCaller settings: Error:
Hi,I have encountered a problem. I have built a script that reads in from several SHP files followed by multiple SpatialFilters against another SHP file (A). The aim of my script is to check which polygons from (A) intersect with any of the other SHP files. I never know how many polygons from (A) will intersect because it is regularly updated.So, here is my problem. The result of this is a CSV and an SHP that contains the polygons that have intersected, and all the attributes from the SHP it checks against are appended. I want to remove all the attribute columns that contain only missing as values, but I have not been able to do that. The result can contain up to 25-30 different attributes! I have tried AttributeManager, NullAttributeMapper, and PythonCaller, but I can't make anything work.Any suggestions would be very helpful!
In the exercise: "Add Appearances to a 3D City Model" I have a problem with the "AppearanceSetter" transformer because I cannot get the colors in the buildings in the PDF output, as described by Liz Sanderson. I always get them in RED in both cases, namely: "AppearancesRed" and "AppearancesBlue". I have run the workspace as I downloaded it but both types of buildings always come out in RED in the PDF.https://support.safe.com/hc/en-us/articles/25407543263373-Add-Appearances-to-a-3D-City-Model
Hi,When using HTTPCaller with an authorization header with a bearer token, the token appears in the log.How can I prevent this ?I’m using FME desktop and server version 2021 so LOG_FILTER_MASK doesn’t work. Updating the version is not a solution. The bearer token is obtained at runtime so I can’t put it in a private parameter (PP cannot be updated at runtime).Thanks.
Hi community, I am using FME Workbench 2024.1 to create parquet files. Those files contain DateTime columns. It seems as those DateTime columns are missing a codec flag in order to enable the loading in other software (Dremio in my user case). Using parquet-tools we can see the problem:java -jar parquet-tools-1.9.0.jar schema ~/Downloads/part0.parquet can not read class org.apache.parquet.format.FileMetaData: Required field 'codec' was not present! Struct: ColumnMetaData(type:INT64, encodings:[RLE_DICTIONARY, PLAIN, RLE], path_in_schema:[_transaction_ts], codec:null, num_values:66, total_uncompressed_size:95, total_compressed_size:113, data_page_offset:35, dictionary_page_offset:4, statistics:Statistics(max:EC 90 D1 36 42 42 10 18, min:EC 90 D1 36 42 42 10 18, null_count:0), encoding_stats:[PageEncodingStats(page_type:DICTIONARY_PAGE, encoding:PLAIN, count:1), PageEncodingStats(page_type:DATA_PAGE, encoding:RLE_DICTIONARY, count:1)]) I include here a small parquet example with two date
I have a workflow that assesses the proximity of points along lines. These points extend along a series of line segments of varying lengths. For instance, the distances to the next points might be 0ft (the start), 100ft, 100ft, 200ft, and 250ft, so I would like for the corresponding field next to these values to display 0ft, 100ft, 200ft, 400ft, and 650ft, etc. I am aware of the workflow to find cumulative sums (Attribute Creator>Statistics Calculator>Attribute Renamer). While this processes as described, I regret that the cumulative sums do not “reset” when there’s more than one batch of point data within the feature table (the initial point of an additional batch of points simply takes the prior set’s end sum and begins adding new distances from there). If there’s a way to prompt the cumulative sums to account for new “initial” points in a feature dataset and so start from scratch each time, I would greatly appreciate any insight on how to prompt that to occur. I’m happy to sha