Using transformers to modify data.
Recently active
Hi All,Anyone could help me with this specific question in FME? I want to remove the repeated words in each row, e.g. A,B,C,A,B in a row, the A, B have been repeated twice, I would like to remove the repeated A and B only. please see attached the sample data. many thanks.
I have created a workbench to detect schema different.I am using two ChangeDector Transformer in this workbench which is producing output as I intend. Figure below shows that part.Workbench for that part This is output data (in csv file) based on that workbench.Output by FME Workbench 2023.1 (FME Form 2023.1)However, when I used the same workbench with the same data, the following output were produced. It got extra error. For error code E302, those not bold should not be there.Output by FME Flow 2023.1.1.1The workbench used is the same.The data is the same.Only output 1 run in FME Workbench 2023.1 (FME Form 2023.1) and output 2 Run on FME Flow 2023.1.1.1 and FME Flow 2023.2.Note: The additional error contain field that is unchanged in first ChangeDetector and does not included in the second ChangeDetector (ChangeDetector3)
I'm trying to build a service catalogue of our municipality's services (WMS and WFS) for easy search and access. I have successfully used xml readers and httpcaller along with xml flatteners to extract the information into a queryable json table and presenting it with datatables (https://datatables.net/ ). Very handy and fast. But now I've run into some trouble that I can't really understand or work around. Getting the info from well over 600 services has been easy but the last one I added does not work. See attatched workspace. "TK OPEN" works just fine. "Miljö" does not. It seems to be related to XML and HTTPS, but that really isn't my strongest field. getcapabilities.fmwAm I doing something wrong? Is there, perhaps, a simpler or other way to do this?
Hello Friends!My current setup: FME(R) 2024.0.1.0 (20240328 - Build 24202 - WIN64) ...from ArcGIS Pro Data Interoperability Extension I am attempting to use the GeoJSON Reader to capture information from api.weather.gov/alerts/active (Weather.gov API). I am not having any problems getting data. The problem I am having is populating all of the available fields within the API - in case I ever do get them from the API. The first time I connected to the URL from the Reader, I got most of the attributes, missing maybe 10 fields. Adding another GeoJSON Reader and connecting to the URL an hour later, I get a different set of attributes. I do understand that the data is changing and their API is not delivering any unnecessary nested attributes.When opening the Reader’s Properties and then User Attributes - there is no way to add attributes. Am I missing something? Is there another way to add attributes?Thank you,--Adam
Hey. I'll try again and better. On the attached fmwt I am trying to assemble the correct geoJson structure - the data structure formatting is done on the "JsonTemplater5" transformer. The sample file for the data structure is the attachment "example_collection"( better to look here: https://github.com/UASGeoZones/ED-318/blob/main/examples/Example_Collection.json?short_path=e3f0eef), according to which I am creating a template (JsonTemplater5). The output from the workspac is "rmz.json" - the whole data structure is not created on jsonTemplater5, so it does not match the complete sample file.My question is. Does it really have to be this complicated and lengthy to define a template on jsonTemplater5 to send data to? Wouldn't it be possible to export the schema using a combination of some transformers from example_collection ( sample data )? Or if I have some schema available, can't I use this schema so that I don't have to create a template in such a complicated way? attachments: rmz_te
Hi All!I’m new to FME and have (hopefully) a simple question. I have created a workspace that reprojects data from one projection to another in ESRI Geodatabase (sde). I have embedded the connection parameters. The workspace successfully runs. I have created a batch file, using the command prompt information from the result of the translation log. The batch file successfully runs when executed manually. When I schedule via Task Scheduler, its fails (0x1). Return code 2147942401The user account that runs the task is my own username. On local PC. Runs with highest privileges set. Under Action, it points to the location of the batch file. What am I missing?Thanks,Leanne
Hello all,I have a two roads datasets with a road_id, a through_distance_start and a through_distand_endI want to do a merge, FeatureMerger, where;road_id equals road_idtdist_start equal to or greater than tdist_starttdist_end equal to or less than tdist_endHow do I accomplish this in FeatureMerger? Or is there a different transformer?
I want to convert the format of a date-attribute from %Y%m%d to %Y-%m-%d by using the DateTimeConverter. This works fine, as long as there are no Null-values. As soon as there is just one Null-value, all other values are set to Null as well (even though the option “Passthrough Nulls, Empties or Missing” is set to Yes).I tried to filter out the features with the Null-values, convert just the features with values and merge all features after the DateTimeConverter again, but here too, all attributes are set to Null.If I set the Null-values to a dummy-date, then it works. But I don't want dummy-dates, but Null-values, so how do I do this?Thanks in advance, Vera
In a process in my company we use a Python script (PythonCaller) to call a download API, we need to modify it, but we get some errors that I don't get when running the script from Python on the Windows command line.Is there a way to debug the execution of the script in FME?
Hi,I have a polygon with a donut hole, and I would like to split it in half, e.g. across the center of the donut hole. The result should be two polygons, as illustrated below. The polygon isn't necessarily always "horizontal," like in the example below; it could be rotated in any direction.Does anyone know how to do this? Any help would be appreciated! A polygon with a donut holeResulting polygons
According to several solved questions in this forum, there are two recommended ways to automatically remove attributes that only conatain NULL or empty values: using a small python script or using the NullAttributeMapper. Unfortunately both suggestions do not work for me (FME Desktop/Form 2022.2). Both the PythonCaller and the NullAttributeMapper keep the Null-Attributes and they are present when I write to Excel (see attached workspace). Is there something wrong in my workflow or is there an updated solution? Thanks for any suggestions!
Hi all,New on the forum but not on FME :) (although not an expert). I come acrross a doubt on how to work with zip files. I know FME reads zip files directly but only if you know what kind of file is inside it. So If I have some URL with zip files but those can have dwg, shp, png...I cant really figure it out how to say FME ok, you shape file go this path, dwg this other path... I know I could use httpcaller and store them, unzip and work with them after unzipping by filtering out the inputs but I want to save disk space and dont really want to unzip.Any idea will be welcome.Thanks!
Hi Community,I am encountering an issue when attempting to connect to Azure IoT Hub using the AzureIOTConnect transformer in FME. Despite successfully generating an X.509 certificate using OpenSSL and being able to connect to Azure IoT Hub through a Python script, I am unable to establish a connection using FME. Here are the steps I followed:Generated the device key and certificate using the OpenSSL;The generated certificate includes the following relevant details:X509v3 Basic Constraints: CA:FALSE X509v3 Extended Key Usage: TLS Web Client AuthenticationThe connection works perfectly when tested through a Python script, but fails when using the AzureIOTConnect transformer in FME. The error message returned by FME is as follows:Failed to connect to the broker. The reason given is '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1000)' Given the details above, I am seeking guidance on the following:Are there any specific configur
My first “proper” Custom Transformer for maybe public consumption is AttributeTruncationChecker (attached)I’m not sure I have totally debugged it, but it seems to work now in most samples I throw at it. Will leave it here for the Community to possibly use it and improve it.This is trying to address possibly one of the more common headaches we run into, in getting the dreaded Truncated Values warnings in the Translation Logs when a Feature Attribute string is too long for the Writer destination field. At the moment, it is very time-consuming to track these down in large Feature datasets/Multiple Feature Datasets being written simultaneously, to design and run some Transformations to try to find the offending Features and which Attributes.This Transformer aims to detect these in the Workspace, by comparing the Workspace Features to Reference Schema Feature(s), typically from a FeatureReader. Originally I found I could “solve” this relatively easily with AttributeExploder based standar
Hi Everyone, I have recently started with FME. I have a case where I have to format an XML from a list. I am using XMLTemplater Transformer to build a request xml which looks like below<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:wnb="http://sample.com.au"> <soapenv:Header/> <soapenv:Body> <wnb:fetchNotificationDetails> <!--1 or more repetitions:--> <notificationNo>121</notificationNo> <notificationNo>131</notificationNo> <notificationNo>141</notificationNo> </wnb:fetchNotificationDetails> </soapenv:Body></soapenv:Envelope> I am passing some ten notification numbers to a XML templater and they are coming out individually like below <soapenv:Header/> <soapenv:Body> <wnb:fetchNotificationDetails> <!--1 or more repetitions:--> <notificationNo>301</notificationNo>
I have streets with LEFT and RIGHT address ranges assigned. In some cases, I need to break the segment into 2 or more segments (for example, when a railway track is crossing the street). How do I re-calculate the NEW address ranges for the NEWly created segments? For example, as shown in the image, parent segment has address ranges of 0-100 and 1-99. After braking it up into 2 segments, I am expecting to calculate and assign address ranges of 0-28, 1-29 and 30-100, 31-99 to the child segments.Which transformers can I use to calculate the address ranges for the new segments?
Hi everyone,I'm facing an issue with the AttributeValueMapper transformer in FME. My goal is to remap all values in the "abbreviazione" field using values I want to input manually. However, I've noticed that I'm unable to set the distinct values of the "abbreviazione" field as input values because the transformer doesn't seem to offer the option to read cached values.The only option I'm given is to import the values from an external dataset, but I'd like to avoid adding too many steps, especially since in other cases, I was able to select the distinct values directly from the layer I'm working on (or at least I think so...).Has anyone encountered a similar issue or have any suggestions on how to proceed? It would be really helpful to understand how I can retrieve and manage these distinct values within the workflow to use them in the remapping.Thanks in advance for your help!
Hi all,is it possible to use the NeighborFinder so that it tries to find the closest feature with a matching attribute inside the maximum distance area first?My goal is to find for my base features the closest feature from the candidate features which has the same attribute, but inside the maximum distance of lets say 20 meters. And only if there is no candidate feature with the same attribute inside those 20m, find the closest feature no matter the attribute inside 20m.What would be the best way to accomplish this in FME?
Hi there. I have encountered a problem with ProjReprojector and FME Form 2024.1.1. Source data is PostGIS and source coordinate system is EPSG:4258. With the <Read from feature> option, only the first object that arrives the transformer is reprojected. Then it fails. I have carefully checked that all features that arrives the transformer is properly tagged with coordinate system EPSG:4258. When I set the source coordinate system explicitely to EPSG:4258, everything works perfectly. Any others that have seen this behaviour? With earlier versions og FME (FME 2020), everything works perfectly. Is this another issue with FME 2024?
I have a table of 1600 polygons (featureset A) with most of those polygons overlapping ; they represent distance discs from a point of origin. I want to relate those polygons to another feature set (B) and calculate the %area of the intersection for each input polygon.I can’t use the AreaonAreaOverlay as is since it explicitly states in the documentation that it doesn’t expect the input features to self intersect, and this is not true for A.When I filter for a single polygon, The AreaonAreaOverlay generally gets me there, but, the problem is I want the output of all polygons. Spatial relate doesn’t give me the intersection of the features, so I can’t calculate the % area overlap. My preference would be to use geopandas.overlay and iterate over each polygon from A, but, my input data is stored in arc.SDE so converting them to gdf objects is more hassle than it seems worth to then convert back out into an FME feature object to then write to an SDE again. I tried using arcpy.analysis.Inte
The functionality works in workbench but not when creating a server app on FME Flow. The goal is to have the user select the file path to the file geodatabase and enter/select the feature class name. The schema will remain the same but the file geodatabase and the feature class name are unknown. In workbench, I can click on the three ellipses and the feature types to read pop up and I can select one. Then the process runs as expected. New feature classes will be added over time so being able to select the one you want is important. Workbench - clicking the ellipses to then select the feature type/feature class to read In the server app I am not able to click on the ellipses to select the feature class I want but instead need to type the feature class name in. When doing that, the process completes but doesn’t actually run the process because it “can’t find a feature class named that” in the file geodatabase. Even though there is a feature class with that name and there is no typo. Ser
Hello everyone,I am reading a google sheet with Google Sheets v4. The reader runs but it does not read some of the columns. I do not get any errors. how is that possible? I thought maybe because those columns cells are empty they can be read and then I added values in some cells but that was not the case. it still does not read them. in the workbench when I view the source data they are there in the sheet.
Hi,Is it possible to add a map of a feature/layer in an HTML report using the HTMLReportGenerator? Thank you in advance
Hello there, How can I clip a Raster image using FME ? Thanks, Siva
As a beginner, Can you give or help me very very detailed step by step process guiding me to build FME workbench for this geometry and attributes gap findings with line shapefile feature in FME ? Please help with any logical approach or steps or solution to find attribute name continuity gaps finding in a line features as with existing roads names or street names on it. For more details you can mail me on my mail id : actual.abhijeet@gmail.com