Using transformers to modify data.
Recently active
Hi Community,I am encountering an issue when attempting to connect to Azure IoT Hub using the AzureIOTConnect transformer in FME. Despite successfully generating an X.509 certificate using OpenSSL and being able to connect to Azure IoT Hub through a Python script, I am unable to establish a connection using FME. Here are the steps I followed:Generated the device key and certificate using the OpenSSL;The generated certificate includes the following relevant details:X509v3 Basic Constraints: CA:FALSE X509v3 Extended Key Usage: TLS Web Client AuthenticationThe connection works perfectly when tested through a Python script, but fails when using the AzureIOTConnect transformer in FME. The error message returned by FME is as follows:Failed to connect to the broker. The reason given is '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1000)' Given the details above, I am seeking guidance on the following:Are there any specific configur
My first “proper” Custom Transformer for maybe public consumption is AttributeTruncationChecker (attached)I’m not sure I have totally debugged it, but it seems to work now in most samples I throw at it. Will leave it here for the Community to possibly use it and improve it.This is trying to address possibly one of the more common headaches we run into, in getting the dreaded Truncated Values warnings in the Translation Logs when a Feature Attribute string is too long for the Writer destination field. At the moment, it is very time-consuming to track these down in large Feature datasets/Multiple Feature Datasets being written simultaneously, to design and run some Transformations to try to find the offending Features and which Attributes.This Transformer aims to detect these in the Workspace, by comparing the Workspace Features to Reference Schema Feature(s), typically from a FeatureReader. Originally I found I could “solve” this relatively easily with AttributeExploder based standar
Hi Everyone, I have recently started with FME. I have a case where I have to format an XML from a list. I am using XMLTemplater Transformer to build a request xml which looks like below<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:wnb="http://sample.com.au"> <soapenv:Header/> <soapenv:Body> <wnb:fetchNotificationDetails> <!--1 or more repetitions:--> <notificationNo>121</notificationNo> <notificationNo>131</notificationNo> <notificationNo>141</notificationNo> </wnb:fetchNotificationDetails> </soapenv:Body></soapenv:Envelope> I am passing some ten notification numbers to a XML templater and they are coming out individually like below <soapenv:Header/> <soapenv:Body> <wnb:fetchNotificationDetails> <!--1 or more repetitions:--> <notificationNo>301</notificationNo>
I have streets with LEFT and RIGHT address ranges assigned. In some cases, I need to break the segment into 2 or more segments (for example, when a railway track is crossing the street). How do I re-calculate the NEW address ranges for the NEWly created segments? For example, as shown in the image, parent segment has address ranges of 0-100 and 1-99. After braking it up into 2 segments, I am expecting to calculate and assign address ranges of 0-28, 1-29 and 30-100, 31-99 to the child segments.Which transformers can I use to calculate the address ranges for the new segments?
Hi everyone,I'm facing an issue with the AttributeValueMapper transformer in FME. My goal is to remap all values in the "abbreviazione" field using values I want to input manually. However, I've noticed that I'm unable to set the distinct values of the "abbreviazione" field as input values because the transformer doesn't seem to offer the option to read cached values.The only option I'm given is to import the values from an external dataset, but I'd like to avoid adding too many steps, especially since in other cases, I was able to select the distinct values directly from the layer I'm working on (or at least I think so...).Has anyone encountered a similar issue or have any suggestions on how to proceed? It would be really helpful to understand how I can retrieve and manage these distinct values within the workflow to use them in the remapping.Thanks in advance for your help!
Hi all,is it possible to use the NeighborFinder so that it tries to find the closest feature with a matching attribute inside the maximum distance area first?My goal is to find for my base features the closest feature from the candidate features which has the same attribute, but inside the maximum distance of lets say 20 meters. And only if there is no candidate feature with the same attribute inside those 20m, find the closest feature no matter the attribute inside 20m.What would be the best way to accomplish this in FME?
Hi there. I have encountered a problem with ProjReprojector and FME Form 2024.1.1. Source data is PostGIS and source coordinate system is EPSG:4258. With the <Read from feature> option, only the first object that arrives the transformer is reprojected. Then it fails. I have carefully checked that all features that arrives the transformer is properly tagged with coordinate system EPSG:4258. When I set the source coordinate system explicitely to EPSG:4258, everything works perfectly. Any others that have seen this behaviour? With earlier versions og FME (FME 2020), everything works perfectly. Is this another issue with FME 2024?
I have a table of 1600 polygons (featureset A) with most of those polygons overlapping ; they represent distance discs from a point of origin. I want to relate those polygons to another feature set (B) and calculate the %area of the intersection for each input polygon.I can’t use the AreaonAreaOverlay as is since it explicitly states in the documentation that it doesn’t expect the input features to self intersect, and this is not true for A.When I filter for a single polygon, The AreaonAreaOverlay generally gets me there, but, the problem is I want the output of all polygons. Spatial relate doesn’t give me the intersection of the features, so I can’t calculate the % area overlap. My preference would be to use geopandas.overlay and iterate over each polygon from A, but, my input data is stored in arc.SDE so converting them to gdf objects is more hassle than it seems worth to then convert back out into an FME feature object to then write to an SDE again. I tried using arcpy.analysis.Inte
The functionality works in workbench but not when creating a server app on FME Flow. The goal is to have the user select the file path to the file geodatabase and enter/select the feature class name. The schema will remain the same but the file geodatabase and the feature class name are unknown. In workbench, I can click on the three ellipses and the feature types to read pop up and I can select one. Then the process runs as expected. New feature classes will be added over time so being able to select the one you want is important. Workbench - clicking the ellipses to then select the feature type/feature class to read In the server app I am not able to click on the ellipses to select the feature class I want but instead need to type the feature class name in. When doing that, the process completes but doesn’t actually run the process because it “can’t find a feature class named that” in the file geodatabase. Even though there is a feature class with that name and there is no typo. Ser
Hello everyone,I am reading a google sheet with Google Sheets v4. The reader runs but it does not read some of the columns. I do not get any errors. how is that possible? I thought maybe because those columns cells are empty they can be read and then I added values in some cells but that was not the case. it still does not read them. in the workbench when I view the source data they are there in the sheet.
Hi,Is it possible to add a map of a feature/layer in an HTML report using the HTMLReportGenerator? Thank you in advance
Hello there, How can I clip a Raster image using FME ? Thanks, Siva
As a beginner, Can you give or help me very very detailed step by step process guiding me to build FME workbench for this geometry and attributes gap findings with line shapefile feature in FME ? Please help with any logical approach or steps or solution to find attribute name continuity gaps finding in a line features as with existing roads names or street names on it. For more details you can mail me on my mail id : actual.abhijeet@gmail.com
The system integrate data from ArcGis API Services(source) to Microsoft SQL Spatial DB(target). When the api side have new row of data, i want to check if the data that is already pulled to DB is already existing using let say object_id field. I cannot use upsert at the writer since the flow is that we do not the want any change of pulled data since it will need to go through manual approval for any data changes. How to insert new row of data only without updating existing data?
Hello,I have a Navisworks 3D file of our facility that needs to be viewed in ArcGIS Pro and published as a 3D Scene. We exported it out to a fbx for City Engine which is then exported to a scene layer package to be viewed in ArcPro and this “works”, but we have a only layer with zero attribution. Is there a way I can either use the original Navisworks file or the fbx to break out the all the features withing the facility and create an output as a multipatch gdb? sorry for my noobnessthanks in advance.-C
If yes please may i request you to share me demo workbench ? If NO, is there a workaround to convert Navisworks files to IFC format ? Any suggestion / idea / workbench is more than highly appreciated forever . I also request FME experts who even have little know-how please requested is your little for any leads . thank you / Br
I am having 100 individual text files consist of Point feature..So..........At a time, How could i convert each files into KML files respectively?
To reproject WGS84 (Lat/Long) to GDA2020 (Easting/Northing) in FME v2024.1, the only option appears to be via GDA94. This moves the data approx. 1.8m NE as it assumes that WGS84=GDA94 and GDA94+1.8m = GDA2020.While this is usually what people use, there is a need for a second option of WGS84=GDA2020 (ie a Null transformation + Geographic to Grid) as other software packages, such as Conquest have adopted this. For more information see: https://www.spatial.nsw.gov.au/__data/assets/pdf_file/0008/224396/WGS84_and_Australias_misaligned_web-maps_Information_Sheet.pdf. Is this something that FME will be implementing anytime soon?
I'm still a newbe to FME considering below.I'm trying to query open data from the Dutch https://public.ep-online.nl/api/v4/Mutatiebestand/DownloadInfo?fileType=csv (API-key required, available from https://epbdwebservices.rvo.nl/) to get the data from a csv-file. HttpCaller works, url obtained with JSONFragmenter (attributes to expose: 'downloadUrl’)+testfilter (to get the actual url). However, I can't seem to use this url in an FeatureReader or further HTTPCaller to actually get the data itself from the csv-file. Copying the url (contains a token valid only one time) does download the file from a browser as a zip-file, but how can this be done in FME?I tried to look at https://public.ep-online.nl/swagger/index.html but couldn't make anything out of it due to lack of technical knowhow. Any ideas? Setup as of now: FeatureReader tried below (the response_body solely contains the url):Also tried '@Value(_response_body)[*.csv]’ for parameter dataset, but also to no avail. Another try with
I have read through this post to try to get a fix but it doesn’t address the issue i am having.The work flow is…use a Directory and pathname reader to get a list of zip file in a specific folder. sort by path created date sample the first 1 feed the path widows to the reader (esri geodatabase reader or featurereader [preferred]) I am trying to capture the feature datasets within the gdbI have gone to the parameters of the fgdb featurereader and exposed the geodb_feature_dataset format attribute as well as the fme feature type.I have also gone into the generic port and exposed the same. it does not show up as an attribute. i also have used an attributeexposer after and it is not possible to expose it because it is not there.If i change the reader to read *.gd this featuredataset still is not exposed in the featurereader.The only method i have had success with is to set the Output Port to One per Feature Type. This is not optimal as any transformer i want to use will have to be done on e
Hi there, Anyone could help? I would like to build a FME workbench to run a multiplication for all the value if they are belongs to the same name. Attached a sample data e.g. I want to get the multiplication for armstrong_2006( 42*10*39*26), then I want the result to be added back in this spreadsheet. thanks.
Hi all,I would like to run FME Desktop automatically. After seeing some posts, the only method that I have found is using FME Server, however, I have not understood how it exactly works. Could anyone please help? Is there any other way?Many Thanks
Hi, I have a workspace that i want to compare timestamps between two rows in a database based on some matching information.i’m using an sql extractor to pull the pertinent information, which spits out something like this: IMAP TIME SUBJECT DATETue Aug 20 02:42:25 EDT 2024 XXXXXXX_#6337.4 8/20/2024Tue Aug 20 02:42:52 EDT 2024 XXXXXXX_#6337.4 8/20/2024Tue Aug 20 04:06:13 EDT 2024 XXXXXXX_#704.0 8/20/2024Tue Aug 20 04:06:41 EDT 2024 XXXXXXX_#704.0 8/20/2024Tue Aug 20 05:33:15 EDT 2024 XXXXXXX_#7274.2 8/20/2024Tue Aug 20 05:33:47 EDT 2024 XXXXXXX_#7274.2 8/20/2024I’m then using a StringSearcher to extract the time with the regex \d+:\d+:\d+ and this gives me the time in essentially a %H:%M:%S format:Extracted original IMAP (for comparison)02:42:25 Tue Aug 20 02:42:25 EDT 202402:42:52 Tue Aug 20 02:42:52 EDT 202404:06:13 Tue Aug 20 04:06:13 EDT 202404:06:41 Tue Aug 20 04:06:41 EDT 2024
Hello, I'm trying to use AffineWarper to transform 3D data (x,y,z) from a local coordinate system to a national coordinate system. This works well, but unfortunately the z coordinates are not changed. They retain the heights of the local system. Does the transformer only work in 2D?The offsetter doesn't help me here because the difference between the z-values is not the same for all coordinate pairs.Please help. Thank you.
Doing a query on a List named “LayerAttributes” worked in FME 2022 but doesn’t work in FME 2024. Any suggestions on how to work around this issue?In FME(R) 2022.2.3.0 (20230131 - Build 22789 - WIN64) I was able to query the list and get results. Here is a cutdown example where the query Value(LayerAttributes{@sum(@Value(LayerNameID),1)}) gives me the same results as LayerAttributes{7} However, in FME(R) 2024.1.1.0 (20240729 - Build 24619 - WIN64) the same query doesn’t work. @Value(LayerAttributes{@sum(@Value(LayerNameID),1)}) gives me <missing> and LayerAttributes{@sum(@Value(LayerNameID),1)} give me LayerAttributes{7}