Using transformers to modify data.
Recently active
Hi all, I have a field with the numberic part of an address. I would like to replace the last 2 digits with XX and combine it with the beginning part. I have gotten as far as using the SubstringExtractor to extract the last 2 characters, but how can I combine it with the first part? Examples I have: 123 12345 Need it to be: 1XX 123XX What does the transformer do with the part of the string that doesn't get extracted??? Thanks!
I have a workflow that:Takes a feature class from an ESRI Geodatabase (ArcSDE) Removes some attributes via AttributeRemover Writes to several different formatsI want to change Step 2 to use an external list (for example, a TXT file or CSV - the format is flexible), because other users will be regularly updating the list of attributes to remove.It seems like AttributeRemover only takes a single string or manual selection of attributes, so I decided to try BulkAttributeRemover, since it can take an attribute value. However, I ran into this problem:Let’s say I want to remove all attributes that contain the string PRIVATE.When I type the string PRIVATE into BulkAttributeRemover: Expression to Remove, the transformer functions as expected. It removes all attributes that contain the string PRIVATE.However, if I create an attribute called “regex” and set its value to PRIVATE, and then use the attribute “regex” as the Expression to Remove in BulkAttributeRemover, the transformer removes all va
Hi,I'm trying to create a workspace that reads any feature class of a certain geometry type (e.g. line or point) from an Oracle Spatial database and writes it into an SDE Geodatabase. Since around 200 feature classes need to be processed, a manual transformation is not possible (meaning that a reader and writer are created for each feature type individually). I have tried different approaches that did not work: - Generic reader: Expects as an input a "source dataset". What I have is an database, however (I need to provide its connection parameters via published parameters instead of providing a dataset) - Dynamic reader (merged feature types): An Oracle Spatial reader together with "merge feature types" and a filter on "Features to Read" -> "Feature Types to Read" (provided by a published parameter) did not work, because its schema does not change with the feature classes dynamically. I finally used a FeatureReader in combination with
I want to find each line of the string below with a regular expression in StringSearcher that looks like this .*\\r\\n, i.e., I expect each line to end up in the all matches list produced by the StringSearcher.METER_NUM,PRODUCTION_DATE,FLOW_TIME_MINUTES,VOLUME,ENERGY,DIFF_PRESS,STATIC_PRESS,FLOW_TEMP,MOL_CO2,ENERGY_FACTOR1006-2001307,20170829,132.283,13.8851,,9.45286000000001,1088.28,83.9825000000001,,1006-2001283,20170829,1440,311.974,,27.9496,162.085,88.4964,,It works in my regex testing tool Expresso, but StringSearcher doesn't seem to recognize \\r\\n.
I don't know if this is already answered somewhere already but couldn't find it.This is in regards to my previous thread about aligning randomly distributed line geometry as per building footprints. Now, I could find GPS traces as point data to align my line geometry as per the points.In the image below: The black line is the approximate required solutionI tried using NeighbourFinder with 3m distance and then MeasureSetter for putting weights. Ideally, I wanted that points towards the center are weighted more than the outer ones. I have attached some sample data for the same. Could someone help me out with any ideas?Thanks.
one of my attributes under my writer output has a red arrow next to it and im not sure why its red. this is an important attribute that i need for my shapefile. does anybody know why its not reading it?
Dear FME community, I have CityGML tiles which I converted into KML or KMZ files. They are in a certain folder structure. Now if I open single KML files they will only display in 2D with the inspector transformer. Is there a way to load all the files that belong to the KML structure? So that it displays in 3D? When I upload it in Google Earth it works just fine. At the end I would like to convert these KML or KMZ files into 3D CAD files like dwg or obj. Thank you very much for your help. With kind regardsChristian Lendl
Hello, I have a simple workspace that is reading in a feature class from a source geodatabase and writing it out to my target geodatabase. I am using (and need to use) a drop and create table handling method to write the feature class. The source feature class I am reading from is contained in a feature dataset. I don't want this feature dataset to be written out to the target geodatabase; I am only interested in writing the feature class it contains. However, the writer recreates the feature dataset from the source gdb, even though I left the Feature Dataset Table Creation Parameter in the writer null. How do I configure my writer so that only the feature class from the source geodatabase is being dropped and created, while ignoring and not recreating the feature dataset? Thank you.
Hi FME community,I am still fairly new to FME. I'm getting stuck with date/time fields in csvs and excel documents. I've attached the field I'm trying to convert into DD/MM/YYYY HH:MM (%d/%m/%Y %h:%m ?), but it writes out empty if I try the DateConverter or AttributeManager> Text editor > DateTimeFormat. It's likely I'm using the transformers incorrectly. Is anyone able to point me in the right direction?
Hello everyone! There were two adjacent DTMs with different resolutions(one is 0,5m and the other is 5m.). I resampled them to bring together and mosaick at 1m resolution. There seems to be no problem with 0,5m res. area in the mosaicked image, but after the resolution is reduced for 5m area, defects like a stair steps appear in 3D view(as seen in the image). So, how can i get a smoother suface for the downsampled area from 5m to 1m. Which transformers or parameters should i use in my workflow. Thanks a lot. @daleatsafe @daveatsafe @takashi @jovitaatsafe @redgeographics @deanatsafe @rylanatsafe @hollyatsafe @lenaatsafe @annabelleatsafe @steveatsafe
Hello, i've got an excel file with two attributes (serial id and a text field) which should be joined to a fgdb feature class by sid. Up to here no problem. But in the text field there are parcel numbers and also owners. My question: Is there a possibility to filter the alphabetic characters in order to keep only the parcel owners in this field? best regards
I have a feature class within a FGDB that contains 2631 fields that are named F1 to F2631 and looking to query certain fields from this main feature class to write out to a new feature class. For example the first i'm trying to write out as a new feature class I want to write out fields F1 to F40. Attribute Manager appears to struggle to remove fields F41 to F2631 when i try to use this transformer and crashes FME on me. I've been trying to figure out a way to query the fields i plan to remove within Bulk Attribute Remover but cant figure out how to complete the query? Any suggestions ?
The error occurs once in a while in a Workspace that is reading a PostGIS/PostgreSQL 12.9 database and writing a GEODATABASE_FILE using 64-bit ArcObjects from underlying an ArcGIS Desktop 10.8 installation. The error is triggered in a DatabaseJoiner, when attributes are joined in from a PostgreSQL table and written to the File Geodatabase simultaneously. The error happens once in a while (every second run or so), but it does strangely not occur on a PC with 64-bit ArcObjects from underlying an ArcGIS Desktop 10.4 installation or in any other variants of the workspace writing to other formats. The error occurs both on Desktop and Server. FME version is 2020.2.3. The error message does not say anything about the GEODATABASE_FILE writer, only the PostGIS reader, but the writer is obviously a part of the problem. If I disable the writer, nothing happens. If I bypass the DatabaseJoiner nothing happens. The message POSTGIS reader: stk::ex::bad_expected_access is discussed elsewhere on this f
I have an attribute string such as 6-6.35@4T. Where the 6 before the "-" is size (which can be a two digits such as 12). The characters between the "-" and the "@" are a depth measurement (and could also be something like 20.50). The number after the "@" is a clock position and can be in the range between 1-12. The alpha character at the end is indicative of a type. We captured the characters before the "-" as a size attribute. We did this using the string function - @Left(@Value(In_A),(@FindString(@Value(In_A),"-"))).What is stumping us is how to capture all characters between the "-" and "@" as a depth attribute and the numbers after the "@" as a clock position?We would like to accomplish this in the attribute manager if possible.Any help is much appreciated.
Hi I need to post requests to SOAP to retrieve some information about Planning cases back in XML and I have read the article: https://community.safe.com/s/article/working-with-soap-services Our process is more complicated as I need to log in and out of SOAP at the start/end of the process. I created steps in a TestCase in SOAPUI and it worked and I have set up a workbench with the following transformers: CreatorXML Templater (for login)HPPTCaller (for login - cookies enabled)XML Templater (to Get data)HTTPCaller (to Get data)XML Templater (to logoff)HTTPCaller (to logoff) While this worked for retrieving one planning case I actually need to retrieve information on multiple cases but cannot get this to work. I tried two things: 1) I created a spreadsheet with multiple planning case references in, read this into FME (replacing the Creator) and changed the value in XMLTemplater to use the attribute value (using fme:get-attribute), instead of a hardcoded value.This appeared to be attempti
I have a curb line represented as one object. However, I would like to split it at its "corners" (marked with the black circles). Apart from the curb line, I also have the street centerline.. Any idea, how I could get a satisfying result? Thanks!
Hi there,I'm using a PointOnAreaOverlayer to try and count the number of features ("UPRN") in a polygon ("DMA"). However, its not working. What group by or parameters might I need to set?
Hi all,So, I've got this small data set of 12 locations in the Netherlands. I would like to create a distance matrix that tells me what the distance (as the crow flies) is between each point. I figured that I could use FME for this. I think I managed to get quite far, but now I'm stuck and in need of some assistance :-(.So what did I do so far? I first created a workspace that reads the data set and then uses the FromToBuilder transformer in combination with an AttributeManager transformer to determine distance from one manually selected point to all other points. A csv-writer writes the resulting table to a csv-file. Secondly, I added a new published parameter in combination with the Workspace Runner transformer to run through the 12 points. So that I don't have to run the workspace 12 times and manually assign a point to the to-port of the from-to transformer. This generates a csv-file of 12x12=144 features. I'm no stuck figuring out how to convert the 1x144 tab
Hi All,I'm trying to create and write dynamic attribute name with the joined workspace. In a FFS file, I have C-SEGMT entities. I would like to count how any entities I have until the current year (e.g. 2023) but also for the two previous years (e.g. 2022 and 2021) and put the result in a dynamic attribute name (for my example, three attributes named "NumEntities__2023", "NumEntities__2022" and "NumEntities__2021" I can create them, as you can see it in my Workbench and the screen capture, but I don't know how to write them in my Writer. It's OK for a FFS Writer, but if I want to write in a GEOJson file or other formats, it fails. I could name them directly "NumEntities__2023", "NumEntities__2022", "NumEntities__2021" ; or I could expose them with AttributeExposer : I can see their name if I have enabled feature caching. But my goal is, if I run the workbench next year, the attributes will be automatically renamed
Is there a possibility to schedule the workbench on our local system rather than putting it on the server due to data confidentiality issues?
I want to test my GML file against a schema to make sure that it is valid. To do this, I am using an XMLValidator transformer. I am not sure how it works.When I add a reader for my GML file, it loads all the different feature types. I also selected my .xsd file containing my schema. Connecting these to the XMLValidator goes excruciatingly slowly and everything seems to fail anyway.If I select my GML file as the "XML Filename" parameter for the XMLValidator, and then my .xsd file as the Schema Location (Validation type "Syntax and Schema", and then unconnect the Feature Type readers, it seems to work, but I have inspectors connected to the "Passed" and "Failed" ports on the XMLValidator and nothing seems to go through either of them, even though the workbench runs successfully. Am I using it right?
I have a CSV file with Longtitude and Latitude fields (more than 10.000 records) and I want to convert these to point (shapefile). After that I want continue use a polygon to select which points intersect with it? I tried to use VertexCreator and SpatialFilter or PointOnAreaOverlayer but it didn't work like my expect. So who can help me to solve this problem. Thank you so much!
I am sure this is a simple fix for this, but I've been testing this for hours now and going nowhere!In AttributeValidator, I want to FAIL an attribute which contains a certain string. The most logical thing I could think of is to use the Contains Regex validation rule. Let's say I want to 'Fail' every attribute where it contains a string which = FME. I have trawled the internet and got the following expression to work within the Regex Editor:\\b(?!FME)\\b\\S+However, when it runs in the AttributeValidator, attributes which i have engineered to contain FME, appears from the Passed output port.1. Is there a much simpler way within the AttributeValidator to fail any attribute which contains a certain string, rather than regex?2. If not, am I doing something fundamentally wrong with the Regex or validation rule?Thanks
I am trying to re-write an IFC file including its Type Objects. The IFC reader can either output them as single IfcTypeObject feature type or as separate feature types (IfcWallType, IfcDoorType, etc). Reading the Type Objects works fine for me. After manipulating the data, I would like to write them back into their relevant feature types, e.g. IfcDoor together with IfcDoorType, related through the ifc_type_object_id and ifc_unique_id attributes (I'm unsure how the unique IDs should be created but I simply used the UUIDGenerator where required) I also tried reading and re-writing the IfcTypeObject features. However, the Type Objects seem to be missing from the outputs, giving me warnings like the below. If anyone could help out here? IFC: Unable to write objects of type 'IfcWallType'IFC: Unable to write objects of type 'IfcDoorType'IFC: Unable to write objects of type 'IfcBeamType'IFC: Unable to write objects of type 'IfcSpaceType'IFC: Unable to write
Hi everyone, I have two data setsOne contains many catchment areas and their sizes (calculated)The second has the total area of all catchments (Calculated)I need to join these two so that I can then use an ExpressionEvaluator on them to determine percentages. How do I do this if they do not have a common attribute to join on? I tried a FeatureMerger assigning them both a value of 1 as I thought I read somewhere that that would work... but can not find that article again :-(Thanks!Iris