Using transformers to modify data.
Recently active
I have a batch file (.bat) that I need to run. What Transformers would I need to use to get this to run using a workspace in FME Desktop?
Please can you tell me how i do this?
I need to do a transformation, like pivot table, using Python, and for that I would need to install libraries, like "pandas"... How can I install custom libraries to FME Python ? I created a environment with Anaconda, installed pandas there, but now, how to use that environment in my PythonCaller?
I am trying to mosaic 228 ECW tiles into one uniform image using RasterMosaicer transformer. The individual tiles are quite large as the 228 individual tiles together are 15GB. I am finding the process using RasterMosaicer extremely slow and is taking close to 24 hrs to complete using FME desktop. Just wondering what the best method is to mosaic these tiles and have a resulting image equal to the size of the individual input tiles, 15GB...or less.
I'm having trouble removing buffer segments correctly when generating a buffer around a line dataset with roads.Input data (road lines with road name and city): When buffering this, I end up with:This method has a lot of overlap that i'm not able to process properly. Is there a way to buffer it like this or process the _overlap >= 2 in a way that this is the result:I tried the areaonareaoverlay and correct the overlap afterwards but did not get the desired result.
When using a FeatureReader to read CSV file, one can manually select in the "Parameters" section which attributes we want to read from the file. Very short question : does something equivalent exist for reading only a selection of attributes in a reader that loads "Esri ArcGIS Portal Feature Service" data ? Thanks in advance
Dear All, I am facing the following very weird issue. I want to test for several attributes if, for each of them, there are some features that contain a specific value.To do that, I have created a Custom Transformer that takes as input 2 user parameters (defined exactly in the same way, published and required) : 1) colname = the name of the attribute ; 2) progname = the value to test if the feature is equals to. When I go back in the Main, right-click on the custom transformer > Properties, in the prompt I can see well the 2 user parameters (colname, progname). But when I click on "Run to this" custom transformer, the prompt only asks for 1 of the 2 parameters (always 'progname', even if I switch the order of the parameters in the parameter manager of the custom transformer). In the Translation log, I see that the following command line is run :fme.exe myscript.fmwThis results in translation failure with the error message :Undefined macro `progname' dereferenced in file [...]When I
I'm trying to write data to a csv. I have a uniqe identifier that has two leading zeros. When writing to a csv file the zeros are removed. When viewing the feature information window the attribute has a string value. When doing the same in the written csv it says 64 unsigned integer even though I have set it to string in the writer parameters. What am I doing wrong? Making sure FME writes it as a string will keep the leading zeros right?
I am trying to get from a table like this example:to this table:I have searched for answers but can't fully seem to get where I need to be- including trying variations of attribute transposer, attribute exploder, attribute exposer, and attribute pivot (this helped me get closer to the answer but didn't allow for text..). Any advice would be much appreciated!
Hello FME community, I am currently working on a deep learning project that involves preparing training samples for building polygons. I have the building polygons in an OGC GeoPackage file with Coord.System EPSG: 26916, as shown in following screenshot: Additionally, I have the PNG aerial images for the same area without projection, which means every aerial image X,Y starts from the upper left (0,-0) and ends at the lower right (2000,-2000) as shown in following screenshot: I need to have the polygons of the training sample in pixel coordinates for each corresponding image. The output should be a CSV file that includes the vertices of every polygon in pixel coordinates as shown in following screenshot: I have attached the datasets for better understanding.Does anyone have any ideas or potential solutions?
How can I convert a Multicurve geometry type to multiline geometry type?
Hi everyone, I have DWG files drawn in local coordinates and I need to transform them to shapefiles in a national projected coordinate system. This is currently done with Spatial Adjustment (Transformation - Similarity) as the links file (.txt) provided has two displacement links: -1000 -1000 "Destination X Value" "Destination Y Value"1000 1000 "Destination X Value" "Destination Y Value" p.s. I've just replaced actual numbers by "Destination X Value" etc. What would be the best way to replicate this process with FME? If it is Affiner how do I utilise the values from the links file? Thanks in advance!
I have a process I need to run on point clouds. In short, I tile them, sort the tiles, and remerge the tiles back into a single cloud. This is for compatibility reasons with a piece of software that doesn't like clouds that are not well-sorted spatially. The workspace is fairly simple; input, tiler, sorter. pointcloudcombiner, output. It works well and does what I want when run on a single input cloud. However, I need to run it in bulk on large numbers of clouds. And performance suffers. I realized this was because it was tiling every input cloud before sorting them and assigning them back to the combined clouds, so I added Group Bys. Then I changed the group bys to "Process when group changes" to keep the data moving. But each step holds another set of data "in process" when set that way. So instead of only keeping one cloud in memory at a time, it keeps 3. If I create a custom transformer of my steps and copy it as many times as I have input clouds, ea
Hello everyone, I'm a beginner in FME and need some help. I'm trying to join (1:1) data from our datalake with an existing GIS layer. My goal is to append the result of this join nightly to another existing layer. I've attempted to use SQLExecutor to query multiple tables in the datalake and DatabaseJoiner to join the query table with the GIS layer, but it doesn't seem to be working. Any suggestions or advice would be greatly appreciated. Thank you!
Dear All, Here is my issue with the FindString() function : I have the following string stored in user parameter value_to_test = "101,102" ; the attribute "ProgNum" contains values that range from 100-900 and I want to set a new attribute "FoundNum" to be = "true" when "ProgNum" is one the (comma-separated) values of value_to_test. As FindString(string,strToFind) should return the index in string that matches strToFind , I've tried the following expression :@FindString($(value_to_test),@Value(ProgNum))I have also tried the FindStringRegularExpression@FindRegularExpression($(value_to_test),@Value(ProgNum))None of the them is working ! Quite despaired I've also tried to revert $(value_to_test) and @Value(ProgNum) in the function call, but it does not work better... Does anyone have an explanation what it is not working and what I am doing wrong ?? Thanks in advance NB : I know that I could work with AttributeSplitter etc. but I would like to know why the FindString and FindRegularExpress
Hi, I have two datasets. Both are polygons. Dataset 1 is the area of a city minus a few areas within that are not relevant for this project. Dataset 2 contains houses as polygons. My goal is to create a third dataset of polygons of random size. The polygons have to be inside of the polygons of Dataset 1, rectangular (preferably not just squared), and within a random distance to the houses. Does anyone know a way to solve this?
Dear all,I am using a CSV reader to read my data from csv file. I have a column that contains almost only integer except 5 values, that contain a letter (e.g. 12a or 12b). Unfortunately the reader set the Type of this attribute to uint32, and the values that contain a letter are set to Null... So I loose this information...It seems that in older versions of FME there was possible to choose "Manual" for "Attribute definition" parameter. But I doesn't see this option in the 2022.1 version of FME.How could I do then to force this column to be read as a string ?I've seen that it is possible to specify manually attribute types in the FeatureReader, but could I use this transformer to read a file ? (I set the file path in the parameter, but I don't know what to pass in input, and the output is empty) Thanks in advance for any help
I would like to manipulate whole ''column'' (list of attribute values) in a PythonCaller but I don't really understand how I can do that. For example in the example here below, I create a new attribute in `input` (=iterating over each each feature, right ?) ; what I am trying to do 1) in `close` : to print all unique values of the newly created attribute(WITHOUT iterating over each feature, so to work directly on the list of all attribute values that my data contain) 2) in `close` (or somewhere else) : to add directly a new attribute to my data by applying a function on the firstly created attribute (again WITHOUT iterating over each feature) Is it possible to do all these things in a Python caller ? import fmeimport fmeobjectsdef processFeature(feature): pass class FeatureProcessor(object): def __init__(self): pass def input(self,feature): progs_dict = globals()[fme.macroValues['VARNAME_DICT']] all_prog_cols = list(progs_dict.keys()) all_
Hi, I will train a model to detect some objects based on aerial photos.For this I used these instruvtions decribed here:https://engage.safe.com/blog/2019/01/fme-does-computer-vision/After indicating the objets witch OpenCV I will preparte the data for training witch " RasterObjectDetectorSamplePreparer". Inputs are the source directory of the negative files and the annotation file. If I execute this I get an error: I don´t know what the problem !Does anyone have experience in this regard?
I have an Excel dataset. I am translating two time attributes named "Occurred_time" and "Reported_time" with this format:%I:%M:%S%p to this format:%H:%M:%S.Although it shows me the Preview Data, the result is rejected. I can't find what is the problem.
I have an SDE with a point feature class that has 120k locations across the entire US. I can read those points in with a normal reader, but it takes a long time ~5 minutes and I really only need to return 100 out of those 120k points. Which 100 changes depending on the input of an excel reader. I pull in a list of 10 or 20 points from an excel sheet, take a 2 mile buffer around those points, and then spatial join that to the big list from SDE. Those 10 to 20 points could be anywhere in the US. Is there a transformer that will let me read in my 10-20 points, then make a query to SDE/SQL with a transformer based on the location, so that I'm only asking the database for the records I need? That way I am not pulling in the rest of the 100k+ records. Right now I am thinking I can take the lat long coordinates from the excel input and create a bounding box of sorts to query the SDE point feature class. But I don't see a transformer that can read from SDE 'inline' with the
Hi,I am currently trying to archive information out of old DWG files in my company. But so far I am still rather uncertain with how I open the DWG files in FME in a way, that I can access the labels within the DWG file and write them into another file format.I found a lot of tutorials with how to write into DWG, but nothing much the other way around.Can someone help or point me in the right direction?
I possess a polygon layer containing sections where polygons overlap and also areas where gaps exist between them. My objective is to address only minor overlapping sections, specifically those where polygons overlap by less than 20 square meters. I wish to consolidate the gaps between polygons as well, targeting cases where the gap between boundaries is less than 20 meters. Is it feasible to perform this analysis using FME? I attempted to utilize the AreaGapAndOverlapCleaner in FME, but it appears to make adjustments for any type of overlap. However, in certain instances, two polygons may possess identical geometry, and I do not desire the overlap to be rectified in such cases.
Hi all, does anyone know if it is possible to draw only the outside walls of a building based on a ground surface and a roof surface. So ground surface is given and should not be modified and a detailed roof shape is given and should also not be modified. It is important for us to keep the original roof and ground and create only the walls.. We are using BrepSolidBoundaryCreator. As input we are taken the roof polygon and we do a negative extrusion to minal z value. But unfortunately we have as result a new ground surface and the old one does not fit to the walls. We want to keep the original ground surface. So is it possible to move the created walls to the ground surface subsequently?This would be necessary to take our original ground surface! Something like this:
Hi, I will create Walls with the transformer BRepSolidBoundaryCreator.Beta. As input I have roof surface and ground surface. The transformer should take ground surface and roof surface and should create wall surface (orthogonal projection). If the roof does not 100% match the ground polygon, the roof should be adjusted. At the moment the transformer creates new walls and in addition new ground. Thats great but nevertheless I will take my own roof and ground surface. Is this possible? Or is there another transformer which I can use?