Using transformers to modify data.
Recently active
hi! I have a very large text file which i would like to split. I tried using attributesplitter, but this transformer deletes delimiter. I want to keep it;)I also tried stringreplacing the delimiter by adding a new delimiter. Like this: <OLD DELIMITER> changed in <OLD DELIMITER><NEW DELIMITER). When splitting using <NEW DELIMITER> i can keep the old one... BUT... this proces is very very slow, since i have millions of delimiters. Also using old delimiter, then losing it, and then adding it back in again, takes ages... any idea's??? prolly there is a python trick. But i'm a complete noob;) thanks and cheers,ronald
Hi, everyone, I have road data of 2 cities, roads at their boarder may have same attributes like name, width and lane and I want to know if their attributes are the same. If there's only 1 road on each side, it's easy by using list. But like the figure below, if there're more than 1 road on each side, I can't find a way to build the right list. Maybe road A in city 1 and road D in city 2 are same road, or road A in city 1 and road C in city 2 are same road, so I want to build two lists with road attributes( road A and C, road A and D). No matter I use PointOnLineOverlayer or Intersector, I can only build a list with all roads attributes in it. Road A, B ,C ,D intersect at same point on the boarder. Can anyone help me out? I want to build 4 lists, A and C, A and D, B and C, B and D
I was wondering if anyone had any information on best practices for junctions and bookmarks use when building Custom Transformer? I have a pretty excessive custom transformer built that takes hours to complete. It is a looping transformer that utilize multiple blocking transformers (Group By), list, and many pipelines branching off junctions. I am trying currently taking steps to tune its performance. The log file had junction appearing over and over, leading me to suspect that eliminating them will reduce overall compute time. I am experimented with collapsed vs expanded bookmarks using cache feature, and yet to find differences--my logic was collapsing bookmarks reduce feature caching, therefore speed up compute. Thanks and cheers!! -Alex
For what specific reason to connect an "inspector" transformer instead of viewing the data through the "magnifying glass" of the last transformer? What is the substantial difference between the two methods?
I have FME running as container, which uses FTP to connect and get a files from FTP location. we are migrating the FME to Azure Cloud and deploying container in VM. we would like to use azure file share instead of FTP. could anyone help us on this ?
Hi all, I have an FME process that I run nightly to refresh a polygon feature class in my SQL Server geodatabase. What I'm trying to do is truncate the feature class first and then insert new features into the feature class. It has worked successfully in the past, but it's not working properly at the moment. The truncate call fails and features get inserted into the feature class. Since the truncate is failing, now I have duplicates in the table. Please see the logs below.. 2021-01-04 10:59:26| 248.2| 0.0|INFORM|Geodatabase Writer: Truncating table/feature class `GISO.Cadastral_FME'2021-01-04 11:21:04| 303.4| 55.1|WARN |Geodatabase Writer: An error occurred while trying to truncate table/feature class `GISO.Cadastral_FME'. The error number from ArcObjects is: '-2147154945'. The error message from ArcObjects is: {Table schema changed after cursor declared [42000:[Microsoft][ODBC Driver 17 for SQL Server][SQL Server]Could not complete cursor operation bec
How can I correct the timing of the data flow in this case? In the first "run" the data stream does not pass through the FeatureMerger. I think the problem is the 2DGridCreator which creates 100,000 cells before the FeatureMerger and which takes a long time. In the attached image I show how the transformation is after the first "run". Which transformer should I insert? And where should I enter it? Should I use the "TimeWindower"? How?
Hi everyone,I have 2 datasets:1. Polyline set (with x,y,z coordinates)2. Points set (with x,y,z coordinates)The second dataset I generated by using my FME workbench to extract the vertices from dataset 1 (with the chopper transformer), and setting its z to the correct value that it got from a 3th dataset (with neighbourfinder) which contained the "right = measured by a surveyor" z value for dataset 1.Now I have dataset 2 that contains points with the correct x,y,z, and a dataset 1 that contains polylines which vertrices correspond with the points from dataset 2, except for the z value.Is there a transformer that sets the z value of vertices from dataset 1 to the z value of any point from dataset 2 which overlap/ are the same as that vertex?thanks in advance.
Hi everyone, I have a points shape (.shp with Z value) that contains a field with measured Z-values (called Zvalue and is a float), which I copied from a different point shape in my workbench. The idea is that the Z value (the coordinate) is set to the value in it's Zvalue field. Does anyone know how this is done in FME?
I have 78 cells divided into 9 groups (attached image), and I need to dissolve in 9 rectangular in the right order (ascending), but, if I dissolve, it doesn't make it in a order ascending (attached images of attributes). Any advice?
Hello there is there a way we could build a 3D model of an entire city, from Pointcloud data and 360 images, the goal is to have a 3D model close to reelThe mobile Mapping that generates point cloud is done for the entire cityThanks
Look at the picture please
I have a polygon dataset, which amongst other things needs to have the coordinates of its vertices rounded to a centimeter. It seems that CoordinateRounder is not topology-aware, so I end up having self-intersections, which I cannot (completely) repair afterwards. I'm new to FME, so maybe I am missing something here?
Dear all, I use FME to perform a complex comparison analysis between two large datasets. I am looking for a way to analyze various subpopulations in succession, in order to reduce memory usage and increase performance. I have defined the desired subpopulations in a Reader file, so they can feed the SQLExecutor transformers to obtain the subsets. After performing the analysis on the first subset (i.e. 'Apple') and writing the results to a report file, the next subset (i.e. 'Banana') should be analyzed and so on. How can I implement such a 'for each loop' within FME? Thanks in advance. Remy
I have a DWG reader reading a single DWG with approximately 60 xrefs and small at 400 KB. It takes just over 5 minutes to read. I have turned off all unnecessary options including read xrefs, so it shouldn't be opening all 60 xrefs. Any tips on speeding this up? Thank you. TylerFME(R) 2020.2.1.0 (20201130 - Build 20806 - WIN64)
Hi,I'm trying to create a workspace that reads any feature class of a certain geometry type (e.g. line or point) from an Oracle Spatial database and writes it into an SDE Geodatabase. Since around 200 feature classes need to be processed, a manual transformation is not possible (meaning that a reader and writer are created for each feature type individually). I have tried different approaches that did not work: - Generic reader: Expects as an input a "source dataset". What I have is an database, however (I need to provide its connection parameters via published parameters instead of providing a dataset) - Dynamic reader (merged feature types): An Oracle Spatial reader together with "merge feature types" and a filter on "Features to Read" -> "Feature Types to Read" (provided by a published parameter) did not work, because its schema does not change with the feature classes dynamically. I finally used a FeatureReader in combination with
Hello, I'm a new FME user and normaly only work with Tableau. I'm used to create calculations inside Tableau but now I want to do this with FME so I can fill our new datawarehouse. I've got a dataset with a start date (dd_begin) and an end date (dd_eind). I need to count distinct the customer id's per Month. In Tableau I've created 12 calculations per year. For example, if I want to count the number of unique customers in January 2020 I need below Tableau calculation: COUNTD(if [DD_BEGIN] < date(#01-02-2020#) and ([DD_EIND] >= date(#01-01-2020#) or ISNULL([DD_EIND])) then [CLIENTNR] end) for February 2020 the calculation would be: COUNTD(if [DD_BEGIN] < date(#01-03-2020#) and ([DD_EIND] >= date(#01-02-2020#) or ISNULL([DD_EIND])) then [CLIENTNR] end) And so on and after this I use those 12 calculated fields to create a bar chart for the entire year so every month is visible inside 1 visual in Tableau. How can I do the same calculations in FME or is there
Hi community! Can I change fonts of labels for a DWG file? Happy new year to you all!
I would like to merge them keeping the first value of the other attributes but, the main problem is that the values that repeat "two by two" must be merged in pairs and sometimes the pair of values repeats, so the "group by" solution is not valid, because it would group all the identical pairs into a single value. Advice? I hope I was clear.
Hello All,Happy Holidays!I have a quick question:I am using a fanout /fanout expression to extract records based on OPID and SUBMISSION_YR Is it possible to get rid of the of the feature dataset from showing up , so only a feature class is within a FGDB (File geodatabase) ?? The output is in ARCGIS see screenshots below
Hi,I am using a Surface Modeller during the creation of TINs for 3D output. The majority of the time this works absolutely fine but some processes seem to stall at the same point (starting to build surface model) and I can't seem to work out why.It succeeds when there are more input coordinates than shown in the final line above. It succeeds with some more complex topographies than those that fail. It fails when there is only 1 Group in the output.I have tried altering the Surface Tolerance to a larger number but this made no difference, except to make the successful output more generalised!It occasionally stalls when building the Voronoi Diagram too, although we use the Triangles output in the process as opposed to Voronoi.It fails in both Desktop and Cloud environments.Thanks in advance!
I have a DWG file that contains some arcs that are recognized as arcs and some as polylines with multiple segments. I would like turn all of them to arc with two nodes (start and end) with center point, radius and a rotation (cw or ccw). What would be the best steps that I would be able to take to achieve this?
new to FME so i apologize for the elementary question, but is there a way for me to bring in all features within a feature service at once so that i can process them with a python caller? i need to run counts and reference external csv files on the data and i can't do it properly without having all of the records. for example, if i wanted to send an email that contained a list of all employee names within a feature layer, how would i do that? currently its sending an email for every employee within the feature layer (100 employees = 100 emails) i need 1 email that contains 100 employee names.
For the life of me, I cannot find a transformer that does this simple, often-used data transformation. I do NOT want to aggregate the values when I pivot as the AttributePivoter forces me to do, and the AttributeTransposer makes no sense to me at all; I've tried about every combination of options with it with no luck. I have given an example of the transformation I am trying to achieve in the attached picture. If anyone is familiar with Power Query within Power BI, it would be the same as pivoting an attribute column on top of a values column (with no aggregation). Similarly, this could be achieved by the pandas' pivot dataframe function.
Hi,I'm using FME to query an API. I'm passing long-lats as a comma separated value to the API, but FME is turning the comma into a url character (%2C) so the request is failing. Is there a workaround for this?ThanksN