Using transformers to modify data.
Recently active
Dear all, I use FME to perform a complex comparison analysis between two large datasets. I am looking for a way to analyze various subpopulations in succession, in order to reduce memory usage and increase performance. I have defined the desired subpopulations in a Reader file, so they can feed the SQLExecutor transformers to obtain the subsets. After performing the analysis on the first subset (i.e. 'Apple') and writing the results to a report file, the next subset (i.e. 'Banana') should be analyzed and so on. How can I implement such a 'for each loop' within FME? Thanks in advance. Remy
I have a DWG reader reading a single DWG with approximately 60 xrefs and small at 400 KB. It takes just over 5 minutes to read. I have turned off all unnecessary options including read xrefs, so it shouldn't be opening all 60 xrefs. Any tips on speeding this up? Thank you. TylerFME(R) 2020.2.1.0 (20201130 - Build 20806 - WIN64)
Hi,I'm trying to create a workspace that reads any feature class of a certain geometry type (e.g. line or point) from an Oracle Spatial database and writes it into an SDE Geodatabase. Since around 200 feature classes need to be processed, a manual transformation is not possible (meaning that a reader and writer are created for each feature type individually). I have tried different approaches that did not work: - Generic reader: Expects as an input a "source dataset". What I have is an database, however (I need to provide its connection parameters via published parameters instead of providing a dataset) - Dynamic reader (merged feature types): An Oracle Spatial reader together with "merge feature types" and a filter on "Features to Read" -> "Feature Types to Read" (provided by a published parameter) did not work, because its schema does not change with the feature classes dynamically. I finally used a FeatureReader in combination with
Hello, I'm a new FME user and normaly only work with Tableau. I'm used to create calculations inside Tableau but now I want to do this with FME so I can fill our new datawarehouse. I've got a dataset with a start date (dd_begin) and an end date (dd_eind). I need to count distinct the customer id's per Month. In Tableau I've created 12 calculations per year. For example, if I want to count the number of unique customers in January 2020 I need below Tableau calculation: COUNTD(if [DD_BEGIN] < date(#01-02-2020#) and ([DD_EIND] >= date(#01-01-2020#) or ISNULL([DD_EIND])) then [CLIENTNR] end) for February 2020 the calculation would be: COUNTD(if [DD_BEGIN] < date(#01-03-2020#) and ([DD_EIND] >= date(#01-02-2020#) or ISNULL([DD_EIND])) then [CLIENTNR] end) And so on and after this I use those 12 calculated fields to create a bar chart for the entire year so every month is visible inside 1 visual in Tableau. How can I do the same calculations in FME or is there
Hi community! Can I change fonts of labels for a DWG file? Happy new year to you all!
I would like to merge them keeping the first value of the other attributes but, the main problem is that the values that repeat "two by two" must be merged in pairs and sometimes the pair of values repeats, so the "group by" solution is not valid, because it would group all the identical pairs into a single value. Advice? I hope I was clear.
Hello All,Happy Holidays!I have a quick question:I am using a fanout /fanout expression to extract records based on OPID and SUBMISSION_YR Is it possible to get rid of the of the feature dataset from showing up , so only a feature class is within a FGDB (File geodatabase) ?? The output is in ARCGIS see screenshots below
Hi,I am using a Surface Modeller during the creation of TINs for 3D output. The majority of the time this works absolutely fine but some processes seem to stall at the same point (starting to build surface model) and I can't seem to work out why.It succeeds when there are more input coordinates than shown in the final line above. It succeeds with some more complex topographies than those that fail. It fails when there is only 1 Group in the output.I have tried altering the Surface Tolerance to a larger number but this made no difference, except to make the successful output more generalised!It occasionally stalls when building the Voronoi Diagram too, although we use the Triangles output in the process as opposed to Voronoi.It fails in both Desktop and Cloud environments.Thanks in advance!
I have a DWG file that contains some arcs that are recognized as arcs and some as polylines with multiple segments. I would like turn all of them to arc with two nodes (start and end) with center point, radius and a rotation (cw or ccw). What would be the best steps that I would be able to take to achieve this?
new to FME so i apologize for the elementary question, but is there a way for me to bring in all features within a feature service at once so that i can process them with a python caller? i need to run counts and reference external csv files on the data and i can't do it properly without having all of the records. for example, if i wanted to send an email that contained a list of all employee names within a feature layer, how would i do that? currently its sending an email for every employee within the feature layer (100 employees = 100 emails) i need 1 email that contains 100 employee names.
For the life of me, I cannot find a transformer that does this simple, often-used data transformation. I do NOT want to aggregate the values when I pivot as the AttributePivoter forces me to do, and the AttributeTransposer makes no sense to me at all; I've tried about every combination of options with it with no luck. I have given an example of the transformation I am trying to achieve in the attached picture. If anyone is familiar with Power Query within Power BI, it would be the same as pivoting an attribute column on top of a values column (with no aggregation). Similarly, this could be achieved by the pandas' pivot dataframe function.
Hi,I'm using FME to query an API. I'm passing long-lats as a comma separated value to the API, but FME is turning the comma into a url character (%2C) so the request is failing. Is there a workaround for this?ThanksN
I have a workspace which has worked previously but now keeps giving me the following error message: Attempting to retrieve global variable 'pmi_is_error' which was not set -- setting it to the empty stringFailed to create process, error was 'Access is denied. Does anyone know what this error might be associated with with or how I could approach solving it? I've tried searching for it but haven't found anything useful so far. Thanks
This must be trivial but after long workday my brains are not working.I have a text file which I need to convert to table. Here is a simple example to demonstrate the problem. Input text file contains street names for cities. City-keyword indicate when the city change.Input text file: Output table:
Hi pretty new to this software/role as BI Developer. I have noticed that when using - @DateTimeParse(@round(@DateTimeNow()),%Y%m%d%H%M%s) In an attribute creator it returns an undefined value of - 19700101000009+00:00. What is the correct way to do this so it returns DateTimeNow.
Hello, After struggling with several transformers, I can't figure out how to simply remove duplicates from 2 identical Excel files with exact same column names, etc ... but with different numbers of rows. First file contains 3130 entries and second one 2926. What I want is to be able to extract the 204 missing lines to check their differences. In my small brain, a simple transformer that checks missing entries between two columns and group them in an inspector should exists but it seems not to be complicated enough ... ;o) Thanks for any help
I have been able to do this using the HTTP caller. The issue I have is the the URL that I am calling changes slightly each week. The number in the URL increases by one each week. Is there a way I can automate the change in the URL so that it will retrive the up to date document from the link? Thanks in advance for any assistance.
We are needing to migrate fiber data from a typical GIS-based fiber platform into the new Esri Communications Data Management model (uses Esri Utility Network), however, the data structures are completely different when it comes to associations for items like fiber strands and equipment ports.
When I select a string function from the left banner of the the 'Text Editor' window, I notice that the function in the main window will only indicate required attributes to specify. For example, when selecting 'FindString', in the main window the example text '@FindString(<string>,<strToFind>)' is shown. However, if I look at the documentation on the string functions: https://docs.safe.com/fme/html/FME_Desktop_Documentation/FME_Transformers/!Transformer_Parameters/StringFunctions.htm,I see that also the following optional parameters can be passed on to this string function: [startIdx] and [caseSensitive]. What I think is a bit strange however, is when I select a math function that has optional parameters, it will show the optional parameters for that function in the main window of the Text Editor/Arithmic Editor. Consider for example the floor function; selecting it will show '@floor(<float>,<precision>)' in the main function of
I am running FME(R) 2019.2.3.2 (20200320 - Build 19825 - WIN64)I am trying to convert a cloud-optimised geotiff into an ecw image (v2)The image has GDA2020 datum, MGA Zone 56 projection, which is EPSG: 7856https://epsg.io/7856 The output ecw does not have the correct values for datum & projection in the header. I can use ERDAS Imagine to read the header valuesThis is the result from FME (as per attached image from_FME.png)Datum EPSG: 1168Projection Name SUTM56EPSG Code 0 To compare, I used ERDAS Imagine to perform the same conversion and these are the header values (as per attached image from_Imagine.png)Datum GDA2020Projection Name Transverse MercatorEPSG Code 7856 I would prefer to use FME as it does bulk translations well. But I need the header information to be correct. How to fix?
It seems to me like I'm looking for a transformer very similar to the ExpressionEvaluator with 'Evaluation Mode' = 'Overwrite existing Attributes', only without the numeric evaluation (that doesn't seemingly 'wrap' the expression in a arithmic editer). I thought about posting an idea about this (e.g. create new transformer or add parameter 'evaluation mode' = (text|arithmetic) to ExpressionEvaluator), but just wanted to check if I'm maybe overlooking existing functionality. Attached is a workspace that illustrates the numeric evaluation possibilities in the ExpressionEvaluator, and an example of how I would like to have the option for a 'text evaluation' possibility to e.g. add '_concat_@CurrentAttribute' to each attribute in the selected group.
When I use any SQL reader if I limit the attribute in the "User Attribute" tab with the option "Exposed Attributes": FME seems (regarding perf) to use a query like "Select Attrib1, Attrib2 from..." in place of a "Select * from ..." when I use Feature Reader It fetch all column even if I output to generic port and expose only some attribute. I have some Huge binary and text collumn that I dont need and I fetch thru a slow connection. Any Idea ?
Hey community! I need to know if I can export in a python script language my total transfomation of my workflow. Can I do it? And how can I make it?
So, I have a linear dataset of rivers. I have developed a RIVER_ID that serves as a unique ID per river. Saying this, one RIVER_ID usually has more than 1 unique ID depending on how long the river is. I firstly created them grouping by name in the network topology and using the Network_ID as a RIVER_ID. Later, I calculated Strahler order and used that to assign to unnamed rivers. So, what I want to locate are rivers (same RIVER_ID, could be multiple unique features) that touch a different RIVER_ID minimum two times (think about river sleeves). So, I went with Spatial Relator but I am now realising that a Spatial Relator will only give me 1 related candidates if my Requestor is touching the same IDd river at two locations and not two as I was hoping to get. Another thing that pops on my mind is using a snipper to get the first and last vertices of my Requestor feature and find a number of overlaps with other features and generate a list with these overlaps and check if there are two sam
Hi,I'am working on my first Augmented Reality model in FME. I have data from cables, sewers etcetera. As an example I found the webinar (https://www.safe.com/webinars/data-in-the-real-world-fme-ar-and-fme-data-express/). This webinar comes with some example workspaces. The workspace CoquitlamInfrastructureAR.fmw seems to do a lot like I would like to do. It looks way more complicated than I'd expected. It's even harder to go through cause the data used is missing. There's a feature reader reading shapefiles, bur the shapefiles are not included. There's also a google sheets reader that I can not use.Can someone provide the data? Thank you in advance,Arno