Using transformers to modify data.
Recently active
Hi Community,I have FeatureWriters creating log text files for me depending on various conditions.If the conditions are not fulfilled the files are still generated but empty and emailer still sends those empty files as attachment.How can i stop this from happening, so that only the log files with any values are sent as attachments with email.Thank you
Hi,I use a CSV list with standard ArcGIS server queries + HTTP Caller to get geojsons. a CSV list contains also layer names etc. I use the File option when saving a response because I need a separate geojson file for every query. Everything goes great, however response files look like "http_download_153028689823_5840.json" which is hard to deal with. Actually I can create a folder with a layer name but I would like to have an option to save a response file itself with a meaningful name and keep granular file structure. Ideally I see it as follows: I run a query within my CSV then get some attributes from the same CSV and supply a response filename from HTTP Caller with these attributes on the fly.There is a way to deal with a new Workspace and rename all the files using filecopy (upd - today I have finally managed with DatabaseJoiner for this purpose), but I am interested in a more integrated solution. Thank you in advance!Evgeny
Bonjour,J’ai une table avec plusieurs colonnes (23) chaque colonne à la valeur 0, 1, 2 ou 3 dedans. J’aimerais savoir une façon de calculer le nombre de fois dans une ligne que l’on peut retrouver la valeur 1, 2 ou 3 et mettre ce résultat dans 3 colonnes distinctes (une pour le 1, le 2 et le 3) voici l’exemple du résultat :Merci pour votre aide!
I know that there is a reader parameter for file geodatabases to only read in spatial data, but is there a way to only read in attribute data? I have an FME workflow that involves working with all of the attribute data and there is no need for me to read in the spatial data/geometry. However, my tool is getting hung up/stuck when reading in features from the file geodatabase - I suspect there's a multi-part feature with geometry issues, so I want to revise the tool to simply read in attribute data & not spatial data. Is this possible? Thanks in advance for any help!
Hello,I’m using HTTPCaller to retrieve a set of data via an API that I know works, as I tested it in Postman. The problem with HTTPCaller is that the response_body is not as full as what I received in Postman (I don’t have the full result), so I cannot continue with my process.I don't know if anyone can help?Thanks!
I have over 30 000 .png files that have been georeferenced. However, some of the files have up to 3 or four bands when only 1 band is necessary. So I want to change this. I want to change the number of band from 4 or 3 to 1 for each file and then replace the old one with the new så I dont loose the accompanied georeferencing files. I also want some type of iterator for this, since i have my files in a directory tree: Directory → Subdir → folder → Files
Hello,I have a API request working in postman, but cant get it to work via FME. If I could see the request headers/body, etc I would be able to debug the issue by comparing the working request vs the fme request.Does anyone have any ideas how I can see this?Thanks
I’m not sure if there is already a Hub or Custom Transformer for this, but posting my own version up called “FeaturePrioritizer” for any comments or use. I may upload this to the Hub if some belief is useful. 😉FeaturePrioritizer is a helper when the user needs to change the order of Features coming out of an Output Port to feed into a downstream Transformer or Writer that has a dependency on the order that features come into it.This is commonly needed when needing to say: Make sure that the first feature entering a Writer/FeatureWriter is a Schema Feature, or say when using an Aggregator that the output Attribute Parameter has been set to take only a single feature in the group (Eg.. The first feature in the group to enter the Transformer) to use as the Attribute output.Not much to it, behind the scenes this just uses a temporary sorting ID and a Sorter, but puts this all into a single Custom Transformer to make it easier.As an example, here is a set of Features their original order
Hey everybody,I'm currently trying to transform LAS file into Cesium 3D Tiles with FME Workbench. Las file is provided by an Agisoft Metashape project with EPSG:3945 coordinates. I would like to transform this points cloud into Cesium 3D Tiles with EPSG:4326 coordinates.I can't use Cesium ion for my project. Unfortunately, I obtain this strange result. First screenshot is the original file and the second one transformed file.Has anyone run into the same problem ? Can someone help me to handle it ?Have a nice day !
I need to export data of an oracle database to XML, it’s data from an electricity network, so not highly structured or nested.I created a workbench using the xml-templater and got some results, so far so good.Are there any limitations using the XMLtemplater compared to the XSD-driven Writer?I don’t want to create the whole export and realize later, that I would have better chosen the XSD-driven Writer for XML.Also a little problem with XML templater:I created an Attribute Value using Attribute Manager, a simple number “0.4”.When I write this value using the XMLtemplater it gets written as “0.40000000000002” to the Textfile.What am I doing wrong?
is it possible to automate my 2D to 3D workflow in ArcGIS Pro in FME? here is the process: 2D polygon layer with a height field → extrusion in ArcGIS Pro → “Layer 3D to Feature Class” Geoprocessing. Here, I group it based on a field so multiple polygons will be merged into a 3D multipatch. → then use this multipatch layer on top of the 3D AGOL terrain layer so they appear good on a 3D scene in AGOL.Does FME have a transformer & writer to automate this workflow? Thanks!
I have two spreadsheets. One is the original data; the other is data that has changed. The schemas are the same. The spreadsheet with the changes has 3 records. I need to add these 3 records into the original data (while deleting the original 3 records in the original data), thus updating the records in the original. What is the best way to do this using ChangeDetector?Currently, when I use ChangeDetector, the records from the changes spreadsheet are output to Inserted; The records from the original spreadsheet are output to Deleted. Deleted includes the 3 unchanged records that are in changes spreadsheet. In the screenshot below the 95,392 in Deleted include the unchanged records that are in Changes. The transformer outputs the Original and Revised separately with no way for me to delete the 3 records in 95,392 and add the 3 changed records?
I built a flow where an Excel spread sheet updates a snowflake data table. The spreadsheet in located in a SharePoint documents library. To get the flow to run on an online FME server I had to put in a SharePointOnlineConnector. When I run the flow, I keep on getting “Could not decode contents of ‘jjjjj.xlsx’ as ‘utf8’. I’m running FME workbench 2022.2 at the present time. Any ideas on how to clear this error would be greatly appreciated.
In a related topic I’ve asked about api’s and JWT authorization: The currently only way is to do this via python. I’ve got myself a script that I call via a systemcaller that runs the python script in a conda environment (not keen on installing extra python stuff in the FME locations). This works as I can fetch my bearer token to do api calls.However there is an expiration time on the bearer token (1hour).So now I’m wondering how to tackle this the best way:I want to do several http calls in parallel (default 25) but if I’m using the token voor 3500 seconds, I want to call my CT for getting a new bearer token. Any not processed calls (I need to make 3k+) should now use the new token. If I again come to this 3500 seconds another refers etc.So what would be a good way to achieve this kind of looping? As the http caller has a limitation that if there is a loop port, multi threading should be set to 1, beating my purpose.I also don’t want to duplicate transformers, because I don’t know ho
Working with crossborder maps that get their data from two different datasets. There are many ways to approach this, and I am toying with the idea to convert the attributes and their values from dataset A to the standard of dataset B. The benefit would be easier downstream processing. A simple example:Dataset A: attribute ‘Lakes’, value ‘yes’Dataset B: attribute ‘water’, value ‘1’Then do some FME-magic to convert everything from A to the standard of B, ie ‘lakes’ → ‘water’ and ‘yes’ → ‘1’I imagine that the only reasonably fast way to set this up would be by utilizing an external lookup table, csv or similar. How would one go about to change both the attribute and values from an external lookup table? I would like to keep it dynamic and easily editable, so preferably avoiding the AttributeValueMapper.
Hello,I was wondering if someone here could give some guidance to a problem we are experiencing.We our trying to connect FME to BigQuery but regardless if it is the reader or writer we are unable to connect using Web Connection.For the Web Connection settings we use or own accounts which have access to BigQuery. In fact when we connect directly to BigQuery we are able to see the project, datasets and tables and we are able to run queries on them.But when trying to establish the connection from FME using Web Connection we get the following error: BigQuery Connection ErrorAny help on how to solve the issue or to even find out what exactly is wrong would be appreciated.Thanks in advance.RZ.-
I have a table of data that I’m trying to map. The locations are essentially polygons with their bounding streets entered as attributes:I have street data and have been able to create polygons using AreaBuilder, but am stuck on trying to add all of the streets back as searchable/joinable attributes rather than items in a list.I’ve tried the ListExploder and gotten a stack of polygons, each one with one street’s attributes, but I aggregating isn’t quite getting me the results I’d expect. In a perfect world, I’d be able to run FeatureJoiner or something similar on every item in the list, but it only seems to allow one.Any advice is very much appreciated!
When python caller errors out.It does not show error line number. Message Type: fme::internal::_v0::py::ExceptionPython Exception <GEOSException>: LocateFailureException: Could not locate vertex.🟢06-Report (PythonFactory): PythonFactory failed to close properly
I’m looking at an area of survey points that can be divided into east and west areas by grouping with an attribute value. I have used Hull Accumulator to draw polygons around a) the whole area (light purple) and b) the two subdivisions (darker purple). The point population remains the same overall, but I get different external boundaries in places, when I was expecting the same. Alpha = 0. What would be useful is any description about how the transformer makes its drawing choices. I also end up with a donut hole. Again, am trying to understand how this occurs, as I assumed the transformer would create an external boundary only.Many thanks
Helly everyone,I am working with data that consists of abrreviations/code items that are strung together in a single field which I am trying to convert to full text. For that I am using a string pair replacer which contains thousands of mapping pairs, but I have encountered a problem which I can’t seem to solve without knowing how exactly the String Pair replacer does its thing. The problem arises when multiple codes share letters. Here`s an example: I have the following pairs somewhere in my replacement pair string**gn green**bn browng gravelF mudSo F:**bn**gn should map to Mud:browngreen but whats happening is this: M:brown**gravelr.I figured that the transformer probably checks its list from start to finish and ordered my list by string length in hopes it would transform longer strings first. The order of replacement items definetely has a big impact on the result and I was succesful of ironing out most errors in doing that but this particular translation and a few others still dont
So, in the context of pipelines we run camera inspections registering different damages and features along the pipeline. We have registered pipelines by ID in a larger gis database. I already have a script running that reads the information and returns all the registerd freatures in the camera inspections and draws it in a arcgis. The issue is that sometimes the lines have bends even with the same ID. making it hard to create new points at the specified distance. So what i need is something that creates a point at a specific length from the starting point ID, that takes into account the geometric changes of the line. in the example picture under, i want to draw a point at the POI. I have the distance(70m) from start. But there does not seem to be obvious functionality for this in FME. Any ideas?
Hi!Is it possible to create (writers etc) a Data Pump Script (DPS) file for store in Oracle in FME Form ?
Hello,I have a set of pairs of points that each have a uniqueID_start and unique_end field. I am trying to extract all of the line segments between each pair of points. The tricky thing is that the pairs can be several line segments long, and have other pairs intersecting them:There could also be several points on a single spot, but corresponding to different pair sets.It’s a mess! Any help would be wonderful!
Hello, I have a reader with a shapefile that has contour lines in it. I have connected that to the ContourLabeller transformer. I have taken the label output port of the ContourLabeller and run it to a DGN Styler where I have styled it. These parts are all working but when I write it using the DGN (V8) writer the label level is not created. What am I missing here? Thanks!
Hi there,I have severale profiles, which fly somewhere in 3D space.The elevation always follows the z axis. The direction varies, but follows one direction in XY level.I started like this:BoundsExtractor get the bounds (aka rotation axis) GeometryRefiner do some geometry refining 3DRotator rotate around the axisResult:3D view2D viewred: rotation axisgrey/black: original geometrygreen: rotated geometryI expected the green geometries to be rotated around the axis, but instead they are rotated into 3d space 😵💫 I also tried to set the axis through the geometry (z = zmax-zmin), but with no success.Can somebody help?Cheers,Maria