Using transformers to modify data.
Recently active
I have multiple json files, each containing a database table schema definitions. For example, the extraction of one json file, breaks down to column names and data types (non-spatial), as shown in the attached Data Inspector image. How can I use this information to dynamically create a table in the database ?(Oracle).
I want to create a raster that shows the distance from each cell to the closest line in a set of polylines (a cable network). This is very easily done in e.g. QGIS, but I want to be able to do it in FME. I can't find any transformer that would allow me to do so. The closest I come is to use a 2DGridReplacer - creating a regular grid of vector points, then a Neighborfinder, and the rasterize it. But that seems the long way round.... any ideas?
I have a series of points, representing points at ten metre intervals along a road.The ten-metre intervals are the "driven distance" of the road, which is not necessarily the same as the geometric distance.Each point feature has its "driven distance" as a numeric attribute "TDIST". The units is kilometres.TDIST0.000.010.020.03and so onI can use LineBuilder to assemble the point features into a polyline feature.The result I want to achieve is to output the polyline feature to a file Geodatabase, with each node of the polyline having a Measure, where Measure is the TDIST attribute.I have considered the MeasureGenerator Transformer, but this will generate measures along an existing polyline based upon the geometric length of the line, not what I want.I already have the appropriate Measure values as the TDIST attribute of each point feature.I'm hoping to;1) convert the TDIST attribute of each point feature into a Measure of that point feature, then2) conve
Hello all,I am running FME(R) 2018.1.2.1 (20190319 - Build 18592 - WIN64)I have a dataset of points (along a road) and I want to use the LineBuilder to create lines.Each point has a ROAD_SECTION_ID (character field) which uniquely identifies the road.Each point also has a TDIST (numeric field) which is the distance along the road.I am trying to sort the points appropriately before they go into the LineBuilder. They should be sorted firstly by ROAD_SECTION_ID and secondly by TDIST.However, the result of the sort is wrong. The features are sorted firstly in ascending order of TDIST, then in descending order of ROAD_SECTION_IDI have reversed the order in the Sorter, but there is no differenceThe outcome is that the LineBuilder does not work like it should
I am attempting to write to CSV file where the filename includes date/timestamp. My situation is the same as the question here: https://knowledge.safe.com/questions/35824/write-csv-file-where-filename-includes-datetimesta.html. I've tried adding the timestamp to the name but it produces a CSV for each row of the original table. I tried a parallel Creator>DateTimeStamper> but it generates two CSVs. I tried using a FeatureMerger but I'm confused as to what to join on; My options are to join on _timestamp and a column from the source data, which don't match. Should I ask a new question for this particular instance of the problem or try posting on aforementioned thread?
I am trying to manage updates between a WFS and a GDB. Note that for this I am using the UpdateDetector as my organisation's current installed version of ArcGIS Pro (using the Data Interoperability tool) restricts me to FME 2018.The issue is that I have a number of fields which are being provided from the WFS in float format, and even when the figure is a whole number it is still represented as such (eg. 1700.0000). When this is stored in the GDB it is automatically trimmed to the whole number (1700) so it isn't being detected as a matching value and giving inaccurate information about updated records. I could in theory us the AttributeRounder to overcome this, but some fields have decimal figures containing useful information, eg. dollar values (2357.87) so rounding to 0 decimal places will remove this. I'd like a blanket solution that can be applied across all attributes- I need to apply the template to ~60 feature classes which require update management, and data changes
I have a workbench which takes two schemas, merges them using a FeatureMerger, takes the output from that and merges it with a third dataset schema. This is then fed into a SchemaMapper transformer, with:Source attribute -> Destination attribute mappingSource attribute -> Filter attribute value filterNew attribute = New attribute valueHowever, the output, when viewed with an inspector and when written to a GDB feature class, is identical to the input, albeit with the _schemamapper_row_ids attribute added (and appearing to be populated with data indicating the mapping has worked. My understanding is at the very least, the new attributes should be appearing to indicate the transformer is behaving as expected.
The goal here is to have FME query our up-to-date database for investigator names, use ChangeDetector to see what’s changed between a feature class in ArcGIS Desktop and our database, and update only what has changed in the local ArcGIS geodatabase layer. So far the workbench successfully updates the investigator name (changes the value from “IS THIS REPLACED?” to “Sandy Jones”. The only problem is that it also fills in null data for the other fields that occupy the same row. Is there a way to have it leave those areas alone?Here is my workbench
Hi,My problem/curiosity is similar to: some-polygons-disappear-with-dissolverdissolver-not-working-as-expected I'm working on FME desktop 2017.0.1.1FME level: RookieSituation: I put a buffer on a road network, and clipped a smaller buffer of that road network out of it. I want to assign a specific weight value to the remaining buffer. Working with two buffers created some extra lines within the polygons which I would like to remove before going on with further calculations. Before using the Dissolver transformer I used the AreaOnAreaOverlayer. Using the Dissolver transformer worked for all, but 6 locations. With working I mean removing the inner lines from the existing polygons. Two examples have been included with my issue. I tried CoordinateRounder, Snapper, and ArcStroker between AreaOnAreaOverlayer and the Dissolver. I don't have any experience with these transformers, but they didn't seem to solve the issue. Example 1, before the dissolve.Example 1, after the dissolve.
I try to relate boreholes (points) to 3D Solids from Cad. (autocad_original_entity_type = autocad_solid3d) FME does read the 3D Solids as surfaces. (fme_geometry = fme_aggregate / fme_type = fme_surface)The way I found to do this is to convert the surfaces to polygons with a SurfaceFootprintReplacer, then use a SpatialRelator to define the relation. But this takes a lot of time as I have quite a bit, quite big solids.So I wonder: Am I doing this correctly? Does an alternative solution exist?Thanks!
I am using the REST api to POST to transformations/transactdata. The Workspace that processes shapefiles works fine on Desktop and also runs in the cloud using Run Workspace, by dragging in a zipped shapefile. However, the job will not work through the API using any of these three methods:- when using the REST API interface, (POST /transformations/transactdata/< repository >/< workspace >)--- though I am able to see zip file data when choosing application/octet-stream as the Content Type, so I would think this would work- when using Postman- when submitting with POST from my own codeIn all cases I receive error 422, and a response like this:{ ... "priority":-1, "statusMessage":"Unable to open shape file `/data/fmeserver/resources/system/temp/tomcat/tempstore/fmerest/sourcedata/pgworkflows-shp2geojson_cloud.fmw/requestdata-ESRISHAPE_1.shp' for read", "status":"FME_FAILURE" } Can anyone provide insight? The full logfile is
I would like to convert GPS Time To North American Daylight Central Time
Hello!Can somebody please try to figure out why in some cases bufferer doesn't work. I have 2 streets with the same name. I'm creating buffer around them gruped by street name. I want those two buffers to become one buffer.But when I put buffer amount = 5 nothing is passed through bufferer. If I change it to 3 or 10 it works. For 1, again it doesn't work. I played with the buffer size but can't figure out some rule why it works with some numbers but not with the other.Here is the data if someone want's to try.
Hello all! I am trying to import either a shape file, or a feature class polygon into Smallworld. It seems that I keep running into a geometry_acp error. Has anyone ran into this before or have any advice? I can't seem to get any other information from FME on what exactly is going on. I am using a GeometryValidator transformer but that doesn't seem to affect my results in anyway either. Any help is appreciated!
Hello!I have SHP file with the bunch of streets. There are many streets with the same name which is ok - streets with the same name can exist. But also there are some streets with the same name which are actually one street but entered as two different features instead of the one feature. So those streets have two geometries and the same name and are actually one street. I want to isolate (select) such streets. Is there any suggestion how can it be done in FME?So, for example, street with the name "Oak street" can have splitted geometry in one location but also there can be many other "Oak streets" in other locations (some of them regular with only one geometry, but also maybe other with also splitted geometries).So, the idea is to select streets with the same name which are in the neighborhood of 100m (for example 100m - the assumption is that those are most likely streets with the splitted geometry).I would appreciate any help and thanks in advanced!
When I execute the query in TOAD there are 1.6 million records are fetching. But when i used the same query inside fme it fetched only 138000(138 thousand records), And i Am using http caller in this fmw file. Is the http caller culprit here?fmw screen shot attached.toad shows 1.47 millon because already 138 k records written to destination and we changed the flag field nax match status to yes. and in this query we take only nax match status is NULLexplanation : I used this statement inside fme using executornext i am creating text string to ingest to HTTP CallerHttpcaller takes approx 1 second to get the responseeverything worked good, fme said successful, No error msg. But only 138 thousand came to destination table. i am really surprised.how can we fix this issue?
Does anyone know how best to use the 'LogMessageStreamer' transformer in FME to capture all logging errors. I know it has to run at the end of the Workspace to do this but as it has no input it is difficult to do this even when you set the run order to be last. I have also tried to make sure it is the last transformer I add to the Workspace but when I look at the log file after a run it has not been executed last, so if an error occurs after this it will not pick up the errors or failures from the Workspace run. I need to do this because these Workspaces are scheduled on FME Server and we need to know if something is failing over the weekend and keep a record of the errors or failures without going through lots of log files on the Server. This transformer is the perfect one to use for my purpose if it could pick up all the errors during a Workspace run as some of these errors do not necessarily terminate the Workspace. I am trying to be a bit more sophisticated with the capturi
I have some vehicle GPS tracking lines. The lines have gaps in them. I want to join the lines.The data is ordered and categorised so I can use a Chopper to chop the lines into vertices and then a LineBuilder with "Group By" set to the vehicle Type and "Group By Mode" set to "Process When Group Changes" to rebuild the lines with gaps joined.However, in the source, consecutive lines may be a different vehicle of the same Type (I can see they are different vehicles but there is nothing in the data to differentiate individual vehicles), so I get unwanted connections.There are two methods I could use to prevent most of these unwanted connections:if the distance to the next line is too far do not create a connectionthe source data contains start and finish times for each track, so if the finish time of the current line is after the start time of the next one, do not create a connectionBut I'm not sure how to do either of these - can anyone help?
Hi,Can any one help me how to use crossing in spatial relations like spatial realtor or spatial filter or any similar.I have seen in LinkPratap
When I run exercise 1.1 (intro to desktop) I get the following error message: "During translation, some features were read that did not match a reader feature type in the workspace. This can happen if the reader dataset is changed, or a reader feature type removed or renamed."My main canvas appears different to the one in displayed in the video. The "clean up attributes" and "fix inconsistent local area names" sections are both blank. As a result the "311-requests" writer is not linked to anything, and has no output. I have wasted a lot of time trying to figure out why this is happening. The video is no help at all. Thanks in advance.
I am trying to connect to AWS for one of my workbenches. when adding a new database connection, it succeeds, but if I try to use the same connection as 'embedded database connection' in joiner, this fails. error says FME edition does not support this reader. FME version 2017.1 being used
Summary:I need assistance applying a header (a feature that only contains field names) to a large dataset that is formatted without field names.Data:I have a massive dataset (~17 million records) that has been split into 17 chunks, each one containing ~1 million records. In each of these files there are ~150 unique fields, none of which have a proper name. So when loading a chunk of 1 million records, the field names default to col0, col1, col2 etc. Then I have a separate header file, which has the correct ordering of field names that needs to be applied over the default values.Problem:Applying the header to a large number of attributes stored in different datasets, without typing them individually.Bad solutions: Attribute manager / attribute renamer would work, however its tedious and not a good use of time to type out the field names of all 150 attributes. Bulk Attribute Renamer has a cool import button that can be used. Unfortunately the import option only works on one half (input o
I need to create a UAV mission route based off a grid of points within a polygon. Any tips, suggestions? I'm new to UAV mission planning. How can I get the points in the correct order?
I am looking at valve turning data from a hand held device which is stored in an access database.The database has torque charts stored as an attribute in one of the tables ,It is an encoded (UTF-16LE)My intention is to split this into separate columns and make an histogram chart from it.Any direction on how to get at it will be very much appreciated.
Hi, can anyone assist?I have thousands of .tif images and reprojected them (esriReprojector (custom transformation)), however they now all have black lines around them and that comes up in the mosaicked image. I found a couple suggestions that I tried; first I tried using the already mosaicked image and then reproject - while the blank mosaic is projected, once I get to the writer and add the rasters back, the data reverts to the original coordinates. Next I tried using the RasterBandNoDataSetter option but that gave me large white blotches all over my images... I am out of options now BUT I need to get this solved as the on-the-fly projection is not an option for our current project.