Using transformers to modify data.
Recently active
Hi,I am writing out to a json writer and intend to stream this json using fme cloud.The output looks like [ { "STAGING_ID" : "9515", "USER_API" : "A2C3601C-40E9-48DC-A0C0-5D4DB0867A2E", "REQUEST_GUID" : "80318a93-f513-4e63-8c08-9e07250af0e2", "ID" : "120654", "UNIQUE_LOCATION_ID" : "5819", "UNIQUE_SAMPLE_ID" : "576683", "ROW_CREATED_DATE" : "18/05/2010", "LAST_EDITED_DATE" : "18/08/2017",etcIdeally I would like to add a few lines before the records return e.g.[ "status":"ok", "body" : { "STAGING_ID" : "9515", "USER_API" : "A2C3601C-40E9-48DC-A0C0-5D4DB0867A2E", "REQUEST_GUID" : "80318a93-f513-4e63-8c08-9e07250af0e2", "ID" : "120654", "UNIQUE_LOCATION_ID" : "5819", "UNIQUE_SAMPLE_ID" : "576683", "ROW_CREATED_DATE" : "18/05/2010", "LAST_EDITED_DATE" : "18/08/2017",Could anyone suggest how this can be achieved. I really don't want to manually template the output from the rows because they are dynamic and remove any null values.Many Thanks,Oliver
I have a bunch of images in MrSID format (150 or more) with sizes from 3Mb to 10Mb each, and I need to get the bounding box that contains them all.What is the fastest way of doing it? My approach is as shown in the picture. It takes 1 minute for 3 .sid files (6Mb each). If reading a single SID file is slow, let alone 150. There must be some better way of doing this. Any idea?
Hi,I have a folder which including multiple files vectors. They have different coordinate system.Example: road.tab (EPGS: 4326), water.tab (EPSG: 32648)I want to make a process converting coordinate look like here:Could you show me how to make filter to reproject?
To get hold of a parameters value in Python, you can easaly use ex. FME_MacroValues['COORDSYS'] If the parameter is of the type Choice with Alias, how do I get a hold of the alias in Python syntax?
The documentation for the PythonCaller points to this page to describe how to install python packages for FME: https://docs.safe.com/fme/html/FME_Desktop_Documentation/FME_Workbench/Workbench/Installing-Python-Packages.htmAt the bottom the target directory is listed as C:\Users\<user>\Documents\FME\Plugins\PythonBut, in that directory are several subdirs: python27, python 34, python35. Do I install instead to a version-dependent subdir?i.e.:python -m pip install <mymodule> --target C:\Users\<user>\Documents\FME\Plugins\Python\python36
Hello,I am wondering if I could update the coded-value domains in FME transformation without re-adding the writer.Because I have spent so much time on re-order the fields in the way I want for many feature classes( 30+), and I don't want to re-do them every time I need to update something in the reader, so I don't think I would want to re-add the writers every time or choose the automatic in attribute definition as it would change back the order to default...So my question is, if I use the manual option for attribute definition, could I still get the updated coded-value domain from the updated reader without re-configuring the writer? For example, I added the coded value "Inactive" into the "Status" domain (first image), how could the "Status" in the writer get the updated domain from the reader? I tried so many times, but didn't have luck yet. The error message saying " The attribute definition used is invalid for the non-existent domain
After using the Orienter on a line the attributes that are 'end sensitive' are no longer in the correct fields because of the flip. I used to use AttributeCopier (v2) which did a fine job. But an "upgrade" has broken the swap, overwriting so fields. I used to just not upgrade, but now the workbench just crashes. I cannot create a new workbench with the old AttributeCopier so I am stuck with a kludge using a whole lot of temporary variables unless there is a more elegant solution. Is there?Just to be clear, I need to do this:Source TargetLTO RFRLFR RTORTO LFRRFR LTOetc for every "end sensitive" attribute. (You will remember that not only do you have to reverse the data values, but it has to be mirrored as well when a flip is done because left and right are also swapped.)Preferably I would like AttributeCopier to revert to the original behaviour.
Hi there. I am trying to import a simple csv file to my CARTO on-Premise account. I could import files before I updated my FME Desktop to 2018.1. Now I am keep getting following error messages.Creating reader for format: CARTOTrying to find a DYNAMIC plugin for reader named `CARTO'FME API version of module 'CARTO' matches current internal version (3.8 20180209)Creating reader for format: CARTOTrying to find a DYNAMIC plugin for reader named `CARTO'FME API version of module 'CARTO' matches current internal version (3.8 20180209)CARTO reader: Directive '_MULTIUSER_ACCOUNT' has value 'NO'CARTO reader: Directive '_API_URL' has value 'cartojp.com/user/carto/api/v2/sql 'CARTO reader: Directive '_API_KEY' has value '**********************************a72a4e'CARTO reader: Directive '_PAGE_SIZE' has not been set. Using default value '500'CARTO reader: Opening account 'myonpremise'CARTO reader: CURL e
Good evening,Is there a way to do a lookup (like a FeatureMerger) within a Python script? I have multiple attributes that I need to lookup against a single list, but I'm not sure how to supply this out-of-band (supplier) to the python without muddying/creating duplicate features. Basically I'm trying to avoid creating 20 FeatureMergers for each attribute lookup I need to do. Cheers,Peter
Is there a way to quickly remove all attributes that contain no data, i.e. every row is null.I don't really want to run a python script, I am wondering if I can to some statistics to count and then something to remove the attribute.Many Thanks,Oliver
Hello. I am using FME 2018. So, I have created a workbench that generates polygons as output. But, some outputs that are generated are polylines and not polygons.Now, if I edit the polylines(in ArcMAP) that are made, it becomes polygon. I don't know if its a bug or something with PointOnAreaOverlayer.Any solutions for this? I have attached the inputs and wb.Thanks
Is there an equivalent to Raster Calculator in FME? I am aware of the RasterCellValueCalculator transformer, and the custom made NDVI transformer, but is there also a single transformer to calculate the results of multiple rasters (more than 2) and/or to apply a custom function on these rasters? And, since the Rasters have to be perfectly overlapping, is there also a transformer to reproject and resample an existing raster so that it will copy the properties of the other raster?Or do I need multiple RasterCellValueCalculator transformers behind one another to get the same result? This somehow does not seem very efficient.
I'm trying to add label points to dissolved polygons and my label point output is far less than the number of dissolved polygons input to the Label Point Replacer. If written to the Inspector the number of label points (~4400) equals the number of inputs but when written to a database the number drops to around 100. Does anyone know what might be the problem and how to fix this?
Hi I am trying to add a grid shift file in workbench 2018.I have followed the steps listed in this page:https://docs.safe.com/fme/html/FME_Desktop_Documentation/FME_Workbench/CoordSys/grid_files_adding.htmHowever there is no option to add a new file.Any help would be much appreciated.
How can I combine a list of the same feature into the same oneFrom this Ply_idLoc_nameLayer_idSpatial_relation1MTL1510OVERLAPS1MTL1511OVERLAPS1MTL1512WITHIN to thisPly_idLoc_nameLayer_idSpatial_relationMsg1MTL1510 OVERLAPS, 1511 OVERLAPS, 1512 WITHINThanks
I have some features that contain an attribute with a GeoJSON string, I also have a Multipolygon in WKT passed as a global parameter to the workspace, what I have to do is to parse each geometry from the attributes (in geoJSON) merge them into a multipolygon and test if this intersects at least once the geoJSON mulitpolygon. At the moment I cannot parse the GEOJSON from the attributes, I tried to use geometryExtractor but the transformer instead of extracting the GEOJSON string extracts the geometry of the incoming feature (which is unrelated), i tried the feature reader but I get an error because the reader is looking fro a file path while I submit an attribute. I parsed the geoJSON with the validation online and it id fine. How can I accomplish this?
Hello ThereI am looking to get export all feature to the image but with the sat image background I used image fetcher transformer, the problem is I am getting only one feature on the image and the translation shows 5for the info I am using the Google Static image url
I'm receiving the following error using the salesforce connector. The connector returns 2000 features and then stops with this error. On another object, it returns 1,000 features then stops. My SOQL query runs fine in salesforce workbench. Any ideas? Python Exception <MissingSchema>: Invalid URL '/services/data/v43.0/query/01g6A00000RzDWPQA3-2000': No schema supplied. Perhaps you meant http:///services/data/v43.0/query/01g6A00000RzDWPQA3-2000?Traceback (most recent call last): File "plugins.py", line 441, in input File "salesforceConnector.py", line 259, in receive_feature File "salesforceConnector.py", line 151, in query File "salesforceConnector.py", line 133, in _get File "C:\\Program Files\\FME\\/python\\requests\\sessions.py", line 521, in get return self.request('GET', url, **kwargs) File "fmehttp.py", line 224, in request File "C:\\Program Files\\FME\\/python\\pypac\\api.py
I have line features from an Autocad DWG import that I would like to turn into polygons by adding some width to the line. The line may consist of one or more line segments. I'm thinking I could break the line into segments, create two copies of each segment, move the segments parallel to the original line in opposite directions and then create a hull around all the points. I'm not sure how to do this. Is there an easier way?
I need to output an SQL script created with SQL Creator in a custom text file. The file is essentially a csv file but with '|' as the seperator, the header row prefixed with a 'H|' the rows of data prefixed with 'B|' and a footer containing F|<file_id>|<datetime_stamp>|<number_of_rows> creating a csv file using '|' is easy enough but I haven't found a way to do the prefixes or footer. I was looking at using XML Templater but not sure if I'm barking up the wrong tree. Any guidance would be greatly appreciated.
We're trying to run a script in FME Server 2018 that replaces a geodatabase with a new dataset using arcpy. In a few months we'll be attempting to use FME's native geodatabase writer, but for now I need to troubleshoot why the following doesn't work:import fmeimport fmeobjectsimport arcpy# Main Functiondef delme(feature): # Variables table_2b_deld = feature.getAttribute('SOURCE_ETL_TABLE') sde_conn = FME_MacroValues['SDE_Connection'] full_del_str = sde_conn + '\\' + table_2b_deld + 'X' # print full_del_str # Process try: arcpy.Delete_management(full_del_str) feature.setAttribute('deletion_result','success') except Exception, e: logger = fmeobjects.FMELogFile() logger.logException(e,2) feature.setAttribute('deletion_result','failure') feature.setAttribute('deletion_result_detail',e) passAfter that script runs, we move data in Oracle from our staging schema into our production schema, then attempt a geodatabase registration script with a virtually identical syntax.If the geodataba
I looking to calculate the intersected area between polygonsI used the Areaonareaoverlay transformer but I am getting one of the feature removed and replaced by another one,
I am converting an excel spreadsheet to an esri geodatabase feature class of points. Working well so far, except for one field of elevation values. There was no standard of input and the values are varied. For example:75 feet75 ft.75 ft75f75 MSL25 m / 75 fIf it weren't for the last example, which occurs too often to waste time updating manually, I'd just use a string replacer to extract just the numbers, but in that last example, I'd end up with 2575 instead of just 75. I'm guess that first I need to remove everything before and including the '/', and then can use a stringreplacer with regular expression '[0-9]+' . Also, I've gotten another spreadsheet of data in which "##m" occurs. Those rows would need to be converted to feet. Hoping this is solvable with transformers, as the data is not to be edited.
I have a custom transformer that caches API call results to a PostgreSQL table. It tries a DatabaseJoiner on the key, and if it finds a matching record, it passes the joined feature directly to the main Output port. Otherwise, it calls the API, passes the result to a SQLExecutor to INSERT it for joining on same key value on subsequent features passed to the transformer.Unfortunately, the DatabaseJoiner is unable to find records stored by the SQLExecutor for subsequent features during the same workspace execution. I've tried explicit COMMITs (with appropriate delimiter declaration) in the SQLExecutor (it says no transaction is in progress) and a 5-second-per-feature Decelerator after the SQLExecutor in case there is some kind of race condition between FME Workbench and PostgreSQL. No luck. I still get duplicate key violations because the failure of the DatabaseJoiner causes the same key to be sent to the API and storage attempted again via the SQLExecutor. I've even tried
Good morning,I have a question about FMEI have many readers and writers that transform files into oracle tables, and he reads everyone and does commit only after all. I would like you to conduct a commit every transaction from reader to writer