Using transformers to modify data.
Recently active
Hello everyone,I was wondering if there is an environment variable available for the R interpreter.If I look at the following post, there appears to be an environment variable for python, but not for R.It would be nice that the R interpreter would be able to look at a certain folder on the C disk for example and when there is different version of R, then the R interpreter would automatically change it.For now, I manually changed the interpreter, however ideally I would like during an update of R, that the R interpreter would automatically change the R interepreter.Does anyone know whether that is possible or might be possible in the near future?
Hello,I am using the SQL Executor to read two tables from oracle using a join.I would like to use Oracle Spatial Object as Format to have the geometry included.This is my sql query:select a.nm, a.fid, b.mlpst, b.featfrom t_csm ajoin t_fl bon a.clss=b.clssBoth table a and b have a geometry (GEOM). When I use SQL Executor - Oracle Spatial Object, how can I know the geometry of which table will be used ? I want it to be the geometry of table b but I don’t how to specify it. The geometries of these two tables are not the same and my next transformer is the neighborfinder so it makes a big different if I use the geometry of table a or table b.Thanks for your help.
Hi,I have 100 polygons that state the population within each polygon, I'm essentially trying to break it down 1 stage below where I have that population essentially scattered across the polygon boundary in the form of points to then get a rough estimation of the population in small selections (and not for the entire polygon) I saw a question asked very similar to what I need where I have the population value in a field and I would like to add that as an input into the RandomPointGenerator (however this transformer does not accept inputs (aside from the fixed value you state), and this does not seem to have been changed in the 5 years since raising this question). Someone had mentioned the use of workspace runner, but I am not sure how to incorporate that into what I'm trying to do.
Hi all,When using a FeatureReader, FeatureWriter, or any other transformers, I can’t seem to access the Coordinate System Gallery. However this doesn’t seem to impact readers or writers (when accessed through parameters anyway).Excel ReaderReprojector (note that Source Coordinate System does use the Coordinate System Gallery)I’m currently using FME via Data Interoperability through ArcGIS, is this a limitation of the extension? Or have I missed a setting to use the gallery in all cases?Cheers for any help.
I am using the HTTPCaller to interact with an API, for that I use the setting Multipart/ Form Data. This is the way another user got the Endpoint to work for them and I just always used it. This works pretty well, the API does what I want, the connection is stable, but there is one problem. Sometimes it takes a lot of time to configure the Multipart-part.For some jobs I need to configure dozens of parameters, this leads to a lot of typing, proofreading, etc. For this particular endpoint I don't just need to give parameters about the object I want to edit (some inspection), but also something like a table definition. The row (row_id) and the value, for each attribute of my dataset. Since this API endpoint doesn't like it when I send in null values as values, I need to run several configurations of the same HTTPCaller one that drops each variation of Null values in the dataset. Now I was wondering if I could somehow build the whole statement about the rows and their values with something
Greetings, I should read and write features one by one, namely in the feature reader I have a condition by which I retrieve data from the database, more precisely the ID. But every time I should first write a feature in the database that gets a new ID, and when reading the conditions for the second time, it should select that new ID, and not immediately select that first ID for me. how to create this?
Hi!I have a shapefile with a lot of lines with the same geometry, but different data. They are the fibers of an optical cable.How can I shift them parallelly to each other?
Hi,I would like to write custom hatches in a DWG file. But our FME Flow is deployed with Kubernetes.So the OS is Linux.Since the RealDWG writer is only available on Windows, I will need to use the other DWG writer (Acad).Where should I put these .pat files to write these custom hatches successfully?The documentations says:The ACAD format writer will look for custom hatch patterns in custom <filename>.pat files located in folder specified by the ACAD operating system environment variable, or in the related folder for any installed AutoCAD product.On windows, I have tried to create an “ACAD” environmental variable, pointing to a folder where I placed my .pat files. But no result.Secondly, will this be possible on a Kubernetes FME Flow?Thank you in advance Jasper
FeatureReader does not recognise attributesHello all, I am having problems with a FeatureReader that does not recognize the attribute columns of a CSV file.The background is, I regularly download the GNAF dataset from Geoscape Geocoded National Address File (G-NAF) | Datasets | data.gov.au - betaThe GNAF arrives as a zip file containing a large number of CSV files. I only need five of them.I have built a Directory and File Pathnames Reader that returns all of the CSV files, which I then filter down to just the five that I want.I am attempting (but failing) to build a FeatureReader that reads the CSV file.I should point out that the CSV file is actually a pipe-separated-value file. The separator is a pipe “|” and the file extension is “.psv” But I have confirmed that a normal CSV reader does successfully read the psv file.Now, the FeatureReader is successfully reading the correct number of features. And FME automatically recognises that two of the columns are LONGITUDE and LATITUDE and
Current Version: 2024.0.3.0 (20240603 - Build 24220) Since upgrading from 2023 to 2024, some weird behaviours have happened.Some critical workflows have not been outputting the correct information on our public facing website.In this case, I’m referring to the Building Control (BC) header on that page after you type in an address.Instead of stating that there was a BC information against a property, it would say no information found.Whilst investigating I found the following: The result of SQL Query was outputting NULL values as a string value called {NULL}.Never seen that before. Therefore, the Conditional Value was returning the wrong information. i.e. If xxx IS NULL and xyz HAS A VALUE….Subsequently I had to add a StringReplacer to fix this anomaly for now. SQL StatementWITH bc_html_aggAS(SELECT blpu_uprn, ARRAY_AGG(bc_html) AS bc_html FROM myproperty.buildc_html_temp GROUP BY blpu_uprn ORDER BY blpu_uprn)Select Distinct(bc_html_agg.blpu_uprn), b.address_string, bc_html_ag
Hi,I want to write a RealDWG using custom hatches on FME Flow.When working on FME Form, we put the hatch files in <FME install folder>/plugins/realdwg/Support and it works. But when uploading the workspace to FME Flow, where should these files be placed? Is there a designated location for them in the Resources?Any help appreciated.Cheers,Julien
Hello!I have a GIS layer with a field named “yttyp” wich contains all various of attributes since its a text type.I also have an Excel-file with allowed attributes. How can i check if there is unallowed attributes in my GIS layer that is not in my Excel-file, and list them in a new excel-file. I have tried change detector, listsearcher, featureMerger but i dont get it to work.
I’m trying to retrieve data from a nested JSON response of an ODATA Feed.For testing, I try to access the “Emails” property of this ODATA example service: https://services.odata.org/V4/(S(hg4u5ox0x0r2rjva5qwwra05))/TripPinServiceRW/PeopleThe JSON structure is as follows:{ "@odata.id": "http://services.odata.org/V4/(S(hg4u5ox0x0r2rjva5qwwra05))/TripPinServiceRW/People('russellwhyte')", "@odata.etag": "W/\"08DC9753B5381A6D\"", "@odata.editLink": "http://services.odata.org/V4/(S(hg4u5ox0x0r2rjva5qwwra05))/TripPinServiceRW/People('russellwhyte')", "UserName": "russellwhyte", "FirstName": "Russell", "LastName": "Whyte", "Emails": [ "Russell@example.com", "Russell@contoso.com" ], "AddressInfo": [ { "Address": "187 Suffolk Ln.", "City": { "CountryRegion": "United States", "Name": "Boise", "Region": "ID" } } ], "Gender": "Male", "Concurrency": 6385516329
Hello all,Since I wasn’t able to find anything in the FME documentation about this and despite reading hundreds of community posts, I wanted to share what I’ve learned about using the GoogleDriveConnector transformer to delete files in Google Drive Shared Drives and Shared Folders.When using the GoogleDriveConnector in FME to delete files you need to make sure you have the right access. Shared Folders and Shared Drives have different types of access levels. For Shared Folders you have "Editor", "Commenter", "Viewer". If you want to be able to "Delete" files from these folders you will have to have an "Editor" level access right to the folder and the files.For Shared Drives, the access levels are "Manager", "Content Manager", "Contributor", "Commenter" and "Viewer".Due to the restrictions on the Shared Drive, its folders, and the difference in the actions each access levels can have, if you want to delete files from a Folder in a Share Drive folder you will need to have "Manager" level
Hello Forum, I need to start uploading data to a MS Azure Blob Storage account set up by our IT service providers.I have a Creator connected to a FeatureReader which is reading in the source file. The FeatureReader is connected to an AzureBlobStorageConnector transformer.But I first need to set up a new Web Connection through FME Options… I asked our IT provider for the Access Key ID and Secret Access Key of the Azure Blob Storage. They responded by saying that I should just use my standard network username (for the Access Key ID) and my standard network password (for the Secret Access Key). This really doesn’t sound right to me. I was expecting Access Key ID and Secret Access Key for the Azure Blob Storage. As I’ve not used Azure Blob before I don’t know what an Access Key ID and Secret Access Key would look like, but I’m sure it’s not my network username and password! Has anyone had successful experience with using the AzureBlobStorageConnector? If so, what did the Access Key ID and
Hi, I’m fairly new to FME and having some issues with the PointOnRasterValueExtractor.All datasets are in EPSG:7850 (GDA2020, zone50) and the PointOnRasterValueExtractor seems to be working as I can see the correct values when I interrogate the points.The issue is when I try to write the output to a Shapefile. The Q__band values are all zeros.What am I doing wrong here?
Hi all,I am looking for a transformer that does the opposite of the ListKeyValuePairExtractor.It needs to create a list with FIELD_NAME and FIELD_VALUE pairs for all the attributes on a feature.I checked the transformer gallery and the FME Hub, but could not find such a transformer.Has anybody a solution?Could be a Python script. Thanks in advance.Erik
I have a bunch of ZIP files which we download the contain data in 3 formats, shape, csv and file gdb. The zips also contain numerous other files such as pdf and xml, which we don't need to read. Now the tricky bit, I am using a PATH reader to check a directory to see what ZIPs are there and to only load the ones that have arrived in the last 24hrs. The load is triggered in a Sub workbench where the Path is passed to a FeatureReader set to Generic. I can add I file type filter to the path that is passed by adding [**\\*.shp], for example to the path of the zip found by the PATH reader and can read the ZIP 3 times with a different filter each time by building this string""@Value(path_windows)[**\\*.shp]" "@Value(path_windows)[**\\*.csv]" "@Value(path_windows)[**\\*.shp]"" but can't work out how to add filters for csv and gdb to read the file once and look for shp, csv or gdb. The above works, but I get error messages for the iterations th
I have a somewhat large (~180k lines, 100 cols) set of features coming out of a database reader that I'm loading into a PythonCaller to run some heavy processing on. I've been doing some profiling and it's a big contributor to slowdown in my workbench. To my surprise, though, the number crunching only represents about 10-20 seconds of the 90 seconds runtime for that node. Most of it is actually spent in the hot loops that run feature.getAttribute (another 25-30s) and feature.setAttribute (about 50s). Meanwhile, using a FeatureWriter and a FeatureReader, all these features can be written to and read from Parquet files in seemingly less than a second. It's not exactly easy due to how FeatureReaders don't output the path of the file they've written to, and PythonCallers only have one input node, making it difficult to handle passing a temporary output file name, but it cuts down my translation time from 90 seconds down to 9 seconds. Are these the only two ways of d
Hello everybodyI am currently working with a watercourse network dataset (lines), which is divided into a new segment each time a new watercourse branch branches off or is integrated. Unfortunately, the dataset contains very obvious "errors" as the width of the watercourse between two consecutive segments is much too large and the orthophoto also clearly shows that there must be an error here.I am not that experienced in FME yet and so far I have simply determined the points between the segments and was able to assign the different values of the touching watercourse arms to them. My main task is actually to provide the person responsible with an extensive data set that quickly shows where the difference between two watercourse widths is greater than XX metres (parameterisable). I have currently arrived at the status in the screenshot below, where the intermediate points with the water widths are listed for each line. Ideally, however, I would like to have only one line per point with t
I have a line feature (electrical wire) and a polygon feature (containing buildings). I want to get the distance from the wire to all buildings, along with the parameters (angle etc.) when using the tool “near” in ArcGIS Pro.Is there a way to do this in FME? Either with a similar tool, or maybe create some sort of connection and use that specific tool in FME. Im working with FME workbench 2022.2
I have requested a raster file from an API (HERE Map Image API v3), and the response comes in the fashion of (I guess) base64 data.I would like to have a way to transform such data and store it in proper folders.Any idea? Regards.
HiI have 2 transformers (A and B) they have to provide a Sorter with inputs.If there isn’t any output from A, then this workflow C has to stop, but not the entire workspace. If there is output from both A and B, it has to continue.If there is only output from A and not B, it has to contiunue. Is it possible?
Hi!Does it exist an easy way (1 transformer) to compare 2 lists from differents dataset ?For example :dataset 1 - feature 1list{0}.NAME = Alist{1}.NAME = Bdataset 2 - feature 1list{0}.NAME = Blist{1}.NAME = CI would like to have the difference so here :=>on the dataset1, feature1 “A” is missing=>on the dataset2, feature1 “C” is missingthe index of the value doesn’t matter, juste the presence of the value is important.I succed with 2 ListExploder and 1 FeatureMerger, and I imagine It would be working too with ListExploder, FeatureMerger and ListSearcher, but maybe it exist a transformer to do that more quickly.Thanks!
I'm working on enhancing an FME Workbench to be more interactive.Currently, my focus is on a project aimed at creating study area polygons.Depending on their purpose, these polygons can vary from one another.Is there a way to incorporate a dropdown menu or a list at that junction to select which writer to run and which not to?