Using transformers to modify data.
Recently active
Hello all,Since I wasn’t able to find anything in the FME documentation about this and despite reading hundreds of community posts, I wanted to share what I’ve learned about using the GoogleDriveConnector transformer to delete files in Google Drive Shared Drives and Shared Folders.When using the GoogleDriveConnector in FME to delete files you need to make sure you have the right access. Shared Folders and Shared Drives have different types of access levels. For Shared Folders you have "Editor", "Commenter", "Viewer". If you want to be able to "Delete" files from these folders you will have to have an "Editor" level access right to the folder and the files.For Shared Drives, the access levels are "Manager", "Content Manager", "Contributor", "Commenter" and "Viewer".Due to the restrictions on the Shared Drive, its folders, and the difference in the actions each access levels can have, if you want to delete files from a Folder in a Share Drive folder you will need to have "Manager" level
Hello Forum, I need to start uploading data to a MS Azure Blob Storage account set up by our IT service providers.I have a Creator connected to a FeatureReader which is reading in the source file. The FeatureReader is connected to an AzureBlobStorageConnector transformer.But I first need to set up a new Web Connection through FME Options… I asked our IT provider for the Access Key ID and Secret Access Key of the Azure Blob Storage. They responded by saying that I should just use my standard network username (for the Access Key ID) and my standard network password (for the Secret Access Key). This really doesn’t sound right to me. I was expecting Access Key ID and Secret Access Key for the Azure Blob Storage. As I’ve not used Azure Blob before I don’t know what an Access Key ID and Secret Access Key would look like, but I’m sure it’s not my network username and password! Has anyone had successful experience with using the AzureBlobStorageConnector? If so, what did the Access Key ID and
Hi, I’m fairly new to FME and having some issues with the PointOnRasterValueExtractor.All datasets are in EPSG:7850 (GDA2020, zone50) and the PointOnRasterValueExtractor seems to be working as I can see the correct values when I interrogate the points.The issue is when I try to write the output to a Shapefile. The Q__band values are all zeros.What am I doing wrong here?
Hi all,I am looking for a transformer that does the opposite of the ListKeyValuePairExtractor.It needs to create a list with FIELD_NAME and FIELD_VALUE pairs for all the attributes on a feature.I checked the transformer gallery and the FME Hub, but could not find such a transformer.Has anybody a solution?Could be a Python script. Thanks in advance.Erik
I have a bunch of ZIP files which we download the contain data in 3 formats, shape, csv and file gdb. The zips also contain numerous other files such as pdf and xml, which we don't need to read. Now the tricky bit, I am using a PATH reader to check a directory to see what ZIPs are there and to only load the ones that have arrived in the last 24hrs. The load is triggered in a Sub workbench where the Path is passed to a FeatureReader set to Generic. I can add I file type filter to the path that is passed by adding [**\\*.shp], for example to the path of the zip found by the PATH reader and can read the ZIP 3 times with a different filter each time by building this string""@Value(path_windows)[**\\*.shp]" "@Value(path_windows)[**\\*.csv]" "@Value(path_windows)[**\\*.shp]"" but can't work out how to add filters for csv and gdb to read the file once and look for shp, csv or gdb. The above works, but I get error messages for the iterations th
I have a somewhat large (~180k lines, 100 cols) set of features coming out of a database reader that I'm loading into a PythonCaller to run some heavy processing on. I've been doing some profiling and it's a big contributor to slowdown in my workbench. To my surprise, though, the number crunching only represents about 10-20 seconds of the 90 seconds runtime for that node. Most of it is actually spent in the hot loops that run feature.getAttribute (another 25-30s) and feature.setAttribute (about 50s). Meanwhile, using a FeatureWriter and a FeatureReader, all these features can be written to and read from Parquet files in seemingly less than a second. It's not exactly easy due to how FeatureReaders don't output the path of the file they've written to, and PythonCallers only have one input node, making it difficult to handle passing a temporary output file name, but it cuts down my translation time from 90 seconds down to 9 seconds. Are these the only two ways of d
Hello everybodyI am currently working with a watercourse network dataset (lines), which is divided into a new segment each time a new watercourse branch branches off or is integrated. Unfortunately, the dataset contains very obvious "errors" as the width of the watercourse between two consecutive segments is much too large and the orthophoto also clearly shows that there must be an error here.I am not that experienced in FME yet and so far I have simply determined the points between the segments and was able to assign the different values of the touching watercourse arms to them. My main task is actually to provide the person responsible with an extensive data set that quickly shows where the difference between two watercourse widths is greater than XX metres (parameterisable). I have currently arrived at the status in the screenshot below, where the intermediate points with the water widths are listed for each line. Ideally, however, I would like to have only one line per point with t
I have a line feature (electrical wire) and a polygon feature (containing buildings). I want to get the distance from the wire to all buildings, along with the parameters (angle etc.) when using the tool “near” in ArcGIS Pro.Is there a way to do this in FME? Either with a similar tool, or maybe create some sort of connection and use that specific tool in FME. Im working with FME workbench 2022.2
I have requested a raster file from an API (HERE Map Image API v3), and the response comes in the fashion of (I guess) base64 data.I would like to have a way to transform such data and store it in proper folders.Any idea? Regards.
HiI have 2 transformers (A and B) they have to provide a Sorter with inputs.If there isn’t any output from A, then this workflow C has to stop, but not the entire workspace. If there is output from both A and B, it has to continue.If there is only output from A and not B, it has to contiunue. Is it possible?
Hi!Does it exist an easy way (1 transformer) to compare 2 lists from differents dataset ?For example :dataset 1 - feature 1list{0}.NAME = Alist{1}.NAME = Bdataset 2 - feature 1list{0}.NAME = Blist{1}.NAME = CI would like to have the difference so here :=>on the dataset1, feature1 “A” is missing=>on the dataset2, feature1 “C” is missingthe index of the value doesn’t matter, juste the presence of the value is important.I succed with 2 ListExploder and 1 FeatureMerger, and I imagine It would be working too with ListExploder, FeatureMerger and ListSearcher, but maybe it exist a transformer to do that more quickly.Thanks!
I'm working on enhancing an FME Workbench to be more interactive.Currently, my focus is on a project aimed at creating study area polygons.Depending on their purpose, these polygons can vary from one another.Is there a way to incorporate a dropdown menu or a list at that junction to select which writer to run and which not to?
I am trying to write PDF file using PDF writer, when I add @Value(Scan_Location) in fanout dataset it creates 2 different files as output. And that is not what i want . I just want this name ….20240621 NI2-HK-L03 As-built_PDMS Comparison Report F1.pdf and just one file
I am trying some network analisys (isochrones and route/gpx creation) and I am obviously doing something wrong, I just can't figure out what.I am using OSM highways, unfiltered (I also tried a local route network, same result). The only manipulation I did is convert it to espg 28992RDnew (because it's in meters) before using it as an input for the topologybuilder (also tried topology calculator). Since I have multiple start points I created a road network Per stop_id and used ‘group by’ on the topologybuilder and networkcostcalculator. The results are very weird. Connected and (way too many) not connected seem random: (green = connected, red = not connected) Zoomed in:I tried pretty much all settings on the networkcostacalculator. When I try the shortestpathfinder it's also clear that something is off, it refuses to use some (random) paths/roads/lines. So it's obvious I am doing something wrong, I just cannot figure out what.
Does anyone know if there would be a performance difference between having linked transformers stored on a shared external drive compared to a local drive?I do not have much experience with linked custom transformers yet, but my current understanding is that there may be slightly longer load times when opening the workspace using the linked transformer, however no performance decrease in translation. Would this be correct?
I tried the FileExistenceChecker to check fo a online xml-document but it seems that it can not check url adresses (result -> DoesNotExist). It didn't recognize an url-Adress that the xml-Reader is able to read (See attached Test Workspace). Any Idea what the problem could be?
When reading FCL from ArcGIS sde database, also the information about the FDS in which the FCL resides is available in FME. There is no problem in writing it to another database in a FDS, even if this FDS does not exist in the target. If one would like to create the new FCL not in a FDS (in my case SQL server database) straight from the input, you will get an error creating a FDS.Geodatabase Writer: Creating feature class `ARCFMUT.FCL_BW_LOGGER' in feature dataset 'BW_intern.ARCFMUT.FDS_BW_DIVERSEN'An error occurred while attempting to create the feature class 'ARCFMUT.FCL_BW_LOGGER'. The error number from ArcObjects is: '-2147220969'. The error message from ArcObjects is: {Must be the owner to perform this operation.}After some time of investigating this issue, I concluded that the writer tries to create a FDS using the credentials of the reader, which of course did not exist in the writer, even if I explicit had left the field for a FDS to blank.My solutio
I have two feature classes (FC) that are linked by a 1:1 relationship. Both FCs contain about 4,000,000 objects, of which I ultimately need about 20,000 objects. I understand that I could join all 4,000,000 objects, but that would take a considerable amount of time, which I want to save.My idea is to select the required objects in FC A and create a new list using the join attribute. Later, I want to use the FeatureReader and a WHERE clause query to read and process the required objects in FC B. Unfortunately, I am having a problem with the creation of the WHERE clause, so no objects are being read.My workflow is as follows:Read and select objects from FC A Prepare the join attribute for the WHERE clause (add quotes) Create the list using the join attribute Convert the list to a string with commas as separators Read the objects using FeatureReaderEverything seems to work except for the last step. If I enter the WHERE clause manually (without the list), it works. Below is my current work
In an sql_executer I run an sql-statement that updates a table, but does not output any features. If there is an error in this update, an error message appears in the log file, but the translation as a whole is successful. How can I intercept this error so that the translation aborts if there is an error?
Hello,I have a general question regarding the RCaller. I have been actively coding in R over past 10 years, and started playing with integrating my code into FME for automating some of the analyses about a year ago, but what bothers me most is that FME is always putting up a fight whenever facing more complex custom R functions, i.e. loops, advanced raster processing involving raster lists/stacks, LiDAR, etc. It is always a problem when trying to get FME recognize the results as fmeOutput, and it is just never easy compared to how other things work in FME...It just seems like the RCaller is so basic in FME, rudimental even, not allowing no unleash the full R potential within the FME environment. Hence is my question to you, have you guys considered developing it further or is it because nobody is really interested in using R inside FME? Or is it just my subjective opinion and I am basically missing something? :)@dmitribaghThanks in advance.-Sveta
Is there a way to specify removal of “white collar” areas in Adobe Geospatial PDF outputs? I don’t see any options in the Page Size parameters to restrict the output to be just the extents of the Reader-specified image source.
Just curious. Currently it takes about five clicks using the AttributeManager to get to AI Assist. Any shorter way?
Hello,I created bounding box in FME and I need to get from this polygon WKT file. How can I reach this in FME please?Thank You so much!
Hello,I’m trying to read specific feature classes from an SDE database using a user parameter within a feature reader.I’ve not had a problem with this previously so not sure if it’s a bug (I’m using FME 2023.1.2.0 (20231110 - Build 23641 - WIN64) The issue I’m having is:When selecting feature classes manually in the ‘Feature types to read’ section of the Feature Reader window I can select as many as I want and they’ll all read fine.When using the user parameter (please see settings in screenshot below):Selecting one dataset reads fineSelecting 2 or more feature classes I get the error “Error creating feature dataset 'feature class a feature class b'. The table name or field names may be too long or invalid for the server” Thanks for you time
Hi.I’m using the snipper transformer to extract portions of a line at specific values from the attribute table.I set these parameters for the transformer. I set these parameters for the transformer.The problem that I noticed is that the program considers the first vertex of the line as 0 and the last one as the maximum, and the snipper will split between these two values.Is there a method to define the start value using another value, not 0? I would like to start from 4.21 km.Thanks.