Shape the future of FME with your ideas
New and Open ideas are awaiting review or have been reviewed by our Product Management team, and are open for commenting and voting.
I was just playing with SQL Notebooks in Azure Data Studio, and I found this quite a fun and interesting concept. Then there are of course Jupyter/Python notebooks as well. (In case you are not familiar with the concept - you basically write a documentation document with notes and descriptions and inside it you embed data cells that perform operations. You can either run all cells, or run them one by one.) My next thought was - what if one could have an FME notebook? I think it would work very well for not-too-complicated one-off workspaces, or for tutorials. The documentation text is central and data cells one could perhaps be like bookmarks in a workspace. I am not sure how it would work and/or how one would implement this... But imagine uploading an FME notebook to a Flow instance to create a server app centered on the documentation and step by step execution of transformers/bookmarks/operations! Migration Note: Idea originally posted on 2024-02-01
When I got a humongous pile of parquet-files I had to find a way to read and filter the data in them. I found this and it works great, https://duckdb.org/docs/guides/import/parquet_import.html.Fast SQL-queries on all files without having to move the data into something else first!Using DBeaver and a DuckDB connection ":memory:" I can do queries like this:select count(distinct Id) FROM "D:\myfiles\**\*.parquet"where startingdate >= '2020-07-27 00:00:00' Perhaps this will be a bonus with the upcoming DuckDB-reader/writer...??? :) Migration Note: Idea originally posted on 2024-02-01
World files like tfws produced by FME are rounded to 10 significant digits when written by the various raster writers even if the original georeferencing is more precise. This is an issue when working with images in Lat-Long when attempting to work with both the original image and the created image because transformers like the RasterBandCombiner or RasterCalculator fail with an error that the x and y spacing doesn't match.Simply writing geotiffs won't work because the tif image is processed in non-geospatial aware software and requires the tfw to preserve the georeferencing information. A common workflow is RasterReader->FMEprocessing->FeatureWriter(tif+tfw)->external processing->FeatureReader-> Further FME processing It would be nice if we could specify the significant digits and/or whether to round them at all on either the writer itself or as a system setting. Migration Note: Idea originally posted on 2024-02-02
All readers today is for only one extension in a time. Sometime you have to upload multiple files in a flow. It would be nice to be able to add a zip with multiple files. And then the path for each files to use in a reader at your choice. This pythonscript read all files in a zip and returns the path. You need a attribute exposer to get the attributes. import zipfileimport osimport fmeobjectsclass FeatureProcessor(object): def __init__(self): pass def input(self, feature): zip_path = r'yourzipfile.zip' extract_to = os.path.dirname(zip_path) with zipfile.ZipFile(zip_path, 'r') as zip_ref: zip_ref.extractall(extract_to) for file in zip_ref.namelist(): f = fmeobjects.FMEFeature() f.setAttribute('file_name', file) f.setAttribute('file_path', os.path.join(extract_to, file)) self.pyoutput(f) def close(self): pass
Supporting CityJSON 2.0 would be very nice. It is currently supporting a version from 2019. Is it in the works to support 2.0 (aligning with citygml 3.0)? I can't find any topics on this and thus want to ask if/when we can expect it.
There is a custom transformer named RandomPointGenerator which can generate random points in polygons. However, it can only use the same single integer value for all polygons. If the possibility would exist to create N number of random points, where N is an attribute value different for each polygon, this transformer would be much improved.
IdeaWhen using Python, return FME_MacroValues["ParameterName"] as an object instead of a string, so we can access both the value and the definition of the Published/Private Parameter.Why would this be helpful?It allows us to see if the parameter was Private, Public and/or Optional.We can see what the Default Value is. It can be handy to have this value in case the Python code needs to resort to another solution if the value that the user entered produces the wrong result.We can get a nice descriptive name of the parameter (from the Prompt field), so we can create nice log messages are another kind of user feedback to tell that a parameter value is wrong for instance.We could read the Configuration for a Choice parameter for instance, in the same order in which the choices are specified in the Definition dialog. Then, we could check if the parameter value is equal to a configuration list item by index, so that we no longer need to hardcode all the choice values in the PythonCaller if we want to run a different function for each choice for instance. This reduces the risk of breaking the code if we change one of the choice values in the Parameter Definition Dialog. It would only still break (or give unexpected results) if we would change the order of the choices, but that I can live with.If we have an object anyway, there might also be an additional property inside that tells us what native Python type the parameter value can be converted to. Or better yet, simply return the parameter value as a Python type...
This would for instance be usefull when dealing with XML, JSON or rasterdata from API's. Kind of like how the RasterReplacer transformer offers this functionality for rasterdata. Of course there is the option of writing to a temporary file (TempPathnameCreator - AttributeFileWriter) and then reading it in from the (temporary) filepath, but I think a direct implementation in the FeatureReader would be nicer (and maybe also a bit more efficient, since you already have the data as a blob).
***Note from Migration:*** Original Title was: Warn when resolving ArcGIS Domains if code (value) not found in Domain (FGDB/ArcSDE) For example when reading a File Geodatabase or ArcSDE layer.It is possible for a field in a layer to contain a value that doesn't match a code (value in FME speak) in the Domain. For example your Domain has:1 = Car2 = Bus3 = TrainBut the Vehicle field for some features in your layer have the value "4" in the actual data. Even worse, the value in the data might be "Car" which makes it almost impossible to figure out that ArcGIS is showing the raw data and not displaying "Car" by looking up "1" in the Domain.When you read the data into FME (whether you use the FGDB Open API Reader or the ArcGIS Reader), you will get an empty string for any values such as "4" that aren't in the Domain. This is because FME will look up the actual value from the data and not find it in the Domain.I would like to suggest that if this happens, the Log file should record a Warning or Error to say something like "WARN: code '4' not found in Domain 'Vehicle_type'".
The new Era, the new FME Form, it would be nice if the maximum number of concurrent translation processes would be increased, or unlimited ideally.
Setting the number of standard engines requested is not intuitive. There is a sub-job running on engine A:3. In this case, the standard setting is actually 3 on host A and 2 on host B. This is the edit page of requested engines - why is 4 shown on host A and what is the effect if i change it to 3 in this case?(FME Flow 2023.2.1)
To date, when I have maintained hosted feature layers in ArcGIS Online or Enterprise, I have frequently used a change detection approach and written the delta transaction. This can result in some minutes during which the service is in flux,especially if the service is large and takes time to read into however many ChangeDetector transformers are needed. It is also possible some writes fail.In a webinar I watched today, the presenter demonstrated a workflow that wraps functionality available to swap the layer source for a hosted feature layer view. This has a managerial advantage in that a candidate replacement set of data can be curated and tested alongside the production data, and a performance advantage in that swapping a layer service view’s source is faster than any other transaction approach.As the ArcGISOnlineConnector uses the administrative ArcGIS REST API under the covers I propose a new Action for the transformer: “View Source Swap” that lets you pick the target view and replacement service, checks the requirements are met then does the swap.
Hi, in FME Flow 2023.2.1 you can´t disable an Active Period once it was successfully created. If you like to disable an Active period but keep the configuration (like a repeat on interval config), you need to remove it or adjust the start/end date.It would be more helpful to enable/disable an Active Period like it is done in the Job Routing / Engine Assignment rule / Schedule / Automation.
I love inspecting my data using the inspector. It is very powerful, but it would be much more useful if there was a data editing tool included with it. I am sure that this would be very helpful for many people.
With REST web services, generally they return JSON/XML, which I then need to manipulate into attributes.The pattern of work is often: Add Creator, HTTPCaller, Logger/InspectorRun workspaceInspect returned JSON/XML - determine if need to fragment and flatten or only flatten.Add JSONFragmenter or JSONFlattener or XML equivalent and Logger/Inspector.Run workspace.Inspect attributes created from flattening.Expose attribute names that were created by flattening.I wish this was less work.
Please evaluate to add a variable in fmeserver that contains the useremail from the logged user if present in the useraccount table. Is useful to send mail directly to the user. Thank you.
If I ask OpenAI to give me GeoJSON defining a box around a city or a point location I get a pretty good result. It saves looking for a dataset etc. Let's add AI Assist to the transformer.
Be able to choose Add as FeatureReader instead of the Regular reader when dragging dropping files into FME Workbench:
I would be keen for a "Annotation Default" in workbench options that would control whether annotation was on/off by default when opening a workbench (defaulted to on of course). I would just like to open workbenches and see a clean bookmarked interface, which I could quickly turn on annotation if I needed too.
Need option to export values from AttributeValueMapper
At this moment the option to redirect writers to inspector is adjustable for every workspace. When working in a large project involving a lot of WorkspaceRunners calling other workspaces, I tend to forget to disable this option in some of the workspaces. This results in having to run the complete process (or at least a big part of it) again when I notice my mistake the next morning.Therefore it would be very useful if there was a way of overruling this per-workspace setting with a global setting.
Like the SchemaMapper which takes mapping rules from external file, the AttributeValidator could read validation rules from a table.This would also allow validation rules maintenance by non FME people.
It would be a real addition if two navigation buttons were added to the DataInspector, being 'GoTo previous extent' and 'GoTo next extent'.Often when I navigate through my geometry in the DataInspector I want to go previous extents. Sometimes because I accidentilly moved my map, other times because I want to compare different features that are not adjecent. I think it is really helpfull when these to buttons are added.
It would be great if it would be possible to give views in the data inspector a name. So I could rename view 1 to "other name".The reason for this is when I am testing I have often have multiple views open. Some of these views I want to keep open the whole time I am working on a workbench. When I could rename the view it gives a better overview.
There are a lot of users that only run FME workspaces. For them it would be nice if you can choose to only install the FME Quick Translator. My suggestion is to add an option to choose which FME components you want to install. In practice this would mean a full install or a light install, leaving out the FME Workbench.I know it is possible to adjust things afterwards, like linking the *.fmw extension to the FME Quick Translator and removing the FME Workbench from the Start-menu. But that takes time and needs to be done after every install and update.
In-depth courses for all FME skill levels.
Explore our extensive collection of how-to guides and tutorials.
Log an issue with us. Our support team is ready to help.
