Shape the future of FME with your ideas
New and Open ideas are awaiting review or have been reviewed by our Product Management team, and are open for commenting and voting.
I have recently being creating some custom HTML reports, combining the output of an HTMLReportGenerator with some additional scripts to add custom graphics. Having zero HTML experience, I created these scripts in ChatGPT and then parameterized them for use in my template. This template is then streamed into a web browser as the output of a Flow web app. It strikes me that HTML functionality could be a incredibly useful extension to the current capabilities of AI Assist.
As an FME Flow Admin I would like to have the ability to set an expiration date for FME Flow profiles. This would likely be an optional parameter that could be set on a user or role level. Some of our project managers have requested this, particularly when we have external users(sub-contractors) accessing our infrastructure. This would help tremendously to manage access automatically as opposed to manually having to remove access or profiles after a projects period of performance.This is just a conversation starter. Removing a user from a role would remove access from an app--for example--but it does not prevent access to FME Flow depending on the users profile settings. Thanks for all you do
I have a large amount (3000+) of structured e57 files that I would like to convert to Leica/Hexagons proprietary format which is LGS/LGSx. e57's, while great due to being non-propritary, are large. Leica's LGS and even great LGSx files can store the same structured point cloud data but into half or a third the space. My 20TB of point cloud data could be as low as 7-10TBs, this is a great savings in storage costs. The only workflow I know is to import e57s into Leica's Cyclone or Cyclone Register 360 platforms and export LGS/LGSx. Doing this 3000+ times might kill me lol. Being able to set this up on my FME Server means I could do it hands off 24/7. Please Safe/FME I know I would not be the only one to benefit from such a transformer.
On the inspector it would be good to be to have a split view of the table view of the data, so you could have a split view of two inspectors at the same time, and can compare columns. This is how I work more as a web developer. I find I want to compare inspectors side by side to check columns in one or the other. Obviously they are tabbed so you have to flick between them often, but I would like an option to have split views. (At least I think it would be good!)
Short lived tokens to SASS providers like ESRI AGOL require reauthorization after establishing the initial connection. I would like a way to create a workspace that could update the tokens on a set schedule instead of doing a manual process.
It would be great if we had the option to embed a warning in the default output value of transformers like the Attribute Value Mapper. It would work in a similar war to the “@Abort()” function, but instead of stopping the translation, it would return a custom warning in the log. This can be really useful in tools that are designed to be run by people who don’t know much FME and/or tools that are designed to run automatically.As a side note, yes, I understand that you can just write a custom default value there and use it to identify the bad values downstream. This idea would essentially be an alternate way of doing that, and would be simpler/more direct.
There are a number of operations that are made much easier and/or more efficient by simply having data be passed around a file on disk instead of as FME features. When dealing with large amounts of features going into a PythonCaller, for instance, it can be preferable to dump them all into a Parquet or Arrow file and decode that file with Pandas or some other dataframe library, rather than extracting attribute data feature by feature in Python. Some other use-cases, like Emailer, also often make use of temporary files.The issue is that having FeatureWriter recieve both a path from TempPathnameCreator and the features it needs to actually write doesn’t work, that path needs to be joined on each feature in any of various clumsy methods. Given most uses of TempPathnameCreator are intended to feed into FeatureWriter, I would suggest that “Temporary local file” be a valid dataset option for FeatureWriter, which would simply generate a temporary path in FME_TEMP, write all features to it and allow the file name to be retrieved as the _dataset attribute under the Summary tag, as is already the case. It would otherwise work in exactly the same way as the combination of TempPathnameCreator and FeatureWriter, but simpler to setup.Any scenario where it is more practical to pass features between transformers as single files rather than large collections of features might benefit from this.
The suggestion I have is to allow for the GoogleDriveConnector Delete action to work with “Move to Trash” HTTP call request to Google API V2, instead of using the Delete HTTP call. The reason for this is due to the difference between the “Content Manager” and “Manager” access levels. A “Content Manager” is allowed to move files to the trash, but not allowed to “permanently delete” files.Only “Manager” access level is allow to “permanently delete” files.As it stands right now the GoogleDriveConnector is sending a request to “permanently delete” files, which in Google Shared Drives would only work for users with “Manager” access level and not the “Content Manager” access level that is generally assigned to team members in an organization.
It would be great to have an Tolerence-Parameter and Automatic Mode for the transformers Matcher and ChangeDetector, similiar to the LineOnLineOverlayer for example.Espacially when you work with geographic coordinate systems (latlong) and data from different systems/formats is it hard to find a proper value for the vector tolerance. An Automatic Mode would help a lot.bestThomas
The Active periods in the FME Flow Engine Management currently (FME 2024.1) allows for a daily and weekly recurrence. It would be useful if there would be more fine-grained control so we could also specify, for example, day of week (so we can set an active period during regular weekday office hours), monhtly or day-of-month
This was posed as a question a few years ago so I thought I’d turn it into an idea as it doesn’t seem to exist as yet. The single merged feature type is a powerful feature of reader feature types and the behaviour should be replicated easily in the Feature Reader - an example being where one wants to scan all feature classes in a file geodatabase and read only the features that intersect the initiator’s geometry.This means not specifying particular feature types to read, as the existing reader functionality goes. The Feature Reader has the option of a single output port but the existing reader behaviour takes care both of the feature types to read and the output port and that’s what I’m talking about here.If I’ve missed an idea then feel free to close this and to redirect.
I find the SchemaScanner essential in discovering schemas of source data and feeding dynamic writers. (FME 2024.1,1 is the version I’m using)I also find the SchemaScanner very useful for merging schemas by ignoring attribute names or grouping by attribute values.This leads me to want more Schema Analysis functionality.I think it would be very helpful to have the ability to extract a common schema from a set of features.for example if all features have the same set of attributes:featureName, featureDesc, featureID. It would be great to extract the common schema to support writing to what would look like a base class (table) and then have the ability to write the full schema and features in another table to extend the base class. Or have an option to extract the common schema and at the same time have the option to pack the remaining attributes of the passed through features into a json array (like the AttributeJsonPacker) thus merging the features into a single common schema.
I think it would be nice to have a checkbox “Retry on canceled” in the “run workspace” element in an automation that would manage resubmitting the job when being canceled (for instance, when it exceeds the max. queue time).
I would like to see FeatureWriter or another transformer updated to allow data being written to a temporary table and an SQL statement (or several) to be run afterwards to manipulate that data, most likely to move it to its permanent location and/or create partitions for the range to be inserted.Related to a community question I opened last year, and to a similar idea I submitted for temporary files.One method I commonly use (at least in Postgres) to efficiently write large amounts of data to large tables is to create a temporary table that’s identical to my target table (minus indexes and constrainsts), use some bulk loading method like COPY to fill that table with the data I’m inserting, and then running whatever statement I need to insert/upsert the final table with that data. This allows the DBMS to use whatever method works best for filling that data and does the actual data insertion as a single transaction. For FME specifically, this allows rows to be upserted without opting out of bulk-mode insertion.As you can see from my community question, this turns out to be very complicated with FME because transactions/sessions aren’t scoped across transformer nodes in a way that makes it possible to use temporary tables, so you need to:Create a schema for fake temporary tables Figure out some way to make a table name that is unique per run, which works reliably for both FME Form and FME Flow and fits within the table name length limits CREATE UNLOGGED TABLE “fme_fake_temp”.temp_table_name LIKE source_table_name EXCLUDING CONSTRAINTS EXCLUDING INDEXES In a subsequent SQLExecutor, run your insert statement and drop the tableDropping the table is often unreliable, insertion errors will often cause the temporary table to remain there and slowly inflate the fake temporary schema, requiring preriodical cleanup. Creating a unique table name is also inconveinient; $(FME_UUID) (which is what TempPathnameCreator uses) would work reasonnably well, since it’s an underscore-separated UUID prefixed with the transformer name, but it doesn’t work with FeatureWriter and requires transformer names to be valid table identifiers and shorter than 25 characters long to fit under Postgres’ 63 character limit. Besides that, FME offers no identifier that is unique per run and available on both Form and Flow.This is a lot of gymnastics for what I believe is a fairly common data insertion pattern, and FME would benefit from supporting it “natively”, so to speak, possibly using actual temporary tables. Other DBMS support similar private/temporary tables, like Oracle’s Private Temporary Tables, MySQL’s temporary tables, MS-SQL’s #TemporaryTables, or the (somewhat more convoluted) SQLite approach of mounting a separate database (possibly one that only exists in :memory:) as a schema, and creating temporary tables there. For FeatureWriter proper, however, these mechanisms wouldn’t work, so having FME track temporary “real” tables and drop them at the end of the flow run would likely be the least disruptive option. Using the actual mechanisms for session-local tables would probably require something more akin to InlineQuerier.
Right now only file.path, file.rootName and file.event are visible: It feels like it shouldn’t be difficult to expose a new output attribute for the file.type, to be able to tell whether the updated item is a folder or a file, or the file.extension, rather than hoping for the best that nobody included the sequence “.shp” in the name of their excel files.Yes, you could easily do this within the workspace, but it doesn’t feel right.
It would be beneficial to have a transformer that displays all attribute values simultaneously, rather than using the TestFilter Transformer. In some cases, I need to process each value, which can be time-consuming when using TestFilter to have each attribute value on its own tab.
For comments and workspace description etc a spell check would be a nice to have. My splling is terribkle. Currently I have to copy all the text into another tool.
I often use the DuplicateFilter to get a quick list of unique attribute values for a specific attribute. For this use case all other attributes that are still present after the DuplicateFilter are not needed and to see the desired result, you either have to remove the superfluous attributes with another transformer like the AttributeKeeper or hide them in the visual preview. That’s why I suggest to add the option to remove unselected attributes to the DuplicateFilter.Remark: I’m aware, that you can already get a similar result with the Aggregator, but if a feature geometry is present all input geometries would have to be processed allthough they are not needed for the above use case.
As FME users deal with increasingly complex data transformations, the ability to efficiently compare workspaces becomes essential. Currently, the "Compare Workspaces" feature is confined to a single window, which can be limiting for those using multiple monitors. Current State: FME allows side-by-side comparisons within a single application window. Proposed Feature: Independent windows that can be placed on different monitors,I propose a Dual-Window Workspace Comparison feature that would allow users to open and view two separate FME windows on different screens. Key Benefits:Boosts productivity by leveraging multiple monitors. Simplifies workspace comparison and troubleshooting. Enhances user experience in managing complex data transformations.
BACnet (Building automation and control network), is a protocol that building utility sensors use. If FME was able to read the sensors directly the information that could be read in would be extremely valuable for aggregating data on building performance/status. I see this working fantastic with FME Data Streaming and pushing this data to ArcGIS services, so we can visual in dashboards which rooms are currently being cooled/the temperature/which rooms are occupied/etc, and so many other things.
Version control is only as good as the business processes surrounding it - being able to commit to a git repo/FME version control is great, but it’s easy to miss in a publishing workflow and is not currently possible to enforce.In order to support more robust SDLC & change control processes, it would be extremely useful to be able to either automate a git commit when publishing/republishing a workspace, or to configure a workspace/repo/flow env to make commits compulsory.My blue-sly vision for this would be that the configuration is set at the repository level on FME Flow, and in repos where compulsory commit is enabled, the publish action in FME Form cannot be used until a commit has been performed with the current version of the workspace.
Would just make workspaces a little cleaner.
When using the Multipart / Form Data option in a HTTPCaller, the MIME Type of the upload is quite hidden. It took me a while to figure out the option even existed. Right now you have to click on the ...-button to change the type while it would be a lot more clear to add an extra column to the Multipart Upload option table which is already there.
I like that in FME Flow it's possible to configure cleanup tasks.However, to me it wasn't so intuitive how the cleanup frequency actually works.For example, if I configure a cleanup task to cleanup files that are older than 1 hour, I was tempted to think that files that would be in the folder would be cleaned up after 1 hour (and surely after multiple hours).However, I learned that that isn't actually the case. I learned that there is actually a relation/dependency between;a) The 'Remove Files Older Than' parameter (at tab 'Tasks' of 'System Cleanup'), andb) The 'Task Interval' parameter for 'Normal Conditions' (that can be configured at the 'Configuration' tab of 'System Cleanup')Specifically, how often the cleanup tasks take place, are configured by the 'Task Interval’ parameter, which I think by default is 1 day.So, for my newly configured cleanup task, I would need to wait at least a day for files to be removed, and not 1 hour.I think it may avoid some confusion, when a warning would show up, in case a user would configure a cleanup task to be scheduled more frequent than the general 'task interval’ in which the cleanup tasks are performed. Ps. This is an idea following ‘item 2’ of my question on the FME Community: How to periodically Cleanup the FME Flow Temp Resource (i.e. '$(FME_SHAREDRESOURCE_TEMP)') | Community (safe.com)
TL;DR: Allow me to set a list of pfererred file types to read in a Generic Reader and FeatureReader.Background: When feeding files into a Generic Reader or FeatureReader, it appears that FME (Form/Flow) selects one format automatically from its available formats in alphabetical order, for example:.gml: AIXML5 .gpkg: GEOPACKAGE_RASTER .json: CITYJSONI’m working with geospatial vector data, so instead I’d prefer:.gml: GML .gpkg: OGCGEOPACKAGE .json: GEOJSONThere’s currently no way set a preference, an allow- or blocklist to set which formats to use or not.Some workarounds I found on these forums:Rename files (not viable for user-provided data) Build my own mapping with a FileNameExtractor to get the extension → AttributeValueMapper and feed it into a FeatureReader (may be error-prone) Create a custom format that starts with AAAAA… to place it at the start of the alphabetical listRelated forum posts and somewhat similar ideas:
In-depth courses for all FME skill levels.
Explore our extensive collection of how-to guides and tutorials.
Log an issue with us. Our support team is ready to help.
