Shape the future of FME with your ideas
Open ideas have been reviewed by our Product Management and are open for commenting and voting.
It would be great if we had the option to embed a warning in the default output value of transformers like the Attribute Value Mapper. It would work in a similar war to the “@Abort()” function, but instead of stopping the translation, it would return a custom warning in the log. This can be really useful in tools that are designed to be run by people who don’t know much FME and/or tools that are designed to run automatically.As a side note, yes, I understand that you can just write a custom default value there and use it to identify the bad values downstream. This idea would essentially be an alternate way of doing that, and would be simpler/more direct.
There are a number of operations that are made much easier and/or more efficient by simply having data be passed around a file on disk instead of as FME features. When dealing with large amounts of features going into a PythonCaller, for instance, it can be preferable to dump them all into a Parquet or Arrow file and decode that file with Pandas or some other dataframe library, rather than extracting attribute data feature by feature in Python. Some other use-cases, like Emailer, also often make use of temporary files.The issue is that having FeatureWriter recieve both a path from TempPathnameCreator and the features it needs to actually write doesn’t work, that path needs to be joined on each feature in any of various clumsy methods. Given most uses of TempPathnameCreator are intended to feed into FeatureWriter, I would suggest that “Temporary local file” be a valid dataset option for FeatureWriter, which would simply generate a temporary path in FME_TEMP, write all features to it and allow the file name to be retrieved as the _dataset attribute under the Summary tag, as is already the case. It would otherwise work in exactly the same way as the combination of TempPathnameCreator and FeatureWriter, but simpler to setup.Any scenario where it is more practical to pass features between transformers as single files rather than large collections of features might benefit from this.
The suggestion I have is to allow for the GoogleDriveConnector Delete action to work with “Move to Trash” HTTP call request to Google API V2, instead of using the Delete HTTP call. The reason for this is due to the difference between the “Content Manager” and “Manager” access levels. A “Content Manager” is allowed to move files to the trash, but not allowed to “permanently delete” files.Only “Manager” access level is allow to “permanently delete” files.As it stands right now the GoogleDriveConnector is sending a request to “permanently delete” files, which in Google Shared Drives would only work for users with “Manager” access level and not the “Content Manager” access level that is generally assigned to team members in an organization.
It would be great to have an Tolerence-Parameter and Automatic Mode for the transformers Matcher and ChangeDetector, similiar to the LineOnLineOverlayer for example.Espacially when you work with geographic coordinate systems (latlong) and data from different systems/formats is it hard to find a proper value for the vector tolerance. An Automatic Mode would help a lot.bestThomas
The Active periods in the FME Flow Engine Management currently (FME 2024.1) allows for a daily and weekly recurrence. It would be useful if there would be more fine-grained control so we could also specify, for example, day of week (so we can set an active period during regular weekday office hours), monhtly or day-of-month
This was posed as a question a few years ago so I thought I’d turn it into an idea as it doesn’t seem to exist as yet. The single merged feature type is a powerful feature of reader feature types and the behaviour should be replicated easily in the Feature Reader - an example being where one wants to scan all feature classes in a file geodatabase and read only the features that intersect the initiator’s geometry.This means not specifying particular feature types to read, as the existing reader functionality goes. The Feature Reader has the option of a single output port but the existing reader behaviour takes care both of the feature types to read and the output port and that’s what I’m talking about here.If I’ve missed an idea then feel free to close this and to redirect.
I find the SchemaScanner essential in discovering schemas of source data and feeding dynamic writers. (FME 2024.1,1 is the version I’m using)I also find the SchemaScanner very useful for merging schemas by ignoring attribute names or grouping by attribute values.This leads me to want more Schema Analysis functionality.I think it would be very helpful to have the ability to extract a common schema from a set of features.for example if all features have the same set of attributes:featureName, featureDesc, featureID. It would be great to extract the common schema to support writing to what would look like a base class (table) and then have the ability to write the full schema and features in another table to extend the base class. Or have an option to extract the common schema and at the same time have the option to pack the remaining attributes of the passed through features into a json array (like the AttributeJsonPacker) thus merging the features into a single common schema.
Hi FME community,From FME 2021.1 onwards the Autodesk ReCaP writer will be deprecated (https://community.safe.com/s/article/FME-Deprecations). Please vote when you currently use the Autodesk ReCaP writer and don't want it to deprecate. BackgroundEngineers of our infrastructural projects use very often the Autodesk ReCap data format for viewing their point clouds in applications like Revit, AutoCAD, Autodesk ReCaP and Autodesk 3ds max. With this efficient format they can compare their designs with the current - often complex - situation. I would suspect that the whole engineering industry benefits from the efficient way of FME indexing raw point cloud data that's ready to use in BIM programs like Revit. Several alternatives we’ve considered for our point cloud workflow:Importing txt, xyz, pts files in Revit, however the indexing happens on PC’s of the engineers where they’re not equipped for.Importing LAZ files into Autodesk ReCap and save the RCP/RCS files to view them in Revit, however this is an error prone manual process that we want to automate. Different versions of FME paralleled to be still capable of using the ReCap writer, however this is not possible due to our use compability issues with FME Server.Viewing the point clouds in Point Cloud Scene Layers, but comparison with the designs in BIM applications is more difficult.If you use other (maybe even better) formats, please let me know. Else, vote for getting the format high on the backlog of FME's Product Managers.Illustration in ReCap of Rijksmuseum, The Netherlands (source: open elevation dataset AHN3)Kind regards,Jan Droesen FME specialist Movares Netherlands
I think it would be nice to have a checkbox “Retry on canceled” in the “run workspace” element in an automation that would manage resubmitting the job when being canceled (for instance, when it exceeds the max. queue time).
I would like to see FeatureWriter or another transformer updated to allow data being written to a temporary table and an SQL statement (or several) to be run afterwards to manipulate that data, most likely to move it to its permanent location and/or create partitions for the range to be inserted.Related to a community question I opened last year, and to a similar idea I submitted for temporary files.One method I commonly use (at least in Postgres) to efficiently write large amounts of data to large tables is to create a temporary table that’s identical to my target table (minus indexes and constrainsts), use some bulk loading method like COPY to fill that table with the data I’m inserting, and then running whatever statement I need to insert/upsert the final table with that data. This allows the DBMS to use whatever method works best for filling that data and does the actual data insertion as a single transaction. For FME specifically, this allows rows to be upserted without opting out of bulk-mode insertion.As you can see from my community question, this turns out to be very complicated with FME because transactions/sessions aren’t scoped across transformer nodes in a way that makes it possible to use temporary tables, so you need to:Create a schema for fake temporary tables Figure out some way to make a table name that is unique per run, which works reliably for both FME Form and FME Flow and fits within the table name length limits CREATE UNLOGGED TABLE “fme_fake_temp”.temp_table_name LIKE source_table_name EXCLUDING CONSTRAINTS EXCLUDING INDEXES In a subsequent SQLExecutor, run your insert statement and drop the tableDropping the table is often unreliable, insertion errors will often cause the temporary table to remain there and slowly inflate the fake temporary schema, requiring preriodical cleanup. Creating a unique table name is also inconveinient; $(FME_UUID) (which is what TempPathnameCreator uses) would work reasonnably well, since it’s an underscore-separated UUID prefixed with the transformer name, but it doesn’t work with FeatureWriter and requires transformer names to be valid table identifiers and shorter than 25 characters long to fit under Postgres’ 63 character limit. Besides that, FME offers no identifier that is unique per run and available on both Form and Flow.This is a lot of gymnastics for what I believe is a fairly common data insertion pattern, and FME would benefit from supporting it “natively”, so to speak, possibly using actual temporary tables. Other DBMS support similar private/temporary tables, like Oracle’s Private Temporary Tables, MySQL’s temporary tables, MS-SQL’s #TemporaryTables, or the (somewhat more convoluted) SQLite approach of mounting a separate database (possibly one that only exists in :memory:) as a schema, and creating temporary tables there. For FeatureWriter proper, however, these mechanisms wouldn’t work, so having FME track temporary “real” tables and drop them at the end of the flow run would likely be the least disruptive option. Using the actual mechanisms for session-local tables would probably require something more akin to InlineQuerier.
Migration Note: Idea originally posted on 2015-10-29
Hello all,Working for Dutch local governmental organisations, I more and more encounter webservices which require a certificate to be send as part of the authentication / authorization process. My guts feeling is, that this is especially the case with API's providing 'personal information', and that it is a hobby of Java developers...Searching the Safe forums, I found this post most prominent, which suggests to use the python requests library (also the most FME compatible solution The Internet provides).I can't see how to implement this in a connection-services-template...So, please Safe Software, make this possible...Martin
Right now only file.path, file.rootName and file.event are visible: It feels like it shouldn’t be difficult to expose a new output attribute for the file.type, to be able to tell whether the updated item is a folder or a file, or the file.extension, rather than hoping for the best that nobody included the sequence “.shp” in the name of their excel files.Yes, you could easily do this within the workspace, but it doesn’t feel right.
It would be beneficial to have a transformer that displays all attribute values simultaneously, rather than using the TestFilter Transformer. In some cases, I need to process each value, which can be time-consuming when using TestFilter to have each attribute value on its own tab.
It would be very helpful to have a Rejected-port on the PythonCaller.Outputting a failed feature could e.g. look like this:self.pyoutput_failed(feature)This would simplify the pattern where you need to set a temporary attribute to indicate success/failure, then a Tester and then removing the temporary attribute.
Instead of creating features with no attributes, enhance the creator to optionally allow a table of attributes and values to be used to populate a series of features, one per row of the table.
Hi, renaming an attribute in large workspaces oftentimes takes a lot of time because of the related manual adjustments needed. I would appreciate a function that is able to change a given attribute to another given attribute within a whole workspace in every Transformer and Writer.Best regardsJudith
For comments and workspace description etc a spell check would be a nice to have. My splling is terribkle. Currently I have to copy all the text into another tool.
Basically add a CoordinateSystemSetter inside the VertexCreator to avoid having to use a CoordinateSystemSetter up front. Just 1 extra 'optional' parameter required.
When you have created geometry you are usually done with attributes supplying the coordinates, GeometryReplacer has an option to drop them, let's do the same for VertexCreator.
I often use the DuplicateFilter to get a quick list of unique attribute values for a specific attribute. For this use case all other attributes that are still present after the DuplicateFilter are not needed and to see the desired result, you either have to remove the superfluous attributes with another transformer like the AttributeKeeper or hide them in the visual preview. That’s why I suggest to add the option to remove unselected attributes to the DuplicateFilter.Remark: I’m aware, that you can already get a similar result with the Aggregator, but if a feature geometry is present all input geometries would have to be processed allthough they are not needed for the above use case.
As FME users deal with increasingly complex data transformations, the ability to efficiently compare workspaces becomes essential. Currently, the "Compare Workspaces" feature is confined to a single window, which can be limiting for those using multiple monitors. Current State: FME allows side-by-side comparisons within a single application window. Proposed Feature: Independent windows that can be placed on different monitors,I propose a Dual-Window Workspace Comparison feature that would allow users to open and view two separate FME windows on different screens. Key Benefits:Boosts productivity by leveraging multiple monitors. Simplifies workspace comparison and troubleshooting. Enhances user experience in managing complex data transformations.
BACnet (Building automation and control network), is a protocol that building utility sensors use. If FME was able to read the sensors directly the information that could be read in would be extremely valuable for aggregating data on building performance/status. I see this working fantastic with FME Data Streaming and pushing this data to ArcGIS services, so we can visual in dashboards which rooms are currently being cooled/the temperature/which rooms are occupied/etc, and so many other things.
Version control is only as good as the business processes surrounding it - being able to commit to a git repo/FME version control is great, but it’s easy to miss in a publishing workflow and is not currently possible to enforce.In order to support more robust SDLC & change control processes, it would be extremely useful to be able to either automate a git commit when publishing/republishing a workspace, or to configure a workspace/repo/flow env to make commits compulsory.My blue-sly vision for this would be that the configuration is set at the repository level on FME Flow, and in repos where compulsory commit is enabled, the publish action in FME Form cannot be used until a commit has been performed with the current version of the workspace.
Would just make workspaces a little cleaner.
In-depth courses for all FME skill levels.
Explore our extensive collection of how-to guides and tutorials.
Log an issue with us. Our support team is ready to help.
