Open ideas have been reviewed by our Product Management and are open for commenting and voting.
https://knowledge.safe.com/questions/38771/can-i-use-a-feature-merger-to-merge-partial-names.html https://knowledge.safe.com/questions/39153/merge-features-by-attribute-that-contains-string-f.html?childToView=39163#answer-39163
Hi,some formats (such as the Text File reader) have fixed schemas that will never change. In the case of the Text File reader the only attribute in the schema is the text_line_data attribute.For these fixed-schema formats, I would like to request an enhancement to the FeatureReader to have the schema automatically exposed, even when the Generic output port is configured. Currently, the schema attributes are only exposed for named output ports.For context, I refer to the following Community question: And to my own support question (to which James Cheng provided a very comprehensive reply):https://support.safe.com/hc/en-us/requests/64419I have reproduced James’ reply below, as it so clearly explains the situation:Thank you for raising this. I understand the frustration as I share the same thoughts and can see why you'd expect the text_line_data attribute to just be available when reading a Text File through the FeatureReader. The behavior you're seeing comes down to how the FeatureReader's generic output port works. This port is a catch-all that merges everything into a single stream, designed to handle any format and feature types with varying schemas. Because of this, it doesn't commit to any schema at authoring time, it doesn't make assumptions about which feature types or attributes will be present, so all attributes come through unexposed by default, even for predictable formats. By contrast, when you use a named output port (or a standalone Text File Reader), the schema is known at authoring time, so the attributes like text_line_data are automatically exposed. In essence, by using the Generic port, the trade-off is flexibility over schema awareness, it can handle anything but at the cost of not pre-committing to any specific attributes. However, as you've rightly pointed out though, the Text File Reader always produces a predictable fixed schema with a single attribute, text_line_data, no matter which files/feature types are being read. So there's a reasonable argument that the FeatureReader could recognize this and auto-expose it on the Generic Port since there's no ambiguity what the output will be. Currently, the Generic port doesn't differentiate between fixed schema and variable schema formats as it applies the same "unexposed by default" behavior uniformly across the board. That said, I think this is a genuinely good candidate for a product enhancement for the generic port. I'd encourage you to post this as an Idea on the FME Community, specifically requesting that the FeatureReader auto expose known attributes for fixed schema formats like Text Files on the Generic port and describing your use case. Community support helps the development team gauge interest and prioritize ideas for future releases, and we can link this ticket to the Idea to keep it on their radar. In the meantime, it's worth noting for this specific format, the feature types are all fixed to text_line, so using the default One per Feature Type output port setting will also route all text file features through a single text_line named port, similarly to the Generic Port, which will expose the text_line_data attribute automatically. If your workflow requires the Generic port, then the only available options are manually setting the Attributes to Expose parameter on the FeatureReader or placing an AttributeExposer downstream. regards,Nic
This idea is a specific use case of the more general:This idea exists simply to assist people searching specifically for information regarding exposing the text_line_data attribute on the Generic port of a FeatureReader when reading Text File data.
All the grids in FME servers are not sortable by users, sorting is the very basic function expected by all users from any software these days.Please allow users to click on any column header to sort ascending/descending/removing sort for a given column. The user should be able to hold the shift or control key to choose multiple columns to sort. The sort function should work for all supported data types in the column, e.g. Number, text, date, user, etc.
On FME Flow you can filter failed jobs. It would be great to be able to filter by error message. This way if you’re trying to troubleshoot an error that is occurring you can easily see which scripts have been impacted.
Is there a way to have feature writer (PNG format), to generate .pgw instead of .wld file?For feature writer (JPEG format) it has option to change .wld to .jgw Just wondering if it would be possible to do that with PNG as well? Currently I change the .wld to .pgw manually. I know there’s a way to use SystemCaller to make it change the file extension. But it would be helpful if we can just skip that entirely because it may not work as expected when we Publish workspace in FME Flow. JPEG Format PNG format
When you add a "file" manual key to an automation which get run by an automation app, the option is to choose a file already in the resources of FME Flow. It would be nice to have an option for a user to upload a file like what's already available in workspace apps. For a User Parameter "File" the user is able to upload a file which gets used by the workspace the workspace app runs. The same would be ideal for automation apps.
Either as a new transformer or as an option in the StatisticsCalculator transformers (standard, raster, pointcloud...list?)
***Note from Migration:*** Original Title was: Add Support for Aggregate Geometry Type (Esri ArcGIS Online (AGOL) Feature Service Writer) Add support for writing Aggregate features with the Esri ArcGIS Online (AGOL) Feature Service Writer.Currently this is not supported: https://docs.safe.com/fme/html/FME_Desktop_Documentation/FME_ReadersWriters/arcgisonlinefeatures/quick_facts_arcgisonlinefeatures.htmExample of community discussion of this feature: https://community.safe.com/s/question/0D54Q000080hRcE/cannot-write-to-esri-after-using-aggregator-mutipart-geometry?t=1619549337262
Would be great to have a Search Filter In the dialog which is handy when there are several workspaces in the same repo.
Idea: cancelling a job will rollback the whole transaction of all features for Snowflake WriterObserved behavior: cancelling a job in FME Flow will stop additional features from being written, but features that have already been written to Snowflake are already committed without any rollback of the transaction. There is inconsistency between “Features Written’ as recorded in the job log and actual features written according to Snowflake logs. The job log records zero features written while there have been many features written to Snowflake despite job being cancelled.Desired behavior: cancelling a job in FME Flow will stop additional features from being written and rollback any partial changes the job has just done. Outcome: improved management of jobs.
Hello,To continue with data platform integration, is Safe discussing with Databricks to be added to its Marketplace, like with Snowflake? @irenemunoz maybe.
Since it does not seem possible to automatically shutdown an instance in FME Flow Hosted, one workaround we found is to apply schedules to shutdown the instances. However, this is done manually and can be easily forgotten. A good way to ensure the instance has one is to suggest a schedule at the creation, with having a certain default value created. Hope this makes sense, let me know if this sparks any more ideas or if you need further information on the subject!
This implementation focuses on adding a dedicated log window to the FME Flow Web Application UI. The log window will display real-time and historical execution logs, enabling users to monitor data flows more effectively. It improves transparency, simplifies troubleshooting, and enhances the overall user experience by providing immediate access to process details.
It should be possible to define default pagination parameters at the Data Virtualization API level, while still allowing bespoke pagination configurations for individual endpoints.Currently, pagination must be configured within each workspace, which can complicate workflows and, in some cases, negatively impact processing performance. This configuration also needs to be repeated for every workspace endpoint, leading to unnecessary duplication.A more efficient approach would be to manage pagination at the parent API level, with the following capabilities: A configuration option at the API level to enable or disable pagination globally A configuration option at the endpoint level to determine whether the endpoint inherits the global API pagination settings A parameter to define the default pagination size (i.e., the number of results returned per page)
Introduce the ability to add Dynamic input ports to the PythonCaller.
***Note from Migration:*** Original Title was: Add support for integrated Windows Authentication (IWA)/ Single Sign On (SSO) for rest services (fmedatastreaming) If you configure fmeserver for integrated Windows Authentication (IWA)/ Single Sign On (SSO), it affects the web UI only, not rest services (as far as I know). This enhancement will make it easier to access fmedatastreaming with published parameters etc. in a secure environment.
The problem The documentation for FeatureMerger mentions a "Suppliers First" mode that can reportedly be very beneficial to performance (and, I would imagine, crucial for Streams), but comes with the constraint that all suppliers must have arrived before the first requestor comes in. To my knowledge, there is currently no way in FME of upholding that guarantee in a reliable manner. The timing between Readers, Creators, FeatureReaders and SQLExecutors is not something I can claim to understand, and the completion order can change based on whether caching is turned on or not. This is already troublesome when editing workbenches, but it can be expecially problematic inside custom transformers, where you don't have control over the delivery order of features in your input ports. This isn't merely an issue with FeatureMerger; it's going to be a problem at any time where you rely on input ordering for a transformer to work properly, with specific attention given to Streams and Feature tables. If I have a PythonCaller that configures itself based on data coming from TransformerA before it's ready to accept data from TransformerB, there isn't a lot of options for reliably dealing with out-of-order input. Holding onto features is illegal when bulk mode support is advertized, so the PythonCaller must either opt out of it (which hurts downstream performance) to accumulate any features it's not ready to process until the configuration features have come in. Even then, it can't know when TransformerA has closed, so unless it only expects one configuration feature, it's dangerous to start processing before close() has been called. This might be clearer when considering the attached screenshot: Python_MapAttributes needs the output line from JsonTemplater in order to work with the data coming from MappedInputLines. If MappedInputLines starts sending features first, the PythonCaller can't do anything yet, and can only crash (undesirable) or start buffering features (illegal in Bulk mode). The solution The idea would be to have some sort of Semaphore transformer, something like a FeatureHolder, but with (at least?) two input ports: a "Priority" port, which lets features through normally, and a "Held" port which buffers features until the other port has closed and no more features can go through it. This would ensure that no feature from the "Priority" side can ever arrive before a "Held" feature, thus allowing workflow and transformer designers to guarantee feature ordering downstream without breaking bulk mode. Other relevent use cases One might also consider having a Terminator node which should stop the translation when an assertion or a join fail in some unexpected way, but should wait until every faulty feature has arrived to give proper context instead of immediately stopping at the first one. Bad features could be sent through the priority port and then the priority port routed to a terminator, so that no feature can be passed to the next step until it has been verified that none exist that would trip the Terminator. This would also allow the Terminator to be changed to wait for all features to have arrived before stopping the translation, instead of aborting at the first one (which currently makes sense, as the more you wait, the more you risk that downstream writers will have already started to write incomplete data).
I see the Aggregator has an option to calculate average values weighted by area. It would be handy to see a weighted by length when used for lines. it’s easy enough to handle this calculation without the help of the Aggregator but it would be a nice little improvement.
Include the possibility to align text (left/right/center and topmiddle/bottom in the PDFStyler.Currently the alignment of text (coming from e.g. a LabelPointReplacer) in the PDFPageFormatter needs to be done by trial and error, and still this does not give satisfactory results when text length can differ.
FME24 provides an overly simplified and bland user interface and experience. Even as a long time FME user I find it difficult to use and even train new users. Personally, I’m yet to find a single benefit of the change.E.g. Simple icon colours which categorise items in the navigator (Purple = Parameters) are gone.Similar to being able to select dark mode, I think it would be extremely helpful to enable users to select the theme and icon styles that improve their user experience.
this string compare mode would help simplify workspaces changing case on keys and such just for comparing.
I’ll start by saying I think FME Form is one of the greatest software tools ever created. With that said I have been waiting for one additional Zoom function since I started using it 15 years ago. Why can we not get a Zoom to Previous Extent action/button? I don’t want to pretend I know all that is involved with doing this, but it just seems obvious that it would be one of the most used features when dealing with extremely large workspaces. Are we ever going to get it?Safe team, please consider it.Thanks for your time!
This question has been asked so many times and is ridiculously confusing and hard for something that should have been added ages ago. I have a table that has table or layer names that I want to process in FME. I should be able to Dynamic SQLExecutor or FeatureReader to read the list and add them in to form without having to know the schema ahead of time. The issue is the schema they read comes from the input table and not the features it reads. The attributes are returned so you can write them with a dynamic writer, you just can see or use them in form. It needs the ability to dynamically return a value as defined in the source table (for example the @layerID or allow a wildcard to return all fields) and the fme_feature_type, fme_basename and fme_dataset not from the initiator but from the dynamically read table/layers. In this case I have the layer/table name and the ID I want to expose in the initiator table there just doesn’t seem to be a way get the attributes to dynamically read from the imitator table or to just dynamically read all attributes back.fme_attributes should come from the feature it is reading not the initiator, same with the schema it should be the schema of the feature it is reading not the initiatorIt can read the list of tables in but cannot read the attribute I want dynamically. For example, when it reads TRAILSIGNS the attribute I want returned is TRAILSIGNID
It would be useful to allow a way to group related transformers together without having to use a bookmark. Bookmarks can take up too much space, or also the grouping might not be logical to put in a bookmark. Would be nice to just be able to select a group of transformers and make them ‘stick’ to each other, so if you need to move them again you don’t have to manually select the whole area they cover (and accidentally select other things you don’t want), all this can make working with FME more productive
In-depth courses for all FME skill levels.
Explore our extensive collection of how-to guides and tutorials.
Log an issue with us. Our support team is ready to help.
