Shape the future of FME with your ideas
New and Open ideas are awaiting review or have been reviewed by our Product Management team, and are open for commenting and voting.
Certain characters are not allowed when creating files & folders in the OS. So if you pick data from some source and then use it to create files/folders without any validation, there's a good chance things will fail because of these special characters (see below for an example in Windows). It would be great to have a string function and transformer that will clean up a string to ensure it is compliant with these restrictions (maybe a way to also map to whether certain characters are just removed vs. replaced with a character such as an underscore, etc.). Currently this is doable with a series of nested ReplaceString or ReplaceRegEx functions for example, but not as easy to format each time you need to do this. This function & "cleaner" transformer would simplify such tasks and ensure you have no failures in your translations and data writing tasks.
for an output table (in my case esri sde) could you add the option to "Truncate - Create if Needed" i seem to come across this need somewhat often
It would be great if the "Run to this" / "Run from this" options within a workspace could be extended to have a "force run" or "rerun" option. Feature caching is fairly smart and will re-run stale caches when required. The problem is when an underlying dataset has changed and the feature caches aren't aware.For example, when using a FeatureReader or DatabaseJoiner, the underlying data may have changed and, currently, the only way to get the workspace to pick this up is to either re-run the entire workspace, or make a temporary edit in the workspace that causes the cached data to become stale.Another scenario in which this is an issue is if a custom transformer is updated, then the main workspace feature caches do not account for this unless the custom transformer interface changes.
***Note from Migration:*** Original Title was: Add a SQL Server Date format to @DateFormatter function and DateCalculator transformer Microsoft SQL Server (at least on 2012) only allows timestamps containing milliseconds (3 decimal places for fractional seconds) to be written to datetime and datetime2 columns. Dates generated by the @DateTimeNow() function automatically include nanoseconds (9 decimal places for fractional seconds). There is currently no way in the @DateTimeFormat function or DateCalculator transformer to only return milliseconds. Using either the "ISO" format shortcut, or using the %S flag always passes nanoseconds. One workaround I found was to wrap the @DateTimeNow() in a @Format() function like this: @DateTimeFormat(@Format(%.3f,@DateTimeNow()),ISO) to truncate the extra digits.I propose adding a "SQLServer" shortcut that returns the timestamp with only millisecond precision, and a flag of some sort that returns the seconds to millisecond precision.
I need to apply a merge cell to a series of cells in the excel writer. if the styler could do this that would be sufficient. It would be nice to be able to manually identify for instance O34:V34 and apply a value to that merged cell.ultimately if it could all the building of the cell range like O( _count+1):V(_count+1) so that it could always follow the data being written in the preceding rows.
The AttributeValidator currently outputs the results in English. Since it is sometimes necessary to provide localized or custom messages, I suggest that the AttributeValidator include "message numbers" in addition to the full message, much like Oracle does with error messages (ORA-nnnn):Currently: Attribute 'abc' with value 'a' fails check for Type is 'INT'Suggested: Attribute 'abc' with value 'a' fails check for Type is 'INT' (FME-AV-1234)Where 1234 would be a unique integer that identifies the message.To translate, one could then search for FME-AV-nnnn rather than a specfic wording interspersed with attribute names etc.
I frequently need to create multilevel XML. FME does a great job of creating one or two levels of XML. Unfortunately a lot of XML I need to create has at least 3 or 4 'sub' levels- or elements within elements, within elements with multiple attributes etc. The only work around I have is using multiple templaters and creating the inner elements first then using a 'key' attribute to join to the next upper element [in another templater]. This is sort of a daisy chain of nested XML until its finally assembled. However, it would be nice to do this all in one templater.Thank you.
It would be great if the AttributeFilter had a Case Insensitive mode similar to the Tester.I had a situation recently where I'm validating incoming data but occasionally they change the case of the data which isn't a problem generally but in order to make it work in FME Attribute Filter had to enter the different cases:I did get around this by using a TestFilter instead which has the case option functionality but is a lot more time consuming to setup:
***Note from Migration:*** Original Title was: Color bar / color scale for raster or point cloud visualization in Data Inspector In the Data Inspector, I think it would be great to see the color ramp used to color raster or point cloud with related values (for example altitude,) like a scaling bar. It would be much easier to interpret and share transformer visual results !
I would like to see a convenient "preview" of which attribute are coming into conflict in any transformer that have a Attribute Conflict Resolution tab.
It would be really, really helpful to be able to search inside log parameters or full body of the logs in FME server/cloud. In our setup we have many child jobs, if there are issues it is very challenging to track down the cause without been able to do more specific search or filtering. Would it be possible to have an elasticsearch service that indexes the logs and enables very quick search capabilities across the tens of thousands of logs we generate each week?
Having some trouble downloading the contents of a remote FTP directory, keep getting error message 'Remote file not found'.It would be great if there was an option to 'Download Remote Folder' so that you could download a remote directory via FTP. Currently, FTPCaller only understands the instruction to download a specific file, which is a little problematic if there are hundreds of files within a directory and you want to download them via FTPCaller.@jlutherthomas
We use Kerberos as the primary authentication for our Data Lake. Our base software is Cloudera and we have used their HDFS JAR files but have not had luck connecting FME to it using the Apache Hive reader or the JDBC connector. In previous conversations with FME support, I was told that connecting via JDBC was not supported via Kerberos
The default character encoding for the Text File Writer is currently set to UTF-8.It would be more straight-forward and logic if it were set to "System Default" as its default value. This has caused us some issues where it took time to discover this was the cause.
ArcGIS Image Server (current software, not the legacy one) isn't supported as an imagery source but given a connection to a portal with an Image Server the exportImage endpoint would be useful for elevation and general image sources.
OpenDRIVE is an open format specification to describe a road network's logic. Its objective is to standardize the logical road description to facilitate the data exchange between different driving simulators. Can FME add a way to convert traditional GIS vector data (shapefiles, geodatabases, etc.) to XODR?
After a recent FME webinar I posed the idea to the FME team during the webinar about accessing password protected zip files. For example if you receive or work with zip files that are stored in a common directory, or are sent to you via some mechanism, either email, or a sharepoint site, or some internal corporate drive location, or even google drive or dropbox, but for some level of security the zip file is password protected, I think there might be some appetite to streamline the process of accessing this data to update an organizations databases. Since the contents of the zip file are encrypted, it's not likely to go to ArcGIS Online as a open data source, but could ultimately make it's way into a ArcGIS Enterprise internal corporate portal. Being able to have a checkbox for password protection, should enable an entry box for password entry. Now the type of encryption options used, would be those supported by the common compression software, e.g. winzip, winrar and 7-zip at the very least, and yes when I state "zip" file, it should also include ".rar" and ".7z". Since there is no fundamental incompatibility that I'm aware of to incorporate this capability in FME, even at this point as an optional transformer for download in beta to see if there's a desire for full production release, I'm wondering if there is any appetite? Since FME can already read zip files and even KMZ, which is a zipped KML file, there should be no fundamental problem incorporating password access. Even incorporating secure authorized access via password authentication or digital signature of encrypted PDFs should be investigated. I'm sure Adobe would work with Safe, even though they might require some hefty licencing fees? Adobe tends to be very heavy handed with the user only using their tools and portals for digital secure authentication.
Add rejected port to AttributeEncoder so that attributes that can be encoded pass and those that don't exit via the rejected port
I think it would be great if password protected zipfiles can be directly created from the Workbench application.
***Note from Migration:*** Original Title was: FME AR feature attribution and modification that inserts/deletes/updates features in a database Someone has previously posted regarding adding feature attribution to the AR models. If this becomes a reality, what I am proposing is a way for an individual to add/delete/modify features. These modifications can then be pushed back to FME Server/Cloud and validated and inserted/deleted/updated in a connected database. It would also be nice if the models could be password protected so that only certain organizational roles are able to do any modification to the existing model.I'll provide an example of what I have in mind. I work for a telecommunications design company and we have numerous field technicians/PMs across various projects in the US. Presently, our data is collected and then reconciled through an FME workspace. Any updates to the field collected data, whether it's adding an attribute value or collecting a new feature, needs to go through the validation step. However, it would be great if the technicians/PM were able to modify the features within the AR app and after submitting the changes, a workspace is triggered to validate the changes. I'm trying to get our field crew interested in FME AR as I see a lot of utility in visualizing the outside plant features, particularly buried features, but I'm getting pushback because FME AR is a different app than what we're presently using and they don't want to keep switching between apps.
***Note from Migration:*** Original Title was: Extend the HttpCaller with an option to create a secure connection with a certificate Please add an option to enable HttpCaller to create a secure SSL connection using a custom X.509 certificate.Preferably enable providing the certificate in multiple format formats, e.g. both certificate/key separated formats and the joined format of pkcs12.I can provide sample code that works in a PythonCaller (Python 3.7+).
Rather than reading exif data from a specific image file format e.g. jpeg could a universal exif data reader be developed similar in functionality to the ExifTool by Phil Harvey. With the ExifTool you can point it at a directory, single file or a variety of file formats (supports approx. 186 formats) and it returns a table of image metadata. It works really well and I've created a FME custom transformer that makes a system call to the tool (https://hub.safe.com/transformers/exifreader). Other FME users might find this tool useful and it would be great as it's own reader.https://www.sno.phy.queensu.ca/~phil/exiftool/
Currently, a custom transformer can have numerous Inputs, but when the user is setting a parameter value with attribute assignment permitted (or of the "Attribute Name" type), all attributes from all inputs are listed. This makes it difficult to use any type of dynamic merger/joiner inside the transformer, and significantly limits the user's ability to control how the transformer operates on different data. It can also lead to issues when both datasets have an attribute by the same name... or when they don't! It would be ideal if, instead of "Default" and "Off" for "Attribute Assignment," we could select a specific input port and only that input's exposed attributes would be listed.
If you have this data : "01/01/2020"With this pattern : "([0-3][0-9])/([0-1][0-9])/([0-2][0-9][0-9][0-9])"You can only extract data by using the number of order of the different groups you made : "3-2-1" to extract this : "2020-01-01"The official PERL documentation (which is the one recommanded by Safe to look at for REGEX) allows you to use names for groups for an easier calling.So, with the previous data : "01/01/2020"You could use this pattern : "(?<day>[0-3][0-9])/(?<month>[0-1][0-9])/(?<year>[0-2][0-9][0-9][0-9])"And extract data by using this call : "g{year}-g{month}-g{day}" to extract this : "2020-01-01".This exemple is really simple but when you have tenth of group for a very complexe REGEX search, it becomes very difficult to use.REGEX doc on Safe website : http://docs.safe.com/fme/html/FME_Desktop_Documentation/FME_Transformers/!Transformer_Parameters/Regular-Expression-Editor.htmPERL REGEX doc recommended by Safe : https://perldoc.perl.org/perlre.html
In-depth courses for all FME skill levels.
Explore our extensive collection of how-to guides and tutorials.
Log an issue with us. Our support team is ready to help.
