Shape the future of FME with your ideas
Open ideas have been reviewed by our Product Management and are open for commenting and voting.
To accept values like: urn:ogc:def:crs:EPSG::4167The context here is I'm building an ArcGIS ModelBuilder tool that sends spatial extents to a Spatial ETL tool from the active map, but in arbitrary spatial references, so the extent needs to be projected to match a WFS feature type DefaultCRS.
I'm working with a customer who is planning a data migration from one document management system to MS Sharepoint. At the moment we are looking at what options there are to use FME to do this via the Sharepoint Web Service to see if its possible to tackle this with a custom transformer or similar.It would be great to write more than just Sharepoint lists, especially as in this instance we're talking about migrating 5 million documents.
As an FME AR app user, I'm very interested in the Geolocation function. With this addition, we'd like to use FME AR in the field.
FME dates are irritating when working with MS SQL as I can handle dates more effectively in MS SQL.I have a workbench that processes 100+ CSV files, each file has a different structure so I process each file dynamically.Dates can be formatted as 19 Sep 2011 13:09:00 or 19 Sep 2011 13:09 which MS SQL is happy enough with but FME isn't.At the moment I use a Python script to load the full file and do a regex replace to resolve the FME date issue.It's really bad that FME simply nullifies any date issue so it is quite possible we are unaware of issues.
In the postgresql database, I work with sequences that are used by several tables (global). Unfortunately it is not possible to fill with FME data sets and to address the global sequence.The following error message currently appears if the ID is left blank on the writer:Bulk copy failed on table 'edit_gbki.spielplatzstammdaten' using delimiter ':'. Error was 'FEHLER: NULL-Wert in Spalte »t_id« verletzt Not-Null-ConstraintDETAIL: Fehlgeschlagene Zeile enthält (null, b474294c-8d3f-4c66-8085-69f59266582d, 15, 210, 1, 2920, Mosergarten Spielplatz, null, ...'
Would be AWESOME to have a transformer where you could pick a list element and have it flatten out the list into multiple attributes on the same record.
while using the "Directory and File Pathnames Reader". I need to be able to iterate through created output to get the latest file created. At present I have to manage the output into separate folders. And the only means to get a latest file in any given folder is to sort by date modified and then sample the list with [1] as the sample rate. It would be nice to be able to determine the exact latest date. this is similar to the idea for a sorter but takes it one step further.
Add function to skip reading some components in a transformer.For example:Feature Merger - If user only needs to read MERGED output, add a skip or turn off function on other output such as Used Supplier or Unused supplierorStatistics Calculator - add possibility to skip/ turn off Complete output and only run Summary output (or vice versa
Hi FME’ers, I recently work with QR Codes and I was curious if FME supports those Codes. As my Server has no access to the internet (security reasons), the provided GOQRCodeGenerator or the google API is not of any help. Is it possible to build a native QR Code Writer for FME? Maybe based on some open source projects like qrencode (linux).Best regardsRoland
Metadata is important. You know this as well. Yet I think the reader/writer support for those formats are lacking, to say the least. I am aware that the title asks for more than one format so lets break them down:OGC Catalogue services for the web uses XML (admit, you love xml ;-) ) and have several good standards for content. A reader for searching and displaying posts from a metadata portal is increasingly important as more and more national metadata portals goes live.CKAN and DACAT-AP are both common government and EUtypes of metadata portals. There already exists a CKAN writer plugin (https://github.com/OpenGov-OpenData/CKAN-DataStore-Writer-for-FME-Software) but I think it sholuld be integrated into the main software.scripting ESRI metadata management is lousy at its best. As with ogc catalogue services it is based on xml (more love) and both a reader and writer would greatly improve life those of us working with enterprise databases, file geodatabases and even shapefiles.I have marked FME Desktop as the space below but I believe one can have great use of this on server as well.
***Note from Migration:*** Original Title was: Respecting field name length limitations and reserved keywords per feature writer type 1) Field name lengthAs the maximum length of a field name in various formats is known (databases, shapefile, etc), it would be very helpful if FME Writers validated this on attributes entered.https://desktop.arcgis.com/en/arcmap/latest/manage-data/gdbs-in-sql-server/enterprise-geodatabase-limits.htmhttps://desktop.arcgis.com/en/arcmap/latest/manage-data/gdbs-in-oracle/enterprise-geodatabase-limits.htmhttps://pro.arcgis.com/en/pro-app/help/data/geodatabases/manage-file-gdb/file-geodatabase-size-and-name-limits.htm2) DB KeywordsIn a similar fashion, there are reserved keywords which can't be used as field names in many formats - this could also be validated on writers.https://docs.microsoft.com/en-us/sql/t-sql/language-elements/reserved-keywords-transact-sql?view=sql-server-ver15Error messaging for this in the translation log could also be clearer.
The time has come. I ditched python 2.7 at my first client when upgrading server to 2019.1And now of course, some workspaces using python don't run anymore, but which among the approx. 100 workspaces do use python, and have the version set to 2.7??I went for the .fmw reader, reading all .fmw files in the repository-path. With the Transformers features, I did find all workspaces containing a PythonCaller transformer.But, for this task it would have been super helpfull If I could read all private parameters (looking for python scripted) and read all of the workspace properties (especially the python startup- / shutdown-scripts and the Python Compatibility setting).So... Add all workspace properties and the private parameters as features to the .fmw reader please, as a help for all future upgrades....Kind regards,Martin
Restart core and engine services from the web UI
Keybase is a key directory that maps social media identities to encryption keys (including, but not limited to PGP keys) in a publicly auditable manner[. Additionally it offers an end-to-end encrypted chat and cloud storage system,] called Keybase Chat and the Keybase Filesystem respectively. Files placed in the public portion of the filesystem are served from a public endpoint,[as well as locally from a filesystem mounted by the Keybase client.Keybase supports publicly connecting Twitter, GitHub, Reddit, Hacker News and Mastodon identities, including websites and domains under one's control, to encryption keys.Keybase is a powerful tool and it would be great to see FME connect to this platform using a connector similar to the SlackConnector
In stead of typing in the right attribute value in de tester parameters, one should be able to select the attribute value from a populated unique values list for a specific field. This option would be the equivalent of the Arcmap "Get unique values" button in the select by attribute dialog box and has two major advantages for more complicated attribute values:- selecting is faster then typing in- selecting decreases the chance of typing errors to 0.
Cross-Chain Message Passing (XCMP) scheme is a subset of the Polkadot protocol. It defines how messages can be passed among parachains with no additional trust assumptions beyond the economic security of the relay chain.The XCMP scheme achieves the following properties:Trustlessness Since the same set of validators secure one parachain as another while they also guarantee correct message passing, XCMP requires no more trust than a single blockchain would.Consistency We provide absolute guarantee that the messages received were exactly those sent, even despite any chain reorgs.Availability Polkadot guarantees that the messages will not be lost and are kept available. This is achieved by distributing erasure-coded pieces that can be used to reconstruct messages.Maintaining the right ordering for messages output by parachain blocks is guaranteed by Input/Output validation.Efficiency The protocol avoids too much bandwidth overhead and allows messages to arrive as quickly as possible.Here is a little bit more information:https://research.web3.foundation/en/latest/polkadot/networking/4-xcmp.html# https://medium.com/web3foundation/polkadots-messaging-scheme-b1ec560908b7 This would allow FME to talk with blockchain technology create parachains that could tie into spatial data.
I would like to write out all file properties such as Version, Status and State etc not just the output attributes available under List
I often find myself needing to do certain things on list members such as removing or extracting the first/last element, or extracting/removing the nth elementh, reversing the list, etc. It would be great to have a transformer that allows these types of operations so you can manipulate list data easily (spoken by someone that doesn't do Python for a living!). I have also found one of @takashi's custom transformers (ListSampler) that is sometimes useful, but doesn't have the options I use most in the Sampler such as "Last Nth Element", which would be a great workaround to drop the last element in a list by picking the unsampled output. Thanks!
I'm just going to put this out there:Takashi needs his own badge. A badge or a statue - whichever one is easier.Ed
FME Server Automations provide the ability to set most workspace parameters via Global Keys but date/time parameter values are not supported. There is no option to select a Global Key as a value for workspace date/time parameters and the parameter value must be set individually for each workspace in the automation.Please add support for date/time values as Global Keys in FME Server Automations.
It would be helpful to have the ability to schedule jobs to repeat on a monthly basis. We recently got an FME Server license and I had planned to move our FME job runs from Windows Task Scheduler to the FME Server Scheduler. However, one of the jobs needs to run on the 7th of every month. But the longest time unit (in the FME Scheduler) for repeat scheduling is "Days". So I think this job will have to stay in Task Scheduler, unless we do some customizing. It would be nice if "Months" was added to the drop-down.
Sieving is removing clumps of contiguous cells in a categorical raster that are less than a threshold tolerance (number of pixels). The replacement pixel value can be one of NoData, default value, majority filter, value of largest neighboring clump, depending on what algorithm is used.This is used to clean up noise in categorical rasters without contaminating the boundaries of 'good' pixels the way a simple majority filter would.GDAL already has an implementation:https://gdal.org/api/gdal_alg.html#_CPPv415GDALSieveFilter15GDALRasterBandH15GDALRasterBandH15GDALRasterBandHiiPPc16GDALProgressFuncPv as does R:https://www.rdocumentation.org/packages/bfastSpatial/versions/0.6.2/topics/areaSieve as well as various python packages:ex. https://rasterio.readthedocs.io/en/latest/api/rasterio.features.html#rasterio.features.sieve but it would be nice to have a native FME version.
In order to test FME you can use python and fmeobjects. You can inject a certain geometry with specific attributes with just a few lines of python code, and of course we can run it using the behave library or your favorite test framework. In the following example, we check the geometry if it has a Z value, and if this Z value is between certain boundaries. The following example is in dutch, but should be fairly readable for non native speakers. check_punt_geometry.feature # language: nl Functionaliteit: Check punt geometrie Afsluitmiddel Punten (punt x,y,z) liggen tussen de drempel waardes en hebben 3 dimmensies Scenario: Afsluitmiddel punt heeft geen Z-waarde Gegeven de testset bevat "CHECK_3D_POINT" En heeft de volgende geometrie """ POINT (10 20) """ En de drempel waardes voor Z liggen tussen 7 en 300 En het attribuut "meta_feature_type" is "Afsluitmiddel" En het attribuut "OBJECTID" is "55017666" Als de check uitgevoerd wordt voor Afsluitmiddel Dan staat de volgende melding in het log bestand """ WARN | Afsluitmiddel (ObjectID: 55017666) heeft geen Z waarde. #1005 """ Scenario: Afsluitmiddel punt heeft een geometrie met een Z waarde tussen de drempel waardes Gegeven de testset bevat "CHECK_3D_POINT" En heeft de volgende geometrie """ POINT Z (10 20 30) """ En de drempel waardes voor Z liggen tussen 7 en 300 En het attribuut "meta_feature_type" is "Afsluitmiddel" En het attribuut "OBJECTID" is "55017666" Als de check uitgevoerd wordt voor Afsluitmiddel Dan staan er geen warnings in het log bestand Scenario: Afsluitmiddel punt heeft een geometrie met een Z waarde onder de drempel waardes Gegeven de testset bevat "CHECK_3D_POINT" En heeft de volgende geometrie """ POINT Z (10 20 0) """ En de drempel waardes voor Z liggen tussen 7 en 300 En het attribuut "meta_feature_type" is "Afsluitmiddel" En het attribuut "OBJECTID" is "55017666" Als de check uitgevoerd wordt voor Afsluitmiddel Dan staat de volgende melding in het log bestand """ WARN | Afsluitmiddel (ObjectID: 55017666) de Z waarde ligt buiten de drempel waardes (minimum: 7.0, maximum: 300.0). #1005 """ Scenario: Afsluitmiddel punt heeft een geometrie met een Z waarde boven de drempel waardes Gegeven de testset bevat "CHECK_3D_POINT" En heeft de volgende geometrie """ POINT Z (10 20 400) """ En de drempel waardes voor Z liggen tussen 7 en 300 En het attribuut "meta_feature_type" is "Afsluitmiddel" En het attribuut "OBJECTID" is "55017666" Als de check uitgevoerd wordt voor Afsluitmiddel Dan staat de volgende melding in het log bestand """ WARN | Afsluitmiddel (ObjectID: 55017666) de Z waarde ligt buiten de drempel waardes (minimum: 7.0, maximum: 300.0). #1005 """ steps.py from behave import * from sys import path path.append(r"C:Program FilesFMEfmeobjectspython36") from fmeobjects import FMEFeature, FMELogFile from src.checks import * import unittest from unittest.mock import patch def test_macroValues(self, mock_macroValues): mock_macroValues.return_value =3D 7 @given(u'het attribuut "{key:w}" is "{value}"') def step_impl(context, key, value): if not hasattr(context, "input"): context.input =3D FMEFeature() context.input.setAttribute(key,value) @when(u'de check uitgevoerd wordt') def step_impl(context): context.logger =3D FMELogFile() context.logger.holdMessages(True) processFeature(context.input) context.logger.holdMessages(False) @then(u'staat de volgende melding in het log bestand') def step_impl(context): heldMessages =3D context.logger.getHeldMessages() heldWarning =3D next((message for message in heldMessages if message.s= tartswith("WARN")), None) assert context.text.rstrip() =3D=3D heldWarning @then(u'staan er geen warnings in het log bestand') def step_impl(context): heldMessages =3D context.logger.getHeldMessages() heldWarning =3D next((message for message in heldMessages if message.s= tartswith("WARN")), None) assert heldWarning =3D=3D None @when(u'de check uitgevoerd wordt voor afsluitmiddel') def step_impl(context): context.logger =3D FMELogFile() context.logger.holdMessages(True) processAfsluitmiddel(context.input, context.testset) context.logger.holdMessages(False) @given(u'heeft de volgende geometrie') def step_impl(context): if not hasattr(context, "input"): context.input =3D FMEFeature() context.input.importGeometryFromOGCWKT(context.text) @given(u'de drempel waardes voor Z liggen tussen 7 en 300') def step_impl(context): pass src/ check.py from enum import IntEnum, unique import fme from fmeobjects import FME_WARN, FMELogFile @unique class Seed(IntEnum): ... CHECK_3D_POINT = 1005 CHECK_3D_LINE = 1006 CHECK_3D_POLYGON = 1007 ... def processFeature(feature, logger = None, seeds = frozenset(range(1000,9999))): if logger is None: logger = FMELogFile() logger.logMessageString("Kunstwerkdeel check gestart.") ... logger.logMessageString("Kunstwerkdeel check uitgevoerd.") def processAfsluitmiddel(feature, seeds = frozenset(range(1000,9999))): logger = FMELogFile() logger.logMessageString("Afsluitmiddel check gestart.") meta_feature_type = feature.getAttribute("meta_feature_type") object_id = feature.getAttribute("OBJECTID") ... process3DPoint(feature, meta_feature_type, object_id, logger, seeds) logger.logMessageString("Afsluitmiddel check uitgevoerd.") def process3DPoint(feature, meta_feature_type, object_id, logger = None, seeds = frozenset(range(1000,9999))): if logger is None: logger = FMELogFile() try: minZ = float(fme.macroValues["Geometrie_MinimaleZ"]) maxZ = float(fme.macroValues["Geometrie_MaximaleZ"]) except AttributeError: minZ = 7.0 maxZ = 300.0 if Seed.CHECK_3D_POINT and feature.hasGeometry(): if feature.getDimension() == 2: logger.logException(ValueError(f"{meta_feature_type} (ObjectID: {object_id}) heeft geen Z waarde. #{Seed.CHECK_3D_POINT}"), FME_WARN) elif not(minZ <= feature.getAllCoordinates()[0][2] <= maxZ): logger.logException(ValueError(f"{meta_feature_type} (ObjectID: {object_id}) de Z waarde ligt buiten de drempel waardes (minimum: {minZ}, maximum: {maxZ}). #{Seed.CHECK_3D_POINT}"), FME_WARN) processFeature(feature, logger, seeds)Running this against behave gives the following output: Functionaliteit: Check punt geometrie # features/check/3010_afsluitmiddel/check_punt_geometrie.feature:2 Afsluitmiddel Punten (punt x,y,z) liggen tussen de drempel waardes en hebben 3 dimmensies En heeft de volgende geometrie # features/steps/steps.py:74 0.793s """ POINT (10 20) Dan staat de volgende melding in het log bestand # features/steps/steps.py:25 0.000s """ WARN | Afsluitmiddel (ObjectID: 55017666) heeft geen Z waarde. #1005 """ En heeft de volgende geometrie # features/steps/steps.py:74 0.803s """ POINT Z (10 20 30) Dan staan er geen warnings in het log bestand # features/steps/steps.py:31 0.000s En heeft de volgende geometrie # features/steps/steps.py:74 0.830s """ POINT Z (10 20 0) Dan staat de volgende melding in het log bestand # features/steps/steps.py:25 0.000s """ WARN | Afsluitmiddel (ObjectID: 55017666) de Z waarde ligt buiten de drempel waardes (minimum: 7.0, maximum: 300.0). #1005 """ En heeft de volgende geometrie # features/steps/steps.py:74 0.800s """ POINT Z (10 20 400) Dan staat de volgende melding in het log bestand # features/steps/steps.py:25 0.000s """ WARN | Afsluitmiddel (ObjectID: 55017666) de Z waarde ligt buiten de drempel waardes (minimum: 7.0, maximum: 300.0). #1005 """This way, you can automaticly verify your FME workbench without relying heavily on the available test sets. Things to improve are the ugly try ... except AttributeError: block to inject a macroValue into the test, possible solution for this is creating a mock. Also a next step might be testing this on a server, while I have been able to run a docker instance of FME server, I have yet to discover where the fmeobject is located so I can run this test. Also obviously, you need an FME license for this, so we need to figere out how to inject the licence key to get the code to work in a secure way. So, you might ask, can we extend this to also test our workbench. Actually, that is super easy and barely an inconvenience! According to this knowledge.safe.com blog post, you can automaticly run a workbench from the python API, hooking this up to a Given step in behave is trivial, you could also check if certain links exists using a textual graph parser in the dot notation for example. I leave this as an exercise for the reader to implement.
Hi,It would be very useful if access to set the text annotation tool in sketchup would be possible. Reason is that we try to establish a sketchup template file, containing a cutout area from a geo-location. The sketchup file is grouped and layered with geometry of terrain and existing buildings. A predefined text annotation would enhance the usecase of the sketchup template. Please see attached screenshot.Thanks!
I finally encountered the first WFS-T service I can to talk to, at a client. With all security in place it was officially switched on today.Thinking about all the neat possibilities, eating a small cake, I found out there is no WFS-writer in FME.Arrrrrgggg!!!!!Will have to take into account I'll have to craft all XML-bodies myself if I propose something.
In-depth courses for all FME skill levels.
Explore our extensive collection of how-to guides and tutorials.
Log an issue with us. Our support team is ready to help.
