Using readers and writers to extract and load data.
Recently active
Hi allI have been batting my skull against this for some time and havent been able to come up with a solution.I have about 60 datasets (and will be receiving about 60 every month from now on) where every line has a date and a time and a value containing a number. See attached file. its a dataset from a callcenter where every line is and hour and the number is the number of staff that are answering phones. the number of staff varies from hour to hour and from callcenter to callcenter. but the dataset only contains date hours where a staff memeber has been checked in, if none have checked in here isnt a line. My task is to find the ‘missing’ lines/hours where no staff has been active to monitor if the callcenter has been unstaffed when it shouldnt have been. I cant figure out a way to find these missing lines and add them in an automated way the i can use over and over again. Has anyone tried anything similar and found a solution?
I’m currently working on a project in relation to building a major road network which can be used for future traffic surveys. I’ve hit a snag where I’m trying to find potential way of detecting branches within the carriageways so that we can set a condition that should the branch be longer than 500 metres, then we need to separate this branch as its own features. What I’m struggling with is trying to detect the branching by applying unique identifiers (e.g. integer values) if branching occurs within a connected groups of features, something similar to the below: Essentially, I’ve built a two-way road network using the Ordnance Survey road network data and I’ve merged the road sections based on their connectivity (using the [NetworkTopologyCalculator] and [LineCombiner] transformers) and also depending on their ownership (i.e. locally owned vs centrally owned roads) and their road classification number (e.g. A303, A1(M) etc.). Within each of these merged features we need to work out
I just updated to version 25.2 and my oracle reader is failing and throwing this warning (not an error):An error occurred while fetching from query `SELECT a.*** FROM *** a' : OCI_SUCCESS_WITH_INFO: ORA-24347: Warning of a NULL column in an aggregate functionWhich causes Translation FAILED with 0 error(s) and 1 warning(s) (0 feature(s) output).My colleague tried the same on version 25.1 and whilst he also gets the warning his translation does not stop and he gets all features to work with. (As did I before my update)Ignoring the fact that the view should be fixed, is there a way to continue the translation with all features? Troubleshoot option Ignoring failed readers simply skips the reader and thus returning zero features.Furthermore I did see a few changes regarding oracle in the changelogs but I could not pinpoint which one is referring to this problem.
Is it possible to switch the Versioning in an Enterprise Geodatabase (on SQL Server) from "traditional" to "Branch Versioning" without touching FME Workspaces containing Esri Geodatase Readers and Writers (GEODATABASE_SDE)? In a first attempt in the test environment, everything seems to be running like before, after we switched to Branch Versioning of our feature classes. A quick look in the geodatabase tables seems to prove that the internal fields like GDB_ARCHIVE_OID, GDB_FROM_DATE, ... are filled correctly. Can it really be that simple? Does FME support the switch to Branch Versioning while keeping the Workspaces untouched? Of course, this would be a dream, but this seems too good to be true.
Hi!I have a workspace containing four Writers that write tables to a geodatabase. One of the writers is not working as expected. When i press “Run workspace” some of the attributes get null on almost all of the rows. It seems to filtered on one of the attributes (an attrribute called Name) since it is the same names that ends up null. The transformer in the step just before the writer looks as it should, so something goes wrong in the writer, not somewhere in the workspace. The strange thing is that when i press “Run to this” on the problematic writer, everything works as expected and there is no null values anymore. I don’t get any error messages. Using FME Workbench 2025.2Hope someone has any idea of what could be wrong, or if it is a bug?
Hi,Into an Esri ArcGIS Portal Feature Service Writer sending data to ArcGIS Portal what is the difference between Publishing and Spatial.There is nothing about Spatial in the documentation.Thanks.
The failed feature type issueWe are reading in many featureclasses using GEODATABASE_SDE reader in workspace and recently noticed an issue in FME Form (24.1) where it only throws a warning“A table name ‘HeritageSites_Exclusion’ could not be found in the geodatabase as either a featureclass or table. This may be due because the current user does not have permissions to see HeritageSites_Exclusion”The rejected feature handling is set to TerminateThe workspace runs in success, although this missing table was joined upon and is necessary for the right outcome. I guess I’m posting here to see if others have/had this issue, where only a warning of a missing featuretype is given… we believe it should be an Error WorkaroundObviously, we can place NoFeatureTesters and connect a Terminator to the NoInput port. This is alot of work, so we looked into parameters of the GEODATABASE_SDE reader moreUnsure how the Reader parameter “Ignore Failed Readers” truly works. It seems designed very early on to
Hello,I am creating PDF files using FME and am experiencing a serious performance issue (the process is very slow).I have a PDF template that I read and merge with my data so that each ID gets its own template. I use specific X,Y coordinates to write text at the correct positions in the PDF, and then I use the ID in the PDF writer to generate one PDF document per ID that includes both the data and the template.The output is correct and matches my expectations; however, exporting all records with the template takes a very long time.I was thinking to use the PDF Page Transformer, but rebuilding the pages also takes a significant amount of time.Is there any way to improve performance and make my current solution faster without completely rebuilding the workflow? thanks in advance FME 2021
Can FME create a pg database tables with attribute names that have upper cases? I know that this is possible in postgres - column names are case-sensitive when double-quoted. But no matter which parameters of PostgreSQL writer I select (standard/dynamic writing, with checked or unchecked option to Lowe Case Attribute Names in Table Creation Parameter), FME keeps replacing double-quotes with underscores and output is always folded to lowercase. Is it something that can be bypassed?
Hi I’m clipping point clouds with a polygon (1km x 1km), the inside result goes through a rasterdemgenerator with a cell spacing of 0.5m and I write it away as a geotif. When I compare the result with the polygon I see that the geotif is 0.25m bigger than the polygon. I know it has something to do with how a pixel is considered. As a polygon or as point in the center or something, but it doesn’t make any sense to me as you can divide 1km perfectly by 0.5m… It should be 2000 pixels by 2000 pixels right? How do I solve this? I tried playing with the clipper settings. Like changing “determine cell location by” to bounds and some other things. I got a result that’s 0.25cm smaller, but never an exact fit.
Hi, I am writing data to Esri ArcSDE in FME Form and have tested using both CoordinateSystemSetter and Reprojector to define the coordinate system. However, when I open the feature class in ArcGIS Pro I get a message that the coordinate system is missing. Has anyone had similar experiences?
We have multiple workspaces that write to ArcGIS Online with multipart geometries. The old ARCGISONLINEFEATURES writer works and correctly writes the geometries. The new (2025) Esri ArcGIS Feature Service Writer returns an error rejecting the features and the following warning:ArcGISFeatureServiceWriter: INSERT error for a feature in Project Stats by State. The error code from the server was '1000' and the message was 'Adding/updating geometries that result in SQL GeometryCollections is currently not supported (SQL Server performance).' Using a Deaggregator seems to fix this in most cases. However, we want to retain the multipart geometries. In other cases, I see a similar warning and the following error after using a Deaggregator:ArcGISFeatureServiceWriter: INSERT error for a feature in Terracon Offices Closest Areas. The error code from the server was '1000' and the message was 'Data is Null. This method or property cannot be called on Null values.' I’ve tried a NullAttributeMapper a
Hi,I’d like to read a folder of documents and a set of points (say from a csv) and then join these together (filename of documents might match an attribute on the points), then write them all to an ArcGIS Online feature service with the documents as attachments. I would ideally like to do this in one workspace and not have to hardcode anything.I don’t think this is possible without using one workspace to write the CSV to feature service, and then another to read it and add attachments using the ArcGISAttachmentCOnnector, as that requires the feature service URL, which may not be known at the time of creating the feature service.Thanks,Marc
I’m struggling with dynamic worflow when i want to change the target MS SQL DB schema (table qualifier). When i want to migrate data between databases with the same DB schema, it’s fine. But when i want to change the schema i’m running into errors. So far I could manage to migrate data from custom schema (e.g. “ldm”) to default “dbo” schema manipulating “fme_feature_type” and fme_feature_type_name” (also not sure about the differnce of these 2 attributes) in Attribute Manager assigning just Table name and omitting the schema qualifier.When i want to use another target schema than “dbo” (e.g. “test”) i always run into error:Microsoft SQL Server Spatial Writer: A feature has arrived with 'fme_feature_type' of test.OPSiteEK, but the feature type is not recognized. Halting translation1) Does anyone know how to reliably change the target schema for dynamic worflow?2) I have also noticed that if I want drop and recreate tables with no data, i have to send at least 1 data feature, because if
When using the new ESRI ArcGis Feature Service I get this error when trying to write fme_aggregate geometry to AGOL: Python Exception <FMEException>: FMEException: 1: <class 'fmeobjects.FMEAggregate'> cannot be converted to esriGeometryPolylineWith the old writer I did not have this problem. Is there some solution for this? Deaggregating is not the solution i neede. The multi part geometry is only contaning lines. I have tried GeometryCoercer, but without effect.
Has anyone in the UK who is using the Ordnance Survey Maps API managed to get them added into FME for use as a background map. I have tried and get nothing to display….Any advice welcome...
Does anyone have experience working with the MCAP file format in FME? MCAP is an open file format designed to store large amounts of time-stamped data, for example from robots, sensor recordings, and other systems that generate a high volume of messages.https://mcap.dev, https://foxglove.dev/blog/introducing-the-mcap-file-formatIn my case, I am working with MCAP files from a sensor box that determines the position of rail vehicles with track-level accuracy and records the track sections where a track-laying machine has been operating.I am aware that there is currently no dedicated Reader, Writer, or Transformer for MCAP in FME, nor is there anything available on the FME Hub. At the moment, I am at a very early stage of understanding and analyzing this data. There is, however, a Python library available for working with MCAP files: https://mcap.dev/docs/python/.
Hi All,I’m using FME 2021 32Bit on GESW 4.1 and unable to read the tables. Screenshots showing starting of TICS server & modules which I used in GESW along with error from FME are attached for reference. Can someone point out to the location which I’m missing anything such that tables can be read… Thank you
Hi,I need to use many hosted GDB from the web. They aren’t in the same root folder, they all have sub folders.Quebec Hydrographic Network GeobaseIs there a way I could use Multiple URL instead of having to use 15 FeatureReader, one for each GDB?Or a way to download everything from the root folder ?Thank you.
I’m getting an error when reading an Oracle database on the Datetime. Error converting from OCI datetime to C character string : `ORA-01805: possible error in date/time operation' These are my current installs
Hi All,I have a complicated question. I created a custom transformer which can read any table from our Postgres/PostGIS database and write in into a list of desired formats (like Geopackage, Excel, Csv, FileGeodatabase).I have an issue when writing to Excel. A certain attribute is shown in a scientific notation. Ik know how to solf it in an none dynamic way. I could use an ExcelStyler setting the cstom number format to 0, or add an attribute xlsx_row_formatting or apply formatting to a attribute by adding an attribute <attrName>.formatting. I just can't figure out how to do this in a dynamic mode. I wish there was a excel writer to disable scientific notation, but there isn't. Any help or suggestions are appreciated. Arno
We are preparing to upgrade to FME Form 2025. I am running a job in the 2024 and 2025 version to compare the output. I have found that in 2024 the FeatureReader didn’t reject any records, but now in 2025 some records are being rejected. FME(R) 2024.1.3.0 (20240927 - Build 24627 - WIN64)FME(R) 2025.1.3.3 (20251104 - Build 25648 - WIN64)The rejected records say:fme_rejection_message buffer Coordinate system does not match first feature's coordinate systemfme_rejection_code buffer INVALID_GEOMETRY_COORDINATE_SYSTEM Any thoughts on why the records fail in 2025 but not 2024?I will continue working backwards to see what is unique about these records.
Hello Hello,I would like to ask about Custom Transformers.I am a beginner and I am very confused. I have three questions.First question:When I input the value 345/1000 into a Custom Transformer attribute include field, it is automatically divided and get 0.345 as a result . I want to keep the value exactly as 345/1000.Even outside the Custom Transformer, the value is 345/1000. I understand that the Custom transformer is treating it as a number. I tried converting it to text before entering the Custom Transformer, but none of my attempts worked.@Text(@Value(attribute)) does not work.I also checked the shared parameter, and it is set as Text.and another issue if I have value 900 A , custom transformer read it as null .I do not know how to solve issuesSecond question:The Custom Transformer works correctly, but when I check the shared parameters, they all appear in red color in the Manage Parameters window. However, when I run the process, it reads the data and produces the correct output.
Why does XML header of the FME generated file displays the XML header as <?xml version="1.0" encoding="UTF-8" > instead of <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
Hi everyone,I have a geometry problem in FME and I’m not sure about the best workflow.In the image below, the red point belongs to a feature that is connected to a black point (the black point is the reference).What I need to do is: Take the line highlighted in green. Starting from the black point, find a position 10 cm along the line. Move (or recreate) the red point exactly at that location — meaning: the new point must lie on the line, and must be 10 cm away from the black point. I know the relationship between the red point and the black point from an ID attribute, but my problem is:How do I calculate this “point on line at 10 cm distance” inside FME?