Using readers and writers to extract and load data.
Recently active
I am trying to see if I can update a feature layer in our ESRI Portal from a FGDB. We know we can insert, update, and delete features and have success on those fronts. However, we will like to also have dynamic schema available as we may change the data in our FGDB by removing/adding fields. I have tried several times to get a dynamic writer working to account for a schema change but without luck. I have tried to delete data in the feature layer to then insert but that had errors. Does anyone have advice or has done this? The FGDB is a testing stage as eventually we would want to update directly from SDE.
Hello community,i am using an Feature Writer to write Feature classes in an Feature Dataset in an EGDB.I found out that i need to enter the Feature Dataset Name/Feature Class Name hardcoded. For Example: FeatureDatasetName/FeatureClassName descriped in the following picture: SDE Feature Writer with hardcoded (static) Feature Dataset and Featureclass NameAn dynamically generated Value like “@Value(FD_Name)/FCName” or even the whole value @Value(FD_name_and_FC_Name) is not working as Feature Class.How i found that out? The Features get inserted in the Feature Class, even if i set Table Handling to drop and create. When i use the hardcoded (static) Feature Dataset and Featreclass Name the Table got dropped and created new, so it was working as expected.SDE Feature Writer with dynamic/Attributvalue Feature Dataset and Feature class NameWhen i use log level debug and use the dynamically (attribut value) Feature Dataset and Feature class Name, i get the following error Message which i dont g
Hi, I have a very simple FME worskspace that reads a shp-file and via Esri ArcGIS Feature Service updates a Feature Layer (hosted) on Portal for ArcGIS.The data is simple plygons with a few attribute. The Feature Layer is published from the same data in ArcGIS Pro.When i run my FME script the polygon ends up aprox 2000 km wrong. It does not matter if I have Feature Type Handling: Use existing or Truncate. If i use the old Esri ArcGIS Portal Feature Service writer it works fine. I am using FME 2025.1, Portal for ArcGIS 11.4 and PCS is EPSG:3006
Hi all.I need to identify (and convert) multisurface /curve polygons in a GDB.The GDBs contain one feature class with many polygons: some of these records are multipart polygons. The data I am working with comes from uncontrolled sources using different GIS programs, so the GDBs are put through a workbench to identify obvious spurious geometries like self-intersections (this works well). However, I am having a recurring problem where the downstream processes in other systems are failing due to multisurface /curve polygons.I have tried exposing the fme_type and geometry, but they are always “fme_areas”. However, if I run GDAL ogrinfo tool identifies lots of Multisurfaces, some are also CURVEPOLYGONs. For example, GDB today with 8 records had 64 multisurfaces.Any anyone help point me in the right direction for how I might Id these issues in FME, or incorporate the GDAL tool somehow?Note: I cannot share example data due to its sensitive nature.Many thanks
What is the process for ensuring that a custom coordinate system defined using the recipe from @daveatsafe below is reliably used by all members of a team using FME Desktop and also Server?(better yet: implement Update all available Coordinate Systems with EPSG Codes so it’s not necessary)
I have many parquet files stored in an s3 bucket; when I use the s3 connector to download the parquet file, I am able to read it successfully, but passing the s3 uri (“s3://”) raises an error:PARQUET reader: PARQUET reader: Failed to open file 's3:\my-bucket-s3-test\fme-test\test.parquet' for reading. Please ensure that the file exists and you have sufficient privileges to read itFailed to obtain any schemas from reader 'PARQUET' from 1 datasets. This may be due to invalid datasets or format accessibility issues due to licensing, dependencies, or module loading. See logfile for more informationWhen I try to pass the url:PARQUET reader: PARQUET reader: Failed to open file 'HTTPS:\s3.us-west-2.amazonaws.com\my-bucket-s3-test\fme-test\test.parquet' for reading. Please ensure that the file exists and you have sufficient privileges to read it When I use the feature reader, regardless of whether or not I specify a web connection, it converts forward slashes to back slashes. How do I work w
Hello everyone,A colleague is trying to read a CSV file in FME 2024.2, but the file appears to be generated by Linux-based software. As a result, FME interprets the entire file as a single block of text instead of recognizing each line as a new row. Does anyone know how to ensure FME correctly identifies each line break in this file?
Hello, its me again.So I’m playing with changing excel documents in Form 2024.1 and its going superb, except from the fact that when Form writes out my changed excel it adds an @ in front of some of the formulas. So for example the formula that gets read in as:=XLOOKUP(blabla)Gets written out as:=@XLOOKUP(blabla)Which then makes the whole outcome excel document breaks.I have ran this with caching and see that at last cache before writing then the formulas are intact, so this must happen in the writer or that the excel gets written out in some manner that my excel setup *misunderstands* it?As always, all comments are welcomed (as long as you not a person just looking to score some points by copy/pasting what AI said the problem was).-Hlynur
Hi all,I am currently using ArcPro 3.5 DI tool that uses,FME(R) 2025.0.0.0 (20250228 - Build 25208 - WIN64)Some of my FME scripts are failing when reading an excel file if it as a python start up script that imports arcpy specifically. If i remove arcpy, the excel reader works fine.Is this a bug with the FME build that came with our ArcPro 3.5 or is it consistent with a stand-alone FME install?Some of the warnings are:----------------------------------------------------------The Startup Python Script changed the current FME process locale from 'English_New Zealand.utf8' to 'English_New Zealand.1252' and FME has restored it back to 'English_New Zealand.utf8'. It is undefined behavior to change locale and doing so may cause unexpected errors. For more information, visit http://fme.ly/PythonLocaleError-----------------------------------------------------------Excel Reader: Could not read cell at row '0', column '1' for sheet 'can't read more cells in trial version'. Error message is 'can'
This morning I tried to import data using a WFS concerning a Dutch data source. I got into a weird error:Used WFS: service.pdok.nl/kadaster/kadastralekaart/wfs/v5_0?request=GetCapabilities&service=WFSDoes this look familiar to someone?
Wow - new marketing campaign?!https://youtu.be/XaXGa_zijlo?si=_6mP-oYSZanl9ZfII think Im a fan
Hey everyone, I'm writing some data into a Snowflake database and getting a geometry validation error on the Snowflake side: SnowflakeSQLException: Geography validation failed.When I run it thru the FME GeometryValidator - it passes.If I run it thru the SQLExecutor and call ST_ISVALID it returns false so in theory, I could call the SQLExecutor for every record however with a million records, this really isn't feasible. I thought about grouping the inserts by 1000 and if it fails, run those 1000 thru the SQLExecutor however I couldn't figure out how to have the FeatureWriter continue if it fails on a group.Does anyone have any ideas on how to get these records to continue. I really just want FME to remove any bad geometries and log which one's failed. Thanks!
After a week+ of trial and error I wonder. Have anyone managed to create an FBX-file with FME that contains terrainmodel + ortophoto draped on top. That can be opened WITH materials/texture directly in Navisworks? I have tried all settings - but can not make it work. The files open correctly in FME Data Inspector, Autodesk FBX Reviewe, 3dsmax, Lumion and even Infraworks. And also the export from Infraworks to Navisworks work OK! However I cant seem to be able to open the FBX-file directly in Navisworks - it only shows the terrain with no texture/material. Even when it shows the texture in the "materials" pane. Material is added on a Mesh-group and displays nicely before writing and as mentioned in the other software above.Some stuff I have tried:- All settings on the FBX-writer- Manually changing the scaleuints on the ASCII- file from 1 to 100 to scale to meters instead of default cm. It seems that there is some scaling issues between geometry / material or bug related to mat
Hello, I just upgraded to FME Form 2025.1.2 and am having a problem with the Google Sheets Reader. The Reader previously worked in FME 2024.x, but I am now getting a very ambiguous error whenever I try to read a sheet into a workspace. I literally just upgraded this morning after initially opening the workspace in Form 2024 and successfully running it, so I know nothing has changed on my computer or anything otherwise. Any ideas what I am doing wrong?
Hi,I wonder if someone could help me?I would like to merge pdf files in one folder (folder1.png)with pdf files in different folder (folder2.png) based on data in excel (merge.png). For an example, merge 1.pdf in folder1 with 421.pdf,422.pdf,423.pdf in folder2 like it says in the table. And 2.pdf in folder1 with 424.pdf, 425.pdf in folder2 and so on...Is that possible?Thank you, Jasna
I have a fileGDB with many records, of which I only want to read a Selection.I know how to use the Search Envelope to select records in a bounding boxIs there also a way of Selecting records where an Attribute equals a value? (actually one of three values)Or is the only way with a Transformer after the Reader?Just trying to optimise
Hi,I want to automate the process of downloading a dataset that comes as a zipped folder. Inside the folder, there is a GeoTIFF file that I want to read into FME. After that, I want to write it to a new GeoTIFF file, which should automatically update when the dataset is updated.What is the simplest way to do this?The file i want to work with is this: https://ext-dokument.lansstyrelsen.se/sodermanland/Geodata/Gron_infrastruktur/LstD_GI_Ädellöv_int.zip
Hi everyone,We recently switched to the new version (2024) of FME. I ceated a new fmw-file to convert gml to dgn. But when opening the dgn, I noticed that every level that uses an attribute value has the colour ‘white’ instead of the intended colour.Is this a bug? Can somebody help me how to solve this?Thanks in advance.
I am trying to get my few MS Excel as a reader and then write them to a new Excel all together however, I got this error that says Excel Writer: xlsx row limit of 1048576 exceeded for New. The first 1048576 features of New will be written to file. Ending translation ? Is there a way around to pass this ?
I would like to fill the holes or patch the gaps that are not really holes (see image), but in the output I don’t get any attributes. Why is that, and what is the best method? I try Areaamalgamator but ii cannot retrieve the attributs in output…thank you
Hi, I have the following csv sheet created with FMEThat is not exactly what I want. I would like to see the following format:The third column is empty, and for the last 4 columns only 1 row is needed. How can I achieve that? The values in the last 4 columns are purely inserted in the user interface at the start, not more than that.Below is the part of FME:The user parameters of the writer are as follows:Any tips?
Hi community!I’d like to know how to make the white background of this image transparent and change the black color of the geographic region to white. It’s important to keep the original resolution and export the file in Uint16 format.I’ve attached the image: could someone kindly apply this transformation and send me back the modified version?Thank you so much!
Hi,I do have a question for anyone that experience about extracted information from a website . I have a geodatabase that I set it up as a field with Domains.The project I am working on is Campground for my Forest here we have many campgrounds. I work on about 18 campground so far…My question is can I go to Recreation.gov site and select a campground using in FME ? For example, in the search box , type Colorado Campground and then it turns to new page and you will see the campground map that has all the Site number you can pick.So click on number three and then click View details...which will bring another page and it gives you all the information about that site number…That is what I plan to use this…I have done input all the information into my geodatabase there. Will HTMLExtractor work for this type of thing ? Am I making myself clear Thanks !
Hi everyone,I’m running into an issue where FME hangs while reading a DWG file using the ACAD reader. The translation process stops and doesn’t progress past the following message:while reading dwg file translation will be stopped by 'LogCount_func' Loaded module 'LogCount_func' from file 'C:\Program Files\FME\plugins/LogCount_func.dll' FME API version of module 'LogCount_func' matches current internal version (4.0 20240212) Additionally, when using a PythonCaller with arcpy, I can confirm that the Python Factory locale is being flipped from UTF-8 (German_Austria.utf8) to ANSI (German_Austria.1252):import locale print("Startup locale:", locale.getlocale()) import arcpy print("After arcpy:", locale.getlocale()) Output:Startup locale: ('German_Austria', 'utf8') After arcpy: ('German_Austria', '1252') This matches the warnings shown in the FME log:PythonFactory script changed the current FME process locale from 'German_Austria.utf8' to 'German_Austria.1252' and FME has restored it back to
Dear community, I am trying to activate a workspace when a survey123 form has been edited and the user confirms that all necessary information has been completed (yes/no field). The work space would then extract data from ArcGIS online feature layer of incidents and the relevant repeat tables. The extracted data would then be used to populate a MS word template which once complete would be saved/exported as a pdf document. This pdf document would then be attached to an email and sent to the manager of who filled in the survey123. My main problem is trying to find a way to fill in the relevant field in the template with the extracted information from the ArcGIS online layer. Is there a transformer or series of transformers which make this possible? Thanks in advance.