Using readers and writers to extract and load data.
Recently active
I have network based resource folder, that folder si on fme flow When file arrive in that folder, i need to write that file on other location, i just need help i don t know what parameter or what pointer should i send to my reader that it knows which file should write
Hi,I have a client that is experiencing some odd behavior when it comes to truncating feature classes and tables in a SQL Server SDE environment. The writer is set to 'Yes' for Truncate table however when fme writes out the features heading to the writer it does not truncate the data first but merely inserts data records.If i add a new reader, move the feature types over, configure it exactly the same and rerun the writer functions as expected (Truncate and Load data).Why do i have to add a new reader that is configured exactly the same to get FME to truncate properly. Do FME writers go stale?FME 2015.1SDE (10.4.1)Any help will be apprecaited.
Hello everyone,I was wondering if there is a simple way to filter one Dataset by comparing key-values with another Dataset. Basically a Join but onlky keeping Left Attributes.Example:Dataset1 Project SubID moreAttributes from Dataset1 ABC 1 ... ABC 2 ... DE 1 ... DE 2 ... Dataset 2: Project SubID different Attributes from Dataset2 ABC 1 ... ABC 1 ... ABC 3 ... ABC 3 ... DE 2 ... FG 3 ... I want to Filter Dataset 2 to only keep the ABC|1 and DE|2 rows because they match dataset 1. I am sure I could accomplish this via join and subsequently throw out all the Attributes that were introduced from dataset 1, but I was wondering if there is an easier way. Thanks all
Difference in feature count of Neighborhood Finder transformer after upgrading from FME 2020 to 2024. The count differs even though the input data and projection are same. Similarly for lineonlineoverlay transformer the tolerance 0 and automatic gives the same output while the expected was different as FME 2024 does not have an option to give “none” in tolerance.
Hi,I am converting SHP data to DGN using DGNV8 writer dynamically (via FeatureWriter). Everything works fine, but I don´t understand why it always uses ACAD writer first, throws this error and warning:AutoCAD Writer: An error occurred for dataset 'dgn\DI\vydej_di_L1.dgn', in function AutoCAD2004Writer::write. Unknown exception thrown. Aborting methodBADNEWS: ACAD writer: An error has occurred. Check the logfile above for details(D:\code\fme\source\engine\engine_api_impl\cpp\pluginbuilder\dynawch.cpp:129) class StatusInfo __cdecl DynaWriteChannel::putFeature(class STFFeature *&)DESIGN WRITER: Failed to write '3D elements' data through a proxy. Skipping such data for all elements writtenThen it writes everything using DGNV8 writer and the output DGN looks good, nothing seems to be missing. The only issue is that it throws quite big number of errors like this. I publish and use this workbench in FME Flow and the number of errors doesn´t look good in the finished jobs.Any idea how to a
Hi,I have a very basic workbench in FME 2022.2.8Reading features from a MSSQL table and writing to another, it is a FeatureReader/FeatureWriter combon in a dynamic workflow using the schema from the FeatureReader. This is using JDBC MSSQL Spatial.If the geometry column is set to EMPTY the writing fails with the message:MS SQL Server Spatial (JDBC) Writer: Error writing feature. Error: IFMEFeature.exportGeometryToOGCWKB: Unable to export geometry to OGC WKBExamining the feature in the database one can find it using SHAPE.STIsEmpty() = 1 and SHAPE.STAsText() returns “GEOMETRYCOLLECTION EMPTY”
Hi, It seems there is no easy way to do this from what I have read on this topic. I really wish for a way to extract coded domains from feature classes in an SDE geodatabase. I don’t even have to have the field names since I have named the domains after the featureclass, subtype and field it is attatched to. How can I get a hold of domain lists within FME? The point for me is to regularly be able to pull out lists of the options my users have in different fields since these need to be looked over now and then.
I have a flat table from a survey that is conducted by clients. I want to aggregate this data at the end of the month and send it to various staff that has had a survey submitted by our clients for their work. I’ve aggregated this data accordingly which produces a list. The aggregated results may differ based on how many surveys get submitted for each staff member that helps our clients. I have also generated a basic email template using the “HTMLReportGenerator” transformer. However, I’m having a hard time iterating through the list populate the HTMLReportGenerator. I’m looking to email each staff member their survey results from the month.Any help or guidance on this matter would be much appreciated. Thanks
I am trying to set up a Web Connection to an OpenID server using the HTTPCaller. According to the documentation there should be an option to set up a new Web Connectionhttps://docs.safe.com/fme/html/FME-Form-Documentation/FME-Transformers/Transformers/httpcaller.htmBut unfortunately I can not find the settings to create new web connections in the HTTPCaller, could someone please point me in the right direction?
Nevermind.I was deceived by a cookie. I was already loged in. Turns out the server required a login form.
Hello, I am set of 168 pipe segments with Global ID which is failing to write to database when using Esri ArcGIS Portal Feature Service. I am sure that the whole write process is failing due to 1 or 2 pipe segments. In order to debug this, Is it possible to write(commit) each pipe segment one by one to the database and figure out which pipe segment is causing the issue. Thanks,Naveen
I have a workspace in FME that reads data from a PostGIS database and updates a Feature Service in ArcGIS Online (AGOL). Since the PostGIS data is updated frequently, I use the ChangeDetector transformer to update only the modified features instead of overwriting the entire dataset (to avoid the "413 Request Entity Too Large" error).Currently, each time I run the workspace, I need to enter my AGOL credentials. I want a solution where anyone in my organization can run the workspace, even if the Feature Service is not in their My Content.What are the best ways to handle authentication in FME for AGOL updates?Is there a way to store credentials securely for automation? Can I use API keys or OAuth2 authentication? How do I allow others to update the Feature Service without adding it to their My Content?
I’m trying to upload a raster to Mapbox, but no matter what I do, the output file always turns the nodata cells to black. I have tried fixing this using RasterBandNodataSetter and RasterPaletteResolver. If I look at the feature cached output from those transformers the nodata cells are still transparent. The issue arises when I use the MBTILES writer. No matter what, it writes nodata as black. I also tried removing the nodata (RasterBandNodataRemover), doesn’t work. I have tried outputting the data both as png, png8, and jpeg. Does anyone have any other suggestions? Thank you in advance!
When using a FeatureReader transformer to read Snowflake spatial data, the following warning is posted into the log.Snowflake Spatial Reader: Skipping data in column '.GEOG' due to error (JDBC Type = VARCHAR, Vendor Type Name = GEOGRAPHY, Error: class java.lang.String cannot be cast to class [B (java.lang.String and [B are in module java.base of loader 'bootstrap'))The process does complete normally, but the geometry is not read automatically. Has anyone else encountered a problem like this? I’m using FME Form 2024.0.1.0, Build 24202.
Hi, I have scanned a map and saved it as a raster image. I need to compress this PDF using FME. I have tried RasterMappers and RasterResampler, but both are making my map smaller without actually reducing the file size. Do you have any recommendations, please?
Hi,A colleague and I are working on a script for a client. In the script, we have a simple custom transformer that performs a loop to match some attributes in the correct order. Its basically just a feature merger with a loop: If I run the entire workspace the loops get stuck and keep iterating forever. However if we run the workspace with for example "run to this" far down the stream, it has no problem and the custom transformer finishes after a few seconds.Does anyone have a clue why this is happening and what we could do to prevent it?Best regards
Hi FME Community,I have a folder containing 100 CityJSON files, each with three main features: Building, BuildingPart, and Metadata.I need my files to be dynamically processed by transformers to create new attributes and then output the modified CityJSON files into a new folder while keeping the original filenames.Do you have any tips on how to achieve this?Best regards,ImanePS: I tried using the reader to specify the folder path, but it doesn’t seem to be working perfectly.
I am trying to download from the Norweigan Surveys Inspire Atom feed using the latest version of the GeoRSS/RSS Feed [Reader]. Oddly the LinkURI does not get exposed by the reader. Looking at the XML source the links are present and are tagged <link></link> but for some reason they do not get interpreted correctly. Any ideas on how to solve this?Link to the feed: https://nedlasting.geonorge.no/geonorge/Tjenestefeed.xml
I’ve seen a few other posts discussing this or similar issues when it comes to writing data to tables in an Excel file, which is now affecting me on a current project. There is a similar problem in named ranges as well, although slightly less problematic. I went ahead and filed a support ticket but in the meantime, decided to post here in case someone knows of a solution.Every time data is written to tables and/or named ranges and a template file is used, the resulting file contains errors that Excel tries to fix upon opening the resulting file. If tables are involved and cells are referenced elsewhere, links to the first table end up broken as you can see below:Here is the resulting error upon opening the Excel file: Data seems to be written as expected, but having to click through an error when opening the Excel file, plus the potential of losing links to table data, is not a good thing. An example workspace template is attached so you can play around and test on your end (in version
I use Fme Form 2024.2.2. If I remember correctly, I used to convert multiple files in one process with an earlier versjon of Fme Form.I have attached a file showing how I structured the fmw for to separate files and for multiple files.I need to convert more than 100 files from Quadri format used in Norway to the GDB format.When I convert two files separately, everything is OK.The sise of gdb files are: Rp2 5,90 MbKdp3 2,62 MBBut when I convert to files in one process something is going wrong. The size of files are:Rp2 – 5,90 MBKdp3 – 154 kBI have tried different combinations of the order of files and numbers of files.It seems to me the first file is always Ok, but it fails from the second one onward.I appreciate all kind of helpEla
HelloHere’s my challenge:I have two datasets that I need to compare after several transformations.- Dataset 1: This contains georeferenced points with distances associated with a linear reference system (LRS).- Dataset 2: This has no geometry, but contains distances from the same LRS.My aim is to associate each value km_start in dataset 2 with the closest value _measures in dataset 1, even if they don't match exactly (I’d like to be able to assigne a tolerance, for ex. max difference of 0.01m). Thanks to this matching, I will be able to assign a geometry to the data in the second set. Below are a few screenshots to illustrate my question: Dataset1 with georeferenced points, _measures are distances in the LRSDataset2 with non geometry features I want to get the closest match from Dataset1 We're talking about fairly important datasets but it shouldn't be too much of a problem if the process takes time.Thank you in advance for your help and ideas.
Hi FME world,I have received a series of NetCDF files (representing rain radar) and I have no information on the projection for them, other than they are supposed to be in New Zealand and maybe Lambert Conformal? I can represent in an app like Panoply (image below) but when I try to project in FME, NZTM or LL84 is incorrect. I have tried creating a custom coordinate system but this is no use either. I’ve included a view of the Feature Information with the exposed list as per this conversation NetCDF - no CRS desposited | Community.Would love some direction if anyone has any thoughts?! Thanks!
I suspect this might be an ESRI thing rather than an FME thing, but is it expected behaviour that if a feature is written to a table which has a non-nullable long field that field gets populated as zero if an attribute is missing? No error, or warning?
i use pyrhon create FMETriangleStrip, how i add normal list for FMETriangleStrip? thanks
I have a workspace where the user should be able to export a DEM grid in several different fileformats (a user parameter for the output filetype). Then I have a FeatureWriter for the output. It works well if I want to output a raster DEM (e.g. a GeoTIFF), but if I want to output the data as a DWG with a point grid, I get several thousands of Features exported.Of course, I could make a workspace with a lot of different filetype Writers instead, but a single Feature Writer would be nicer. Is there a way to consolidate all the thousands of DWG points into one DWG? I dont really want a DEM consisting of one multipoint either.