Using readers and writers to extract and load data.
Recently active
Hello,I am trying to string together attributes. The difficulty is that the attribute names can vary by dataset.Generally what i am trying to do is that may look something like this:DEPTH [m] Measurement 1 Measurement 2 Measurement 3 Measurement 4 0.5 <NULL> 1 4.5 2.1 1 <NULL> 2 2.3 2.2 1.5 3.5 1 1 2.1 2 3.0 1 4.5 <NULL> What I am trying to accomplish is something like this DEPTH [m] Measurement1 DEPTH [m] Measurement2 DEPTH [m] Measurement3 DEPTH [m] Measurement4 <NULL> 0.5 1 0.5 4.5 0.5 2.1 <NULL> 1 2 1 2.3 1 2.2 1.5 3.5 1.5 1 1.5 1 1.5 2.1 2 3.0 2 1 2 4.5 <NULL> Depth and measurement values can be tabstop, space ore comma-seperated, doesnt really matter. In a later step I want to route all Measurements into different features using the AttributeExploder and Aggregator while still keeping the depth information. I realise this could be easily done using the AttributeMa
Hi ,I'm building a workspace in FME where I need to copy multiple tables from an SDE geodatabase to a File Geodatabase (GDB). I use an Excel file to map the source and destination table names.Excel structure:Tablename Destination SDE_TABLE_1 GDB_TABLE_1 SDE_TABLE_2 GDB_TABLE_2 Workflow: I read the Excel file using a FeatureReader to get the list of tables. For each row, I use another FeatureReader to read the source table from the SDE using the Tablename attribute. I then use a FeatureWriter, setting fme_feature_type to the value of the Destination attribute so it writes to the correct table name. The problem:The destination tables are created with the correct names in the GDB, but they do not contain any attributes from the source tables only OBJECTIDs. What am I missing here? My tables are in Excel and will be dynamic, so I can't simply explode all attributes manually.I have a solution using a PythonCaller, but I'm interested in a more native solution using FME transfo
Is there a way with an ArcSDE Reader with an archived clause to expose the attributes GDB_FROM_DATE and GDB_TO_DATE ?Thanks
Hello,I need to make a model but I honestly don't really know how to start it. I will try to explain.I need to create groups of “municipalities” whose data should be published together. I already have a list of which municipalities is intertwined with which municipality based on geometric overrides but the same municipalities may be intertwined with another municipality, which in turn may also be intertwined with another etc....Eg Municipality 1 should be charged together with municipality 3 and municipality 4. I have this list, but then I also have to search in the same list if municipality 3 should also be charged with other municipalities, in this case yes, together with municipality 6 and municipality 7. Then I have to do the same for borough 4 and borough 6 and borough 7 and so on until I have everything and can then make it 1 group. Is this clear? Can somebody help me with this model or give me a tip how to start it. I attach a sample of my starting table “Samen publiceren”I alre
I wonder if anyone knows a knows what would be a good format to write out solid geometry. I have worked with filegeodatabase but that is multipatch. I need spatial indexing to be able to use a spatial intersect with a feature reader. So a big dataset with solids where we often only need a few of.The multipatch feature class (FGDB) kind of works but then I need a solidbuilder and that takes a lot of time.Thank you!
Hi FME Community,Our team is looking to connect with users who have working experience with the Protobuf reader. If you’ve used it in your workflows or projects, we’d love to hear about your experience!We’re especially interested in:How you’re using the reader in real-world projects Workflow examples or use cases Performance insights Any challenges or suggestions you may haveIf you're open to sharing your thoughts - or even jumping on a quick call - please comment below or send me a direct message.About ProtobufProtobuf is a compact, open-source format developed by Google for serializing structured data. It is widely used in APIs, messaging systems, and IoT applications where efficiency and portability are important.The new reader supports .pb (binary) and .json data files defined by .proto schemas. It works with Proto2, Proto3, and Editions 2023. Geometry is not supported.Available in build: FME Flow 2025.1 (Build 25587+)
Hello everyone! I'm a novice in FME Form/Flow. Maybe somebody can help me.I have a workspace that has multiple PostgresSQL readers, and these readers have a reference to the User Parameter in the WHERE clause.This workspace is published to FME Flow and triggered by an Amazon SQS trigger. The SQS message content is a JSON with parameters that are required for the readers in the workspace.My question is, how do I pass/parse incoming content to separate parameters in the workspace?
I have generated a raster that has integer data values between 1 and 6 and needs a nodata value. As the RasterInterpretationCoercer has a “Data Interpretation Type” of “Int8” I thought my task would be as simple as:Use the “RasterBandNodataSetter” to set the Nodata Value of -128. Use the “RasterInterprationCoercier” with the Data Interpreation Type of “Int8”. Use the GEOTIFF writer to output a tif file.When I try this type of workflow and then use FME to read the generated tif file, I find that the raster band is written with the type Uint8 and that my data value has been changed to 128. I am using FME 2025.0.3.Is it possible to get the GEOTIFF writer to write raster data with the datatype of int8 and a negative Nodata value? If not, which raster types in FME will allow this?
Hello,I would like to know if the IfcZone are managed with the version 2024?Best regardsBruno
Dear FME Community,I would like to use the WorskspaceRunner for optimize the performances of my treatments, but I’ve difficulties to configure them. More precisely, I don’t how can I configure the reader/writer of child workbench, whereas I try to do same things like this tutorial.In this example, I try to have in input, a filter on limited objects, but in output I’ve all objects.This is my main workbench :And this is my child workbench : How can I can I do ? Thank you very muche.
Hi, I have and issue trying to connect to snowflake DB using keypair authentication, it is not recognising my schema.I'm using FME(R) 2024.1.2.1 (20240906 - Build 24624 - WIN64). I can connect fine in DBeaver using the keypairs. I'm using the advance JDBC settings to add the parameters in a new snowflake database connection.The error I get is ' COM.safe.fme.jdbc.TranslationFailureException: java.lang.IllegalStateException: Connection established with no current schema. Make sure that schema 'dbo' specified in connection settings exists and is accessible by the user ‘.I know you need to have the case correct in the parameter I have checked with the DBA and have been told that the schema is definitely lowercase. I can’t show all the jdbc connection string, but this is the schema bit ‘ schema=dbo ’ so it is passing the lowercase parameter. Again, I am using the lowercase in DBeaver and connecting fine. I’ve also tried the scheme DBO, “dbo”, “DBO”. There is another schema in the database
anyone know why Im getting these errors. Its an export to excel from gogov. I think the export is a bad file format?Excel Reader: Failed to open the dataset 'R:\Rubicon FME\GoGOV\Input\**\*.xlsx'. Error message was 'File does not exist'Excel Reader: Failed to open the dataset 'R:\Rubicon FME\GoGOV\Input\*.xlsm'. Error message was 'File does not exist'Excel Reader: Failed to open the dataset 'R:\Rubicon FME\GoGOV\Input\report (17).xls'. Error message was 'It is not a xls file. Use the xlCreateXMLBook() for a xlsx file.'Failed to obtain any schemas from reader 'XLSXR' from 3 datasets. This may be due to invalid datasets or format accessibility issues due to licensing, dependencies, or module loading. See logfile for more informationWorkspace generation failed. Parameter file was C:\Users\pearly\AppData\Local\Temp\FME_1750261216546_26960.
Hi, I use a DGNStyler to set the color of line features.Color is set by RGB Color (Color Type = RGB). In the resulting DGN, the color of the line is replaced by the best matching color from the color table. Result (Color 59):Desired Result (204,0,0):Is there a way to get the desired result? Jasper
I’m using an Oracle non spatial reader in FME Desktop 2019.1.3.1 to read an Oracle view. Using Oracle client (i.e. SQL+), the total count of this view is 6,982,320, but the FME Oracle reader reads in over a million less, and the total count differs each time I run it. I’ve played around with some of the reader parameters - bulk read size, persistent connection, etc. - but each run is still different. Two examples below which read in two different total numbers and while they’re close, they’re way off the total count according to SQL. Any thoughts on this?
Can I please have some help, I have list of attributes as example:attribute{}.name = ID and attribute{}.native_data_type =longhow can I use jsontempleter to have as output to json ID(long)? Thank you,M
Dear FME Community,I should to optimize my job, and for that I need to make group processing with an optimization of objects numbers like that :And after, as you can see, make a parallelize of objets for not having only a linear treatment. Do you know how can I do this ? Thank you very much.
Why do I get this error message upon opening the FME generated excel file? Based on the observation, the records are corrupted in some instances with missing values in the features. I have attached the screenshot for the reference.
I have encountered a strange issue where FME doesn’t recognise a cell named ‘1’ in any of the cell files I have tried using it with.This arises both using the DGNStyler, where the cell is missing from the pick-list (and it’s the only one missing),and also by applying the idgs_cell_name format attribute, where the dgn writer gives the warning that the cell could not be found in the cell file and it will be dropped from the output.This issue is in the 2022.2.4 FME Workbench currently our workplace standard, but NOT in an older 2017.1 installation.Any suggestions would be appreciated.
Hi, I have a bunch of workbenches which routinely read layers from our AGOL environment. There are multiple readers in the same workbench, all reading different layers. Until recently everything was doing fine, but now one reader is throwing an error. See part of the log here:ArcGIS Online Feature Service Reader: Querying layer/table 8 for object IDs, where '1=1'ArcGIS Online Feature Service Reader: Got 731 results for layer/table ID 8ArcGIS Online Feature Service Reader: Splitting reading of 731 features into 1 batches of up to 1000 featuresArcGIS Online Feature Service Reader: Getting feature results 0 up to 1000 of layer/table 'tekentools polygon lopend'ArcGIS Online Feature Service Reader: Encountered an unexpected error. The error code from the server was '400' and the message was: 'Cannot perform query. Invalid query parameters.'. Details: 'Unable to perform query. Please check your parameters.' I checked the source layer in AGOL and nothing has changed for the last 5 years, so t
I have 4 classes that I read from ArcGIS Online, reformat and write out to new classes, all are small and to create each output I've set "create if needed" to yes on the writer and each class is set to "use existing". All 4 classes are created. I then set the model to "create if needed" to no and each class to truncate existing. 3 of the classes update. the 4th with 55 rows truncates existing but then the writing times out with an HTTP 504 error. I've tried setting the features per request to 1 and that times out also. If I delete the class in AGOL I can still create it from the mode but i can't update it.
Hello,I have an FME 2018.1 tool that updates an ArcGIS Online-hosted feature service (Writer Mode: INSERT, Truncate First: YES). The feature service has about 15,000 parcel polygons. Typical changes: a dozen or so new polygons per month, a dozen to a hundred attribute table changes per month.The tool/transformers all work when attempted on a local geodatabase feature class. Therefore I am able to rule out transformer problems. But when the tool runs on AGOL-hosted feature services sometimes it works. Other times it fails and returns these various errors: - The error code from the server was '504' and the message was: 'Your request has timed out.'- The app domain with specified version id (xxxx) was unloaded due to memory pressure and could not be found- Could not connect to server. Check that you are connected to the internet and that no firewall is blocking outbound connections. The error was '('Connection aborted.', error(10053, 'An established connect
Hello,So we have some workbenches (FME 2018.1.1; 20181203 - Build 18578 - WIN64) that either read or write to AGOL after performing a bunch of operations on the data. These workbenches were working fine up until a week ago, when all of a sudden they started throwing a very generic 503 error message and failing completely.If I understand correctly, a 503 means it's a server side error (so it's likely something with ESRI), right?We've been having issues with our organization's ArcGIS Online portal for the last couple of days (layers being randomly dropped from maps, apps refusing to load) and I'm starting to think that their December 2018 AGOL update messed some things up, including the way FME's readers/writers communicate with it. Is anyone else having trouble working with AGOL lately?Thanks!
I’m working on a flood-affected buildings project in St. Pauli, Hamburg, using FME’s SpatialFilter transformer. I uploaded an image of my setup (see attached). Almost all buildings are being rejected, and I’m not sure why. My buildings are the "generic object" and flooding areas are the clipped ones
I’m hoping to use a DTM service provided by the Environment Agency to extract the height for locations I have (points or polygons I haven’t decided yet). They provide WMS, WCS, WMTS, OGI API: LIDAR Composite Digital Terrain Model (DTM) - 1mI believe it’s the WMTS I want, it has the elevation data layer within it. https://environment.data.gov.uk/spatialdata/lidar-composite-digital-terrain-model-dtm-1m/wmts?request=GetCapabilities&service=WMTS&version=2.0.1My plan was to use a WMTS reader, and the PointOnRasterValueExtractor to get the heights for the points I supply.I can’t even seem to read in the WMTS data, because I’m not putting in the correct parameters?I’m not sure what format I need, or if it even matters for what I’m doing – vnd.jpeg-png8 seems to be the most versatile so that’s what I’ve been trying? Not sure what the Tile Matrix means? The different zoom levels, but I don’t know how I’d know what I need? I obviously don’t want to read in the entire WMTS service, so I
Hi, I am trying to run a custom python code that uses QGIS library in PythonCreator. However when I run the code I get this error:Python Exception <ImportError>: DLL load failed while importing _core: The specified procedure could not be found.I tried everything, I followed the documentation/manual how to setup custom python interpreter, I tried to setup dependencies and environment variables but none of that worked. Can someone tell me what am I doing wrong? Thanks!