Data

Hi guy’s A workbench that worked before gives me some crazy errors that’s driving me nuts.I used the workbench before and it worked normally.  It has to write some geotif tiles with deflate compression. No matter what I do, I always get errors about me using jpeg compression. I tried everything. Clearing cache, deleting cache manually, restarting pc, copying everything in new FME file, updating all the transformers, making a new wb, try running it on another pc with more recent version of FME... I had a similar question in the past and than the solution was to use a bulkattributeremover with ‘geotif_’. This doesn’t work either. This is the wb, it’s just shapefiles and geotifs as import: Any one with ideas?

I’m using Pro 3.2 and Data Interop extension with a simple workspace to pull point features from a SQL server database and publish them as a layer in AGOL, all within AWS on an ec2 instance. I have this scheduled to run every 15 minutes so users get fresh data. The workspace also uses an HTTP call to make the layer editable just before overwriting it, and then a second one after writing, to disable editing, because the layer is public. Seemingly intermittently, the file geodatabase used to publish the layer will not get deleted. I have the fgdb that was originally used to publish the layer still there in the same folder as the layer, but every so often another fgdb will be created and left in that folder too, so I get duplicates. Some mornings I check and there are 3-5 or more of them. I can manually delete them, but I’m trying to determine why they’re not getting deleted in the first place. I’ve checked the logs and can’t see anything out of the ordinary. I tried adding a 30-second De

I have a non-spatial table with about 5 million records of XY coordinates. I need to update the table with the Lat Long values in the so-named columns.I am facing problems with writing these to the table. When I choose UPDATE/UPSERT/fme_db_operation, the writing takes a seconf for every 10 records. It’s impossibly slow.When I choose, Insert with DROP and CREATE or TRUNCATE EXISTING, the script gets stuck with following last 4 lines in the log: Writer `MSSQL_ADO_2' of type `MSSQL_ADO' using group definition keyword `MSSQL_ADO_2_DEF'Microsoft SQL Server Non-Spatial Writer: Opening `GISReporting' for write operationMicrosoft SQL Server Non-Spatial Writer: Read 1 DEF line(s).  Found 9 attribute(s)Microsoft SQL Server Non-Spatial Writer: Trying to connect using Microsoft OLE DB Driver 19 for SQL Server...MSSQL_ADO writer: Splitting bulk features into individual features  NOTE: I have set BULK Read and Write at 100000 records for every transaction.

I’m having trouble writing dynamically to a SDE feature dataset using the ‘truncate’ option in FME 2023.1. The features are written, but there is no ‘truncating’ taking place before the writing and instead the data doubles up... It works as expected in FME 2021.2.6 however, maybe due to the different way in which the feature dataset is specified in this version under ‘Table Creation Parameters’. How do I make it truncate also in 2023.1 version? Current set up in writer for FME 2023.1: 

Hi all,i know it will maybe an easy one….I’ve tried with substringextractor or with AttributeTrimmer as well but i couldnt reach the right result.i have attribute like these for example:he0_wasser_WSG_TK_20143_AGE_f_r01_231030 orni0_geobasisdaten_gemischte_industrie_gewerbe_flaechen_20256_AGE_f_r01_240118i have to split in this case “he0” and “ni0” and write that in an attribute xi have to split “wasser” and “geobasisdaten” and write that in an attribute yi have to split “AGE” in an attribute…….and so onnow comes the challenge:the text between “wasser” or “geobasidaten” and the 5digit number (20143, 20256) has sometimes 2 words, sometimes 3 or 5 words…..but i have to write the whole text in an attribute z….ho can i do that? i am not so well in regexp…..so i ask here for help Thank youCheersFranco  

Hello everyone,For one year, I’ve been running different workbenches daily (Windows Scheduler), reading from MSSQL tables to write into various hosted feature layers, used in various dashboards. I noticed an unusual consumption of the storage on our GIS server, coming from the “pgdata” folder of Arcgis Entreprise, so the underlying postgresql database (up to 20go additional Gb per month, while the size of all my hosted features stay constant at around 1gb only in total). After several tehnical sessions with ESRI, there is no doubts that this is caused by my FME workflows. Those workbenches are far to be optimised (as I was only starting to use FME) but they are working fine. It was easy to truncate all features to insert new features, sometimes up to 150 000 features daily, for one layer (see below, simplified version). The inserted features are polygons, from the same source hosted feature layer, daily joined with most recent MSSQL data. NB: Objectid is removed with “attribute manager

Hello , i have too many points I would like to find the points and lines between start of path and end of path in different paths ?i would like to have excel file that include First row Start point (a) - next point (b)- lines (line id 1)Second row From b point - next point C - lines (line id 2 )and so on till end point (blue point )fllow direction from start point to end point as picture fme 2021 thanks in advance 

I have followed this tutorial:Updating ArcGIS Online Feature Services using UPSERT (safe.com)As the last screenshot shows, I have 1495 records. Comparing the original CSV used to do the initial data load and the updated CSV, the new records in the updated CSV have not been added and existing records with changed attributes have not been updated.The tutorial suggests this is possible using the UPSERT operation. Example update I expected with the highlighted text the changed attribute value: 1000,ZYX3116631180,-123.093432,49.281221,31180,-53.7761,-21.7304,1,2011-12-02 22:01 Example add I expected: 9000,ABC1069610710,-123.093414,49.281637,10710,-79.377,-17.3218,1,2021-12-06 1:41

I am creating a workbench where I will bring spatial object assets from an oracle database into a file geodatabase. I want to sort the assets into feature datasets as I go but I am confused as to how to go about this. I have only done a few FME workbenches so I don’t have a great deal of experience. So far, when I have been creating File Geodatabases, I add a Esri Geodatabase (Open API) writer and then the file geodatabase is created and any features I connect to that writer are added in. So its kind of a two for one where the file geodatabase is created plus my features are converted and added to it. How would I go about creating a feature dataset during this process? Is it possible with FME 2018 or does that come with newer versions. We will not be upgrading for several months and my co-op term will be over by that time. Thank you!

Hi fellow FME enthousiasts,Another case i'm struggling with today.I have a list of parcels that need to be added into a Excel template. In most cases I can sort the parcels by parcelnumber and concatenate them grouped by the Section number and write them to the Excel.But the field in the template can fit a maximum of 5 parcel numbers. So if there are more then 5 parcels they need to be split in groups of 5. So say i have a file with 13 parcels, I would need to split them into 3 groups (2 of 5 parcels and 1 of 3 parcels). I can get the number of parcels with a StatisticsCalculator, but i struggle with the part where i have to split them into groups to write to the Excel. How would you do this? Cheers!

The steps I have so far: 2 layersSpatialRelator, which generates the list attributes with the intersects.Then I test the output to find out which features have more than 1 relation (indicating overlapping with more than 1 feature).Then I explode that list and delete the unimportant attributes with AttributeManager. I now have a table like this. First column containing a unique identifier for Layer 1, the second row containing the unique identifier for Layer 2.  A feature of Layer 1 can overlap with multiple features of Layer 2, and vica verca, hence the duplicates on both sides.Layer 1  Layer 2 ID1 123 ID1 345 ID2 987 ID3 456 ID3 123  Now the hard part. I can FeatureJoiner them together, and then dissolve, but then I miss a lot of features and don't dissolve enough overlaps. It doesnt create the perfect output.What I want to achieve: I want all features of Layer 2 to dissolve together, based on the overlaps with features from Layer 2.I added a drawing to visualise wha

Hello fellow FME enthousiasts,I have a case which i can't seem to find a solution for.We have a process that adds a score to incoming records based on certifications added as hyperlinks in a table.During several steps these certifications have to be manually checked by an employee. Finally when everything has been approved the records are written to a central database (this can only be done when all the records have been approved, so the whole dataset has to be kept together during the whole process). Right now i have the process setup to output all data to an Excel table (with a template to make it readable for the employee). Here the employee can select if a certification is approved or not. This Excel table is then put back into the next process untill all necessary checks have been done.Each time a proces is run i would like to check at the end if all records are approved or not. Depending on the outcome two things can happenIf ANY are not approved, ALL records needs to be output t

Hi All,I need to run a join between an excel file to a feature class (GDB) with attachments by an unique ID. The GDB has been used to collect data within a Survey123 form.I did some testing but can’t retain the attachments in the final output,Any idea how to retain the attachments after running the join?

I am building a workbench that uses spatial filters, I have several readers input to the spatial filters that have different source coordinate systems. Originally, I used the ESRI reprojector to convert everything to 3TM (EPSG: 3776) prior to the spatial filters. This included a file with a source coordinate system of WGS84 Spherical Mercator (EPSG: 3857). However, my end result is about 4 feet off of where it should be. I tried selecting a transformation (I used Pro to see which transformation they used when I used the project tool and selected that one) but the result was still the same. I performed the projection in Pro on the same file and it was successful so it was a bit confusing as to what I was missing in FME. I decided to instead project my other files to WGS84 Spherical Mercator and work within that coordinates system. It was successful. I am guessing I may be using the ESRI reprojector incorrectly, or missing an input. 

I’m reading a geotiff file with embedded scaling and offset,  snippet from gdalinfo:Band 1 Block=1024x1024 Type=Byte, ColorInterp=Red  NoData Value=255  Overviews: 6000x6000, 3000x3000, 1500x1500, 750x750  Offset: -1,   Scale:0.008Band 2 Block=1024x1024 Type=Byte, ColorInterp=Green  NoData Value=255  Overviews: 6000x6000, 3000x3000, 1500x1500, 750x750  Offset: -1,   Scale:0.008Band 3 Block=1024x1024 Type=Byte, ColorInterp=Blue  NoData Value=255  Overviews: 6000x6000, 3000x3000, 1500x1500, 750x750  Offset: -1,   Scale:0.008When working with the data in FME just the the 8-bit value is shown (0-255) when I was expecting values between -1 and 1.   In my particular use case the scaling is constant and I can just apply a correction, but I was wondering if there is a setting I’m missing to have FME apply the scaling/offset automatically.

I am trying to update approximately 10 attribute fields in a point layer in AGOL using FME.  When I run the script, it keeps failing and I’m not sure where the hang-up is based on the warning and error messages I am getting.  Below is what I keep getting once my script has failed, with the ones in red being the error messages (the last two bullet points):  Feature Caching is ON The workspace may run slower because features are being output and recorded on all output ports regardless of whether or not there is a connection. ArcGIS Online Feature Service Reader: Reading from from feature service 'https://services1.arcgis.com/9yy6msODkIBzkUXU/arcgis/rest/services/HII_2020_2022/FeatureServer' using filters defined in 'HII_2020_2022' ArcGIS Online Feature Service Writer: Feature Service 'HII_2020_2022' does not support geometry updates. If features contain geometry, the geometry will be ignored ArcGIS Online Feature Service Writer: Attribute 'OBJECTID' on feature type 'HII_2020_2022' is mar

Hi,I am hoping someone can convert a Geodatabase provided by the United Nations (Unitar & Unosat).It is published under creative commons 4.0.The map shows each building damaged in Gaza.I do not have any product that opens Geodatabsae files.I would be appreciative if someone could convert the file into KML so I can open the file in Google Earth.The zipped geodatabase file is 2.35Mb.The link is the location of the Geodatase file. https://data.humdata.org/dataset/unosat-gaza-strip-comprehensive-building-housing-unit-damage-assessment-march-2024#I have also attached two images.The first image shows the page with the GeodatabaseThe second image shows how the UN have used the file to present to the public.I have submitted a request to UNTAR however they have not responded.ThanksHarry Sullivan 

Hello FME Experts,I'm currently working with road network spatial data, which contains a multitude of line features representing roads. Each road feature intersects with others, and I need to determine the correct start and end names for each road segment (i.e., the names of the roads at the start and end nodes).I'm wondering if there's an efficient way to accomplish this using FME. Specifically, I'm looking for guidance on:How to identify the start and end nodes of each road segment. How to extract the names associated with these nodes.Any suggestions, workflows, or examples would be immensely helpful. Thank you for your insights!Best regards,Vishnu N V

Hi,I am creating some admin areas which feature multiple islands within each admin areas.I am using the Aggregator to group the areas, and that is looking fine in the Inspector.When I come to write these features to an esri ArcGIS Portal Feature Service, I receive the following error:<type 'fmeobjects.FMEAggregate'> cannot be converted to esriGeometryPolygonI am building Homogeneous Collections where possible, but even the admin areas that are made up of a single contiguous area have "fme_aggregate" as "the fme_geometry(string)" attribute in the inspector.

I am trying to bring in some files that are dgn/bentley related (cit and nm1) and I can’t figure out how to read these files or view them. I found in a search this site that says how to read a .cit file but I can’t find this or how to view these?https://docs.safe.com/fme/html/FME-Form-Documentation/FME-ReadersWriters/ingr/ingr.htmI do not see this INGR in my lists or anything when I put in reader format .cit 

Hi all,i have following problemi join a gdb (with included domains) with an excel sheet…...if i don’t resolve the domains the feature class is written but no geometryif i resolve the domains i have geometries right but no domains anymore Can somebody help me? Thx Franco

Hello,I have a workbench, which when published to FME Server - is not updating the data, rather - I am getting the cached data.  Is there any way that when I publish the workbench to FME server- can I use the option of “Rerun Entire Workspace” rather than “Run Workspace”. Please find the below snapshot of the options in workbench :    Also, want to mention that - when I did ran the workbench in FME Desktop - it updates the data only when I use the option “Rerun Entire Workspace”, but when I use the option “Run Workspace” - I am seeing the cached data. Please let me know If I am missing anything.

Hi all,i’m building an FME desktop workspace for eventual use with a server automation. we’re taking some emails, parsing them for some specific data, reading text files with additional information, and then putting it all together to be sent out via an API for a software called Desk Alerts.Where i am running into issues is - the text files are UTF-8 encoded and contain line feeds and tabs that the API isn’t happy with.so for instance i might have a message that looks like: now you can see i’ve got 4 tabs and almost 20 line feeds.this text is actually the combined result of several files and the content is variable so for readability’s sake maintaining the line feeds (and preferably the tabs) is paramount.When sending to the API, i get the following error response: {"status":"Error","errorMessage":"Request model invalid","data":{"validationErrors":["[queryParam] : The queryParam field is required.","[$.body] : '0x0A' is invalid within a JSON string. The string should be correctly escap

Hello Experts, I have two Layers PD and AA. I want to align the PD layer to AA layer. Any help would be highly appreciated. Thanks in advance