Using readers and writers to extract and load data.
Recently active
I have followed this tutorial:Updating ArcGIS Online Feature Services using UPSERT (safe.com)As the last screenshot shows, I have 1495 records. Comparing the original CSV used to do the initial data load and the updated CSV, the new records in the updated CSV have not been added and existing records with changed attributes have not been updated.The tutorial suggests this is possible using the UPSERT operation. Example update I expected with the highlighted text the changed attribute value: 1000,ZYX3116631180,-123.093432,49.281221,31180,-53.7761,-21.7304,1,2011-12-02 22:01 Example add I expected: 9000,ABC1069610710,-123.093414,49.281637,10710,-79.377,-17.3218,1,2021-12-06 1:41
I am creating a workbench where I will bring spatial object assets from an oracle database into a file geodatabase. I want to sort the assets into feature datasets as I go but I am confused as to how to go about this. I have only done a few FME workbenches so I don’t have a great deal of experience. So far, when I have been creating File Geodatabases, I add a Esri Geodatabase (Open API) writer and then the file geodatabase is created and any features I connect to that writer are added in. So its kind of a two for one where the file geodatabase is created plus my features are converted and added to it. How would I go about creating a feature dataset during this process? Is it possible with FME 2018 or does that come with newer versions. We will not be upgrading for several months and my co-op term will be over by that time. Thank you!
Hi fellow FME enthousiasts,Another case i'm struggling with today.I have a list of parcels that need to be added into a Excel template. In most cases I can sort the parcels by parcelnumber and concatenate them grouped by the Section number and write them to the Excel.But the field in the template can fit a maximum of 5 parcel numbers. So if there are more then 5 parcels they need to be split in groups of 5. So say i have a file with 13 parcels, I would need to split them into 3 groups (2 of 5 parcels and 1 of 3 parcels). I can get the number of parcels with a StatisticsCalculator, but i struggle with the part where i have to split them into groups to write to the Excel. How would you do this? Cheers!
The steps I have so far: 2 layersSpatialRelator, which generates the list attributes with the intersects.Then I test the output to find out which features have more than 1 relation (indicating overlapping with more than 1 feature).Then I explode that list and delete the unimportant attributes with AttributeManager. I now have a table like this. First column containing a unique identifier for Layer 1, the second row containing the unique identifier for Layer 2. A feature of Layer 1 can overlap with multiple features of Layer 2, and vica verca, hence the duplicates on both sides.Layer 1 Layer 2 ID1 123 ID1 345 ID2 987 ID3 456 ID3 123 Now the hard part. I can FeatureJoiner them together, and then dissolve, but then I miss a lot of features and don't dissolve enough overlaps. It doesnt create the perfect output.What I want to achieve: I want all features of Layer 2 to dissolve together, based on the overlaps with features from Layer 2.I added a drawing to visualise wha
Hello fellow FME enthousiasts,I have a case which i can't seem to find a solution for.We have a process that adds a score to incoming records based on certifications added as hyperlinks in a table.During several steps these certifications have to be manually checked by an employee. Finally when everything has been approved the records are written to a central database (this can only be done when all the records have been approved, so the whole dataset has to be kept together during the whole process). Right now i have the process setup to output all data to an Excel table (with a template to make it readable for the employee). Here the employee can select if a certification is approved or not. This Excel table is then put back into the next process untill all necessary checks have been done.Each time a proces is run i would like to check at the end if all records are approved or not. Depending on the outcome two things can happenIf ANY are not approved, ALL records needs to be output t
Hi All,I need to run a join between an excel file to a feature class (GDB) with attachments by an unique ID. The GDB has been used to collect data within a Survey123 form.I did some testing but can’t retain the attachments in the final output,Any idea how to retain the attachments after running the join?
I am building a workbench that uses spatial filters, I have several readers input to the spatial filters that have different source coordinate systems. Originally, I used the ESRI reprojector to convert everything to 3TM (EPSG: 3776) prior to the spatial filters. This included a file with a source coordinate system of WGS84 Spherical Mercator (EPSG: 3857). However, my end result is about 4 feet off of where it should be. I tried selecting a transformation (I used Pro to see which transformation they used when I used the project tool and selected that one) but the result was still the same. I performed the projection in Pro on the same file and it was successful so it was a bit confusing as to what I was missing in FME. I decided to instead project my other files to WGS84 Spherical Mercator and work within that coordinates system. It was successful. I am guessing I may be using the ESRI reprojector incorrectly, or missing an input.
I’m reading a geotiff file with embedded scaling and offset, snippet from gdalinfo:Band 1 Block=1024x1024 Type=Byte, ColorInterp=Red NoData Value=255 Overviews: 6000x6000, 3000x3000, 1500x1500, 750x750 Offset: -1, Scale:0.008Band 2 Block=1024x1024 Type=Byte, ColorInterp=Green NoData Value=255 Overviews: 6000x6000, 3000x3000, 1500x1500, 750x750 Offset: -1, Scale:0.008Band 3 Block=1024x1024 Type=Byte, ColorInterp=Blue NoData Value=255 Overviews: 6000x6000, 3000x3000, 1500x1500, 750x750 Offset: -1, Scale:0.008When working with the data in FME just the the 8-bit value is shown (0-255) when I was expecting values between -1 and 1. In my particular use case the scaling is constant and I can just apply a correction, but I was wondering if there is a setting I’m missing to have FME apply the scaling/offset automatically.
I am trying to update approximately 10 attribute fields in a point layer in AGOL using FME. When I run the script, it keeps failing and I’m not sure where the hang-up is based on the warning and error messages I am getting. Below is what I keep getting once my script has failed, with the ones in red being the error messages (the last two bullet points): Feature Caching is ON The workspace may run slower because features are being output and recorded on all output ports regardless of whether or not there is a connection. ArcGIS Online Feature Service Reader: Reading from from feature service 'https://services1.arcgis.com/9yy6msODkIBzkUXU/arcgis/rest/services/HII_2020_2022/FeatureServer' using filters defined in 'HII_2020_2022' ArcGIS Online Feature Service Writer: Feature Service 'HII_2020_2022' does not support geometry updates. If features contain geometry, the geometry will be ignored ArcGIS Online Feature Service Writer: Attribute 'OBJECTID' on feature type 'HII_2020_2022' is mar
Hi,I am hoping someone can convert a Geodatabase provided by the United Nations (Unitar & Unosat).It is published under creative commons 4.0.The map shows each building damaged in Gaza.I do not have any product that opens Geodatabsae files.I would be appreciative if someone could convert the file into KML so I can open the file in Google Earth.The zipped geodatabase file is 2.35Mb.The link is the location of the Geodatase file. https://data.humdata.org/dataset/unosat-gaza-strip-comprehensive-building-housing-unit-damage-assessment-march-2024#I have also attached two images.The first image shows the page with the GeodatabaseThe second image shows how the UN have used the file to present to the public.I have submitted a request to UNTAR however they have not responded.ThanksHarry Sullivan
Hello FME Experts,I'm currently working with road network spatial data, which contains a multitude of line features representing roads. Each road feature intersects with others, and I need to determine the correct start and end names for each road segment (i.e., the names of the roads at the start and end nodes).I'm wondering if there's an efficient way to accomplish this using FME. Specifically, I'm looking for guidance on:How to identify the start and end nodes of each road segment. How to extract the names associated with these nodes.Any suggestions, workflows, or examples would be immensely helpful. Thank you for your insights!Best regards,Vishnu N V
Hi,I am creating some admin areas which feature multiple islands within each admin areas.I am using the Aggregator to group the areas, and that is looking fine in the Inspector.When I come to write these features to an esri ArcGIS Portal Feature Service, I receive the following error:<type 'fmeobjects.FMEAggregate'> cannot be converted to esriGeometryPolygonI am building Homogeneous Collections where possible, but even the admin areas that are made up of a single contiguous area have "fme_aggregate" as "the fme_geometry(string)" attribute in the inspector.
I am trying to bring in some files that are dgn/bentley related (cit and nm1) and I can’t figure out how to read these files or view them. I found in a search this site that says how to read a .cit file but I can’t find this or how to view these?https://docs.safe.com/fme/html/FME-Form-Documentation/FME-ReadersWriters/ingr/ingr.htmI do not see this INGR in my lists or anything when I put in reader format .cit
Hi all,i have following problemi join a gdb (with included domains) with an excel sheet…...if i don’t resolve the domains the feature class is written but no geometryif i resolve the domains i have geometries right but no domains anymore Can somebody help me? Thx Franco
Hello,I have a workbench, which when published to FME Server - is not updating the data, rather - I am getting the cached data. Is there any way that when I publish the workbench to FME server- can I use the option of “Rerun Entire Workspace” rather than “Run Workspace”. Please find the below snapshot of the options in workbench : Also, want to mention that - when I did ran the workbench in FME Desktop - it updates the data only when I use the option “Rerun Entire Workspace”, but when I use the option “Run Workspace” - I am seeing the cached data. Please let me know If I am missing anything.
Hi all,i’m building an FME desktop workspace for eventual use with a server automation. we’re taking some emails, parsing them for some specific data, reading text files with additional information, and then putting it all together to be sent out via an API for a software called Desk Alerts.Where i am running into issues is - the text files are UTF-8 encoded and contain line feeds and tabs that the API isn’t happy with.so for instance i might have a message that looks like: now you can see i’ve got 4 tabs and almost 20 line feeds.this text is actually the combined result of several files and the content is variable so for readability’s sake maintaining the line feeds (and preferably the tabs) is paramount.When sending to the API, i get the following error response: {"status":"Error","errorMessage":"Request model invalid","data":{"validationErrors":["[queryParam] : The queryParam field is required.","[$.body] : '0x0A' is invalid within a JSON string. The string should be correctly escap
Hello Experts, I have two Layers PD and AA. I want to align the PD layer to AA layer. Any help would be highly appreciated. Thanks in advance
Hi All, I want to update geometry column (esri SDE) from postgresql writer. What I need to do ? esri keeps the point value like. 2D000000010000000100040001000000110000000101000090D1D3C2A00380A6E4AFBA0880A8D6B90700000000 I can reading from the postgresql as sde.asText / asBinary but I can not write
Hello, I have a folder containing multiple shapefiles, for example, (a, b, d, e, f...). My objective is to utilize the AnchoredSnapper transformer in the following manner: initially, snap shapefile 'a' with 'b', where 'a' serves as the anchor. Then, the resulting snapped shapefile will be used as the anchor and snapped with 'c', and so forth until all shapefiles are snapped. I tried to implement loops but the problem is that the output have to be the new output after every iteration, so i can't get a configuration of the loop. I appreciate any assistance or proposition you can provide. Thank you.This is my wokspace and the custom transformer , using a loop.
HelloHow could FME detect unwanted spike and unwanted zigzag and fix it without changing geometry of lines ?I have used spike remover and zigzag remover but sometimes it catches data that has no relevant with spikes ,what is best angle for transformers?How to keep the original geometry without changing it ,after fix the problem with zigzag or spike remover ?what is the best method to detect dangle lines and fix it ?thanks in advance fme 2021
Hello all,I want to understand the difference between two MapInfo WritersI am running FME Desktop version FME(R) 2022.1.3.0 (20220928 - Build 22630 - WIN64)The Writers are;MapInfo TAB (MITAB)Precisely MapInfo TAB (MAPINFO) Is one Writer better than the other?
Hi, I have some questions about how to make a CityGML more efficient. Context: I have about 1.8m pipelines and cables. I want to visualise these as 3D cylinder/pipes (not actual ‘FMEPipeGeometry’), regardless of the ‘is it necesary to do this’. All features also have attributes like diameter and such. I want to write this to CityDB and CityGML. This results in huge filesize (somwhere like 15gb or 20gb). What I did: My thought process was it would be more efficient to use relative geoetries (a circle) that I extrude/scale/offset/etc to the proper location using the transformation matrix. Of course I cut the original geometry into pairs of 2 vertices. I wanted to reaggregate this to 1 feature once I had the relative geometry up and working. This all worked just fine until I discovered that in CityGML schema the cardinality is 0-1 on features with relative geometry. This means I cannot gather all separate bits of the cables again, giving me an even more absurd amount of features. I also
The Esri ArcGIS Portal Feature Service Reader has most of the capabilities needed to read from a Utility Network including being able to set branch versioning. However there doesn't seem to be a way to resolve domains that are defined at the subtype. The Utility Network uses the subtype layer concept where domains can be applied differently on each field by subtype. Is accessing the subtype domains something the FME can/will support?
I am trying to edit features (location) that are within a trace network from FME.When editing in ArcGIS Pro I’m able to move features and have the network adjust (example: Move a point connected to a line and the line moves with the point) but I am unable to replicate this behaviour when running from FME. It simply moves the point feature away from the line. Notably I did have this working in a geometric network but since completing a recent upgrade and moving to the trace network in ArcGIS Pro it’s not showing the same behaviour.Is it even possible to maintain the network structure using a trace network out of FME?
I have been working on FME for a while now. But not much with geo-related issues.A user receives requests : With and without address. Requests without addresses do contain coordinates (in both RD-Amersfoort and ETRS89)The user would like to see a map with a polygon in it.What steps/attributes do I need to realize this?I have already tried with the following attributes : AttributeReprojector, AreaBuilder, HullAccumulator Please give your advice and tips