Using readers and writers to extract and load data.
Recently active
Context : The reader file has 14 attributes ( No formulas ) and the format is Excel File. For the writer file, I am trying to leverage an existing template file (with 2 sheets - Standard & Limits) and the final output should be written to the Standard sheet of the template file. The template file has 80 attributes in total. The value for some attributes are based on the value from the reader attributes, some attributes are hardcoded in the template itself and the remaining attributes will be populated based on the formulas from the existing sheet i.e Standard and the Limit Sheet. What is the ideal approach to achieve the result ensuring no formulas are overwritten in the template file - standard sheet in the process and that my final writer output has the exact same sheets ?
Hi - I’m trying to create polygon geometry from the GeoJSON in the attached file. This is a string attribute returned from an API.I was thinking I could use a GeometryReplacer with the encoding set to GeoJSON. However when I do this the record is rejected with an ‘INVALID_PARAMETER_GEOMETRY_SOURCE’ message.I have passed the GeoJSON attribute through a JSONValidator and this doesn’t flag up any issues.Any suggestions for what I’m doing wrong, or alternative approaches would be very welcome.Thanks in advance.
I am trying to create a workflow that will update a feature layer on ArcGIS Online that is powering a dashboard but keep running into several roadblocks regarding updating the layer. The points themselves are being updated from two sources (other AGOL feature layers) and I am using FME to combine them. First they go through an attribute manager to get the schema matching (They are similar, but different attribute names and one is significantly longer because it just has more data. This longer one is the base for the "master layer"), then those two go through an Update Detector transformer with both of those being compared to the "mater" feature layer. That update detector finds which existing records have been updated (I hope) and then those updated or new records are linked to the master feature layer writer in Update mode. The problem I am running into is when I hit run, an error occurs and says "your layer is missing the ID attribute 'OBJECTID' neede
I have a list containing 93 different WFS URLs, my goal is to be able to read these in one workbench without clutter and for the workflow to handle changes in the URLs for example, a new WFS is released and gets added to the list, I want it to work without having to change or add more to the workbench.Im using FeatureReader to read the format OGC WFS (Web Feature Service) but it wont accept an attribute value as the Dataset, I mean I can put one in there put it does not know what to do with it. If I put the URL directly as the Dataset it knows exactly what to do and I can use the Feature Types to Read and select all an they end up as text separated by a space between each Feature Type. This theoretically means that I could just write a list of every Feature Type I need, use the Attribute Value as Dataset and get results. However, what I really need now is a way to get every Feature Type from the WFS List so that I can ad it in place of Feature Types to Read.Dataset = Attribute ValueFea
Hi,I have a very complex KML file with many folders and subfolders. I would like to save the folder name or full folder path where each placemark is located — something similar to the "FolderPath" field when converting KML to GDB in ArcGIS Pro.How can I achieve this?
Hi,I have error while running the workbench from How to Expose Feature Attributes from KML Tags or HTML Tables – FME Support Center It failes at point Method 2: XML> 2. Fragment XML Any idea why it does not work. I have not changed any of the settings in that workbench
would like to know if FME can read annotation classes and text symbol information that are defined in the Feature Class properties (e.g., in a File Geodatabase or Enterprise Geodatabase).Specifically: Can FME access Annotation Class Names? Is it possible to read text symbol properties like font, size, color, or placement defined in the annotation feature class?
I have tried to read GML file using OGC GML reader but I got some errors. I Tried different version reader still the issues coming all those files.
I have three datasets containing points which I want to bring down to one complete set.Dataset A contains most of the points but not very accurate in terms of locationDataset B contains less points but is more accurate then dataset ADataset C contains the least points but is the most accurate of all three sets. So I am looking to create something like the following: When there are three points (dataset A, B and C) within one meter of each other merge the A and B with the point from C. When there are two points, for example dataset A and C, merge the point from dataset A with the one from C. And so on.. A way of ranking I would say. Now I have tried using the NeighborhoodAggregator but I cannot specify which point I want to merge which point. I have also looked at the NeighborFinder but I cannot seem to get that working either. Is it possible to have all three datasets analyzed together? Or do I have to do two at a time?
I'm trying to read some USFS wildland fire data. There's a link to the kml on their website:http://www.wfas.net/google-earth/wfas_nfdr.kmzWhen I use the url in the kml reader, it downloads it to a temp file location successfully (I can see it there) and it's unzipped. It's got a folder structure about 5 deep.Then I get FATAL |KML: Could not determine the root kml file of `C:\\Users\\xxx\\AppData\\Local\\Temp\\FME_OGCKML_KMZ_1562607566639_14376' If I go to the temp file location and navigate down to the kml, I can read it in no problem.C:\\Users\\xxx\\AppData\\Local\\Temp\\FME_OGCKML_KMZ_1562607566639_14376\\fsfiles\\office\\wfas2\\dir-obs\\wfas_nfdr.kml
I have two layers: one raster with population data for each cell (blue) overlapping polygons (buffer of 100m) around existing road segments (red)I would like to know the total population contains in each polygon. How do I do this in FME?
Hi Everyone, I have this “open” dataset provided by the City of Toronto. I am using a few from the city that are provided in this manner. The only way to bring it into ArcPro and then extract it to a geodatabase is to use the json address and add by url, then use the ‘feature to point’ geoprocessing tool to extract the data to points in gdb format, then use as needed. I need to get this layer into FME from the live feed so I can analyze it and then push it out via other format(s), one of which is AGOL feature layer.For some reason I have been unable to bring it into my FME script via any JSON reader or HTTP caller and I’m wondering if anyone else here will have more luck.The link I am working with is as follows:https://gis.toronto.ca/arcgis/rest/services/cot_geospatial2/MapServer/64The json link is:https://gis.toronto.ca/arcgis/rest/services/cot_geospatial2/MapServer/64?f=pjson If anyone has the steps to bring the content into FME (geometry and attributes) to have it available for f
I'm converting 3D data from Postgres to an Esri Indexed 3D Scene Layer (slpk). I'm converting this to an i3s service on Azure (not ArcGIS Online). When loading this as an external layer in ArcGIS Online, the LOD seems totally random, buildings are being visualised but not the way I'd like to see (not based on distance). Is there a way of optimizing this in FME?
I’m working on a data inventory project and need to figure out what data sources are being used in a large number of aprx’s. Is there a way to determine the data sources of an ArcGIS project by directing a workbench to an aprx file path?
Hi everyone, i have few questions about coordinate systems1. How can i know in which coordinate system is file that i upload2. And how can i reproject that coordinate system into 32634(I have been trying with reprojector but it’s not exactly right when i add project in qgis on my map)3.Which ever coordinate system i tried, it never is 100% matchSo i need help if someone can tell me what should i do?Thank you in advanced
Hi ExpertsI would like to suggestion and GuidanceDoes FME Form has Time as field type I’m using FME 2024Thanks
Hi,I’m finding that the date shown for data updates on ArcGIS Online Hub is misleading and not showing the date that we are actually updating the data, has anyone used the Metadata Updater to update any of the dates in metadata? I don’t see it listed in the dropdowns for the updater. Or should I be using the XMLUpdater?Thanks,Lorinda
Hi, Based on the sample table, I want to implement a logic. Same name, different policy # and same address - Goes to file 2 Same name, different address and different policy number - The first record goes to file 1 and rest of the records go to file 2. All the manual should again go to the file 2.Policy Number New/ Renewal/ Manual Name Address To process Split 123 New John Chris 123 Oxford St Y File 1 110 New Mary Jane 45 Caron ave Y File 1 111 New Mary Jane 360 Caron ave N File 2 112 Renewal Kent Bryan 333 Queen St Y File 1 113 Renewal Kayla Jan 333 Queen St Y File 1 114 Renewal Jung Chen 5 Queen St Y File 1 115 Renewal Jung Chen 5 King St N File 2 116 Manual Ray Ban 5 abc st N File 2 116 Manual Kan Can 6 xyz st N File 2 116 Manual Harry Potter 8 niagara st N File 2 116 Manual Rob Smith 9 India St N File 2 116 Manual Tony Stark 10 peter st N F
I am having trouble writing a few rasters to a raster fgdb. Most are fine but a generate errors of undefined. But previous definition seems fine. bit of a stumper. don’t think it’s a projection issue. Any thoughts? Error on write for 1 or 2 rastersPrevious step has all as fme_aggregate and fme_type as raster.
We are using FME 2021 and planning to bump postgres version to 14.Are there known issues with that combination?I have tried to find a compatibility sheet for FME Desktop/Server and Postgres and/or other Software, but did not succeed.
Hi guys!I'm trying to understand how the STAC Asset reader works.In my case, I have a URL to an item, but when I try to access it, it complains that I don't have authentication. When I then check the "Use authentication" box, I only get options related to AWS, and not the service I created.I've used the URL to retrieve data via Radiant Earth Stac browser with my credentials and it works there.Has anyone used this reader to extract data from sources other than the default ones?
data source CSV file each day populated with date stamp, my requirement need to load CSV each day to SQL server
Hi,I am new to FME, and I am trying to build an FME workspace that reads multiple .csv files from a folder and write each file to separate table in SQL server. Each CSV file:has different schema should became it is own SQL tableHere is what I tried so far:I used Directory and File Pathnames reader I used Feature ReaderThe <Generic> port of Feature Reader shows no data, I am assuming because the schema of each is different. the <initiator> port give me the correct file name and path.
The website Dynamic Workflows: Destination Schema is Derived from a Lookup Table > Creating a Table-Based Schema > Geometry Type tells me that valid values for the `geometry_type` column are the values from the `fme_type` format attribute. It provides the full list of values under this link. The link is incorrect. This is the list of FME internal data types. These are NOT valid values from the `fme_type` format attribute and these are NOT values from the `geometry_type` column in the screenshot above in the tutorial.Where is the correct list of values for the `geometry_type` column or the `fme_type` attribute?
Hey Everybody, In my work, I generally get the question from clients to build a FME Server Web Application. Often, the question involves a MS Excel file, which needs to be analyses and converted to points/lines/polygons (GIS). Mostly, I use a Creator to kick-off the Workspace and a FeatureReader to load in the Excel. Also, I force the FeatureReader to have one single output port (Generic) and load in the Excel file using a User Parameter when kicking off the Workspace. This means that I need to Expose all attributes individually in the Workspace.However, this limits the flexibility. When I want to create a Web App from the Workspace, the input Excel always needs to have the same columns (so a Template Excel as input). An potential extra column is therefore not exposed an not in the result of the Workspace. However, in some cases, it is for my clients not possible to always use an Excel Template. Or they want to add additional information. My question therefore is, is it possible to dyn