Using readers and writers to extract and load data.
Recently active
I’m working on a data inventory project and need to figure out what data sources are being used in a large number of aprx’s. Is there a way to determine the data sources of an ArcGIS project by directing a workbench to an aprx file path?
Hi everyone, i have few questions about coordinate systems1. How can i know in which coordinate system is file that i upload2. And how can i reproject that coordinate system into 32634(I have been trying with reprojector but it’s not exactly right when i add project in qgis on my map)3.Which ever coordinate system i tried, it never is 100% matchSo i need help if someone can tell me what should i do?Thank you in advanced
Hi ExpertsI would like to suggestion and GuidanceDoes FME Form has Time as field type I’m using FME 2024Thanks
Hi,I’m finding that the date shown for data updates on ArcGIS Online Hub is misleading and not showing the date that we are actually updating the data, has anyone used the Metadata Updater to update any of the dates in metadata? I don’t see it listed in the dropdowns for the updater. Or should I be using the XMLUpdater?Thanks,Lorinda
Hi, Based on the sample table, I want to implement a logic. Same name, different policy # and same address - Goes to file 2 Same name, different address and different policy number - The first record goes to file 1 and rest of the records go to file 2. All the manual should again go to the file 2.Policy Number New/ Renewal/ Manual Name Address To process Split 123 New John Chris 123 Oxford St Y File 1 110 New Mary Jane 45 Caron ave Y File 1 111 New Mary Jane 360 Caron ave N File 2 112 Renewal Kent Bryan 333 Queen St Y File 1 113 Renewal Kayla Jan 333 Queen St Y File 1 114 Renewal Jung Chen 5 Queen St Y File 1 115 Renewal Jung Chen 5 King St N File 2 116 Manual Ray Ban 5 abc st N File 2 116 Manual Kan Can 6 xyz st N File 2 116 Manual Harry Potter 8 niagara st N File 2 116 Manual Rob Smith 9 India St N File 2 116 Manual Tony Stark 10 peter st N F
I am having trouble writing a few rasters to a raster fgdb. Most are fine but a generate errors of undefined. But previous definition seems fine. bit of a stumper. don’t think it’s a projection issue. Any thoughts? Error on write for 1 or 2 rastersPrevious step has all as fme_aggregate and fme_type as raster.
We are using FME 2021 and planning to bump postgres version to 14.Are there known issues with that combination?I have tried to find a compatibility sheet for FME Desktop/Server and Postgres and/or other Software, but did not succeed.
Hi guys!I'm trying to understand how the STAC Asset reader works.In my case, I have a URL to an item, but when I try to access it, it complains that I don't have authentication. When I then check the "Use authentication" box, I only get options related to AWS, and not the service I created.I've used the URL to retrieve data via Radiant Earth Stac browser with my credentials and it works there.Has anyone used this reader to extract data from sources other than the default ones?
data source CSV file each day populated with date stamp, my requirement need to load CSV each day to SQL server
Hi,I am new to FME, and I am trying to build an FME workspace that reads multiple .csv files from a folder and write each file to separate table in SQL server. Each CSV file:has different schema should became it is own SQL tableHere is what I tried so far:I used Directory and File Pathnames reader I used Feature ReaderThe <Generic> port of Feature Reader shows no data, I am assuming because the schema of each is different. the <initiator> port give me the correct file name and path.
The website Dynamic Workflows: Destination Schema is Derived from a Lookup Table > Creating a Table-Based Schema > Geometry Type tells me that valid values for the `geometry_type` column are the values from the `fme_type` format attribute. It provides the full list of values under this link. The link is incorrect. This is the list of FME internal data types. These are NOT valid values from the `fme_type` format attribute and these are NOT values from the `geometry_type` column in the screenshot above in the tutorial.Where is the correct list of values for the `geometry_type` column or the `fme_type` attribute?
Hey Everybody, In my work, I generally get the question from clients to build a FME Server Web Application. Often, the question involves a MS Excel file, which needs to be analyses and converted to points/lines/polygons (GIS). Mostly, I use a Creator to kick-off the Workspace and a FeatureReader to load in the Excel. Also, I force the FeatureReader to have one single output port (Generic) and load in the Excel file using a User Parameter when kicking off the Workspace. This means that I need to Expose all attributes individually in the Workspace.However, this limits the flexibility. When I want to create a Web App from the Workspace, the input Excel always needs to have the same columns (so a Template Excel as input). An potential extra column is therefore not exposed an not in the result of the Workspace. However, in some cases, it is for my clients not possible to always use an Excel Template. Or they want to add additional information. My question therefore is, is it possible to dyn
I am reading dwg file, but i cant set him the right coordinate systemThe file i am reading is from autocad 2012In autoCad coordinate system is set to EPSG : 31277, i dont have that system in my fme workbench Thank you in advanced
I have various values in a 'code' field.A sample would be: A33, B111W, B12, C3V etc, etc.This issue is, there should be decimal points between all numbers, i.e, these should correctly read: A3.3, B1.1.1W, B1.2, C3V. There does not need to be points between the letters.I can string replace “33” with “3.3” and continue to do that for all values if I have to, but I’m hoping that there is a more efficient way of capturing these values and using regex or something to force decimal points between the numerical characters. I’m not well adept at using the stringformatter, but maybe there is something there?
I am trying to create a workspace that identifies consecutive duplicate point in DWG polylines. and outputs their coordinates for future review.As per my example, Autodesk AutoCAD DWG/DXF (normal reader or FeatureReader) does some autorepair on the features while reading. The DWG feature has 12 vertex in AutoCAD but only 7 in FME.Please see attached sample feature in dwg format.I want FME to read the same geometrie that AutoCAD gives.Using FME 2024.2.1.0 build 24801
I am writing data to an excel file but when I open the written file, the first row of the file (that includes the column headings) is hidden and I have to manually expand it to show the headings. Is there a way to fix this in the writer at all?
Hello everyone,I’m working in FME 2023.2 and using the ChangeDetector to compare features coming from PostGIS and ArcGIS Online (AGOL). Visually, the geometries look identical (no edits were actually made), but the ChangeDetector keeps reporting that a large number of features have updated geometry.I’ve already tried:Turning on Lenient Geometry Matching in the ChangeDetector Setting Vector Tolerance to a small value (like 0.0001 - Takes more time)While that helped reduce the count somewhat, I’m still seeing a substantial amount of features flagged as updated when they shouldn’t be.I suspect it might have to do with differences in ring orientation, vertex ordering, or the presence of Z/M values, but I’m not certain how best to address these discrepancies so the ChangeDetector recognizes that there hasn’t really been a change.Any suggestions or advice on how to make the geometry truly match (or be treated as unchanged) would be greatly appreciated. Thanks in advance!
The process is to read from a sql database, write to excel file, then import that excel into another software.The software expect one of the fields to be a date field. The data coming out of the database is datetime.I used the DateTimeConverter to go from 20190101000000 to 01/01/2019. I'm using an excel template so I'm setting that field to be of type date. The writer in the workspace is set to be of type date. But the data that is written is text. I can tell it is text because it is left justified instead of right. If I click into the cell and press enter, the data is then right-justified and I'm able to import the excel into the software. I've looked up this issue and the most common solution I found is to do a "Text to Columns" under the Data tab in excel. I did this in the template, but the data written is still a textI thought maybe it was the DateTimeConverter transformer, so I created a view in the database to make it a date instead of a datetime. It is
I have a text file that looks like this:"CommonCode":"EI","IATA":"EI","ICAO":"EIN","CarrierName":"Aer Lingus""CommonCode":"AM","IATA":"AM","ICAO":"AMX","CarrierName":"Aeromexico"I want it to look like this:CommonCode IATA ICAO CarrierName EI EI EIN Aer Lingus AM AM AMX Aeromexico I want it to create the column names dynamically from the text file as the column names will change and increase or decrease in number.In R there appears to be a command called: “unsparsify” Which according to this post seems to do it.https://datascience.stackexchange.com/questions/55879/file-format-where-column-names-are-repeated-on-each-rowI can do it using a combination of AttributeSplitters, ListExploders, ListIndexers, AttributePivoters and AttributeExposers. But I can’t seem to get away from having to Explicitly input the Attribute names in the AttributeExposer and I need it to be dynamic.Any ideas?
When i import OSM.pbf data in FME with the OSMPBF reader the following warning pops up:During translation, some features were read that did not match a reader feature type in the workspace. This can happen if the reader dataset is changed, or a reader feature type removed or renamed.When i open the data in QGIS, the missing lines are there. Version 2024.2
I am reading an IFC-file using a FeatureReader with the following configurations: I am using the <Generic> port to connect it to a Reprojector-transformer. After reprojecting it, I want to use a FeatureWriter and write it as a new IFC-file. I am however stuck on how I should set up the FeatureWriter. I have managed to get a new IFC-file, but the file size is quite small. I am interested in automating the process, and want the FeatureReader to accept any types of IFC-files. Any tips on how I can solve this?
Hello FME Folks,This is related to MapInfo Tab files. Since we upgraded from FME 2018 to FME 2024.2.1. The MapInfo coordinate System (Non Earth Meters) are not detected which used to happen in FME 2018. Instead it is taking wkt_auto_1 and is misaligned. Please let me know what changed.
LineOnLineOverlayer after upgrade from FME 2020 to FME 2024 output there is a count difference in outputs. The data used for both version is same, In FME 2020 the tolerance parameter was none while there is no such option in 2024, so used 0 as tolerance value. Still showing count difference.
Dear All,I’m working with two datasets that share a common key: Station ID. Each dataset provides coordinates ( latitude and longitude) for these stations. My objective is to determine whether the station with the same ID have identical location in both datasets or if there are discrepancies.To achieve this, I need to calculate distance between the corresponding coordinates from both datasets. While this seems straightforward, but I encountering challenges in implementing it effectively. I would appreciate any help on the best approach.Thank you in advance.BR//
Hi, I have a workspace using an ESRI ArcGIS Server Feature Service reader with a token authentication. I am trying to read 3 feature types from the map server and 2 of them read fine but one comes back with the error: "ArcGIS Feature Service Reader: Encountered an unexpected error. The error code from the server was '500' and the message was: 'Error performing query operation'. Details: ''" Does anyone have any ideas of why only one of the feature types would not work? The three feature services should all have the same naming standards and schema so there is no reason why they should behave differently that I can see. Are there any suggestions for how I might be able to find the source of this issue? Any help would be greatly appreciated! Thanks.