- - Community Resources
Ideas
Product Updates
Events
Blog

Home
Forums
FME Form
Data

Data

Using readers and writers to extract and load data.

6,807 Posts
25,077 Replies

When you subscribe we will email you when there is a new topic in this category

6807 Posts

Most replies

Most views

Newest first

loicgueninrandeContributor

asked in Data

Geoparquet Reader on FME Form 2024

Hi @Safe !I have a probleme with Geoparquet reader on FME Form 2024. It does’t work with the file in attachment but work fine with Apache parquet on FME Form 2023. The error is :Thanks for your help.Loïc

283

1 month ago

petergoedbloedContributor

asked in Data

Flow can't access template files

Hi All,I have been building workspaces that get data from an api and write that data to Excel, using Template files. FME Flow can’t access the location of the template file, so what I did was: Make them a published parameter and publish. Then once one Flow, I change the parameters to the Flow resources path where I uploaded the template file as well. I do this in the Workspace properties. However, this doesn’t work, when running the workspace, the log says it can’t open the Template File and still pointing to the old/local location of the template file.What would be the right workflow here?

233

1 month ago

aaronp_osniContributor

asked in Data

Using lists efficiently

Hi folks,I’m hoping this is relatively straightforward and I’m making this more difficult than it seems. I’m collecting images in the field and trying to store the images as blobs in a database. There can be up to 6 images for each property. In the below example only 3 images were collected. I’ve got to the point in my workspace where I’ve joined the images (as blobs) to the survey data, but I’ve got 3 features instead of one. I have a unique survey id. How do merge the data so that I have one feature, with up to 6 images stored as blobs? I’m trying and failing to use lists in the most efficient way possible; via the Aggregator then List Element Filter, then somehow bring the data back together, but this is the point I’m stuck on. Any help is greatly appreciated. Many thanks,Aaron

322

1 month ago

marta.podsiadSupporter

asked in Data

Save polygon as gpx

Hi,I’m trying to save a polygon as a GPX track and am running into an issue. Here’s what I’ve done so far:Converted the polygon to a line using GeometryCoercer Reprojected to EPSG:4326 Created track_id, source, and name attributesThe translation completes successfully, but when I try to open it, I only see the attributes and no geometry. Any idea what I might be doing wrong? 2024-10-29 11_22_11-Window.png

986

1 month ago

jokerfmeContributor

asked in Data

Writing Data into an Arcsde Geodatabase ?

Hello , this is my first post , and I hope to receive an answer I have in FME an ESRI license that allows me to read from and write to an Arcsde geodatabase . I'm interested in knowing if anyone has successfully edited specific feature classes , made modifications in a workspace (such as adding data , deleting data or moving points to new positions), and then writed the output back to the the same feature classes after editing by ArcSDE geodatabase writer.After editing and modifying the data, I understand that in ArcMap, data typically undergoes compression, tracking which data is added and which is deleted.Is it possible to write the output directly to these compression tables ? Is this step necessary to do by FME ?Second question : about writing into Relationship classes I read an article https://support.safe.com/hc/en-us/articles/25407490028813-Writing-Geodatabase-Relationship-Classesthat uses an XML workspace as a template for writing to relationship classes, but I didn’t f

917

1 month ago

jacklonsdaleContributor

asked in Data

Excel Reader showing formula values as missing

Hi all,As part of a development of some routines, I have created a many tab excel file where i inserted Formulae using the AttributeManager and the Cell.Formula naming convention. Due to the nature of the VLOOKUPS i used (dynamically i suppose), i could not do this calculation in FME's Arithmetic Editor.Opening the excel sheet shows the values i do want, given by the formula in the cells. However, when i now come to read this sheet in my next routine, the values are all shown as <missing>. Has anyone encountered this before or have any ideas?Thanks all, kind regards,Jack

9913

1 month ago

timboberooskyContributor

asked in Data

How do I handle possible errors when writing multiple feature types with post data processing

I’ve alluded to my question on a few other posts about FeatureWriters:When FeatureWriter fails there is no output | CommunityAdd a Rejected port to FeatureWriter | CommunityBut figured I’d ask a more directed question in its own posting. I’m in a situation where I want to do some post-processing after writing data to each of my 40 feature types. I have a FeatureWriter for each feature type, in an attempt to catch and do no post-processing for feature types that fail, but continue with post-processing on feature types that write successfully. I’ve come to realize that any FeatureWriter that fails, stops the entire workspace. Setting “Rejected Feature Handling” to “Continue Translation” doesn’t apply to a failed FeatureWriter. The “Rejected port” link above is suggesting adding such a port to the FeatureWriter, which would solve my predicament, at least as I understand it, but it’s not implemented as of FME 2024.1.1, and I don’t know if there are plans to do so. I’ve also tested out sett

324

1 month ago

robert_puntContributor

asked in Data

create time segments

Hello, i have a question for a scanner. the scanner drive through our city. the scannner makes a couple of rounds per day. But i have many record with a timestamp. between the rounds there is a time diffrent say 1 hour or more. how can i crerate a round number that the scanner makes.for example the scanner start at 7.00 en ends at 10.00 (thats round 1) and then starts at 13.00 and ends at 16.00. (thats round 2) the next day is somewhat the same but the count must go on. Beste Regards Robert

597

1 month ago

marta.podsiadSupporter

asked in Data

xml does not open

Hi, I have downloaded data from WFS (GML) as xml (attached). It shows the location of a parcel. However, I am struggling to read information in FME and create a polygon representing parcel.I tried to read it as:OGC GML (Geography Markup Language)OGC GML SF-0 (Geography Markup Language Simple Features Level SF-0 Profile) I am getting this error XML Parser error: 'Error at file: 'C:\Users\MPODS\AppData\Local\Temp\FME_XSD_CACHE\fmexsd_1729842858321_34548.xml' line: 1 column: 1 message: invalid byte '‰' at position 1 of a 1-byte sequence'An error occurred while attempting to parse the XML Schema document 'C:\Users\MPODS\AppData\Local\Temp\FME_XSD_CACHE\fmexsd_1729842858321_34548.xml'The XML Module halted on error, see the logfile for detailsAn error has occurred. Check the logfile above for detailsFailed to obtain any schemas from reader 'GMLSF' from 1 datasets. This may be due to invalid datasets or format accessibility issues due to licensing, dependencies, or module loading. See logfile

7510

1 month ago

thomastupuivaoContributor

asked in Data

using 1st row for column header for a variety of text files

I have a request to build a FME workspace that can read a text file and load it into an Oracle database with the filename with a suffix appended as the table name (ie. filename_ext).That is fine, the problem is the column headers and fields will differ, depending on the filetype (file A might have 10 columns, file B will have 17 and the column names will be different between A and B)I’ve considered using a tester that directs the flow to an attributemanager based on a “user parameter”. This becomes problematic if the source text file changes column structure or an additional file type (file C, D and E) is introduced.Is there a way for the text file to be read (as generic in the featurereader), have the attributes exposed, split and output with the 1st row as the column headers. Ideally without having to know the column names prior.Thanks Thomas

492

1 month ago

dor1411993Contributor

asked in Data

Error with reading Pipe and Structure Data Fields (Attributes)

Hello FME Community!I’m trying to read a Civil 3D file in FME, but I’m not able to access the specific attributes I need.I'm trying to retrieve Pipe and Structure Data Fields (Attributes), but I'm running into errors. Here are the errors that I’m seeing:```2024-10-27 15:13:47| 5.1| 0.0|ERROR |AutoCAD Reader: An error occurred for dataset 'C:\Users\Dor\OneDrive\Perach\Ayalon\חומר ראשני\R6407_01-RME-DR-SVD.dwg', in function AutoCAD2004Reader::readC3DSchema. Exception thrown: 'Not that kind of class' (Object of class AcDbProxyObject can't be cast to AeccDbNetworkPartDef.)2024-10-27 15:13:47| 5.1| 0.0|ERROR |An error has occurred. Check the logfile above for details2024-10-27 15:13:48| 5.6| 0.0|ERROR |Failed to obtain any schemas from reader 'CIVIL3D' from 1 dataset. This may be due to invalid datasets or format accessibility issues due to licensing, dependencies, or module loading. See logfile for more information.2024-10-27 15:13:48| | |ERROR |Workspace generation faile

240

1 month ago

matthewbContributor

asked in Data

Conditional Adjacent Feature Value Grouped by Attributes

I have a dataset that has customer data, vehicle data, planned driving distance (with a bunch of other scrubbed attributes). In this data, there are many days when a customer was not planned to have accumulated any distance but did use a machine. If the planned driving distance to date is missing, I want to copy the value for the previous planned_Driving_dist_to_date that has a value, but I want this sorted by date and grouped by customer and machine. The attached spreadsheet shows my dataset.

241

1 month ago

mr_fmeEnthusiast

asked in Data

"Shadown" in final Mosaic

Hi,I'm using the Raster Mosaicker to create a mosaic with overlapping images. However, the final result generates 'shadows' as shown in the image below. What is the best configuration or additional transformer I can use to solve this problem?Thank you

GEOTIFF Writer Interleaving Type option doesn't seem to work

When writing to Geotiff I would like Band interleaved by Pixel (BIP, interleaved RGBRGB) rather than Band Sequential (BSQ, per channel RRGGBB) interleaving but whichever option I use, the Geotiff always saves using Band Sequential interleaving.I have checked using Photoshop and also GDALinfo and both agree that INTERLEAVE=BAND rather than PIXEL. Is this a bug/can anyone else replicate this?

1163

1 month ago

nsmartParticipant

asked in Data

Reading CSV and Writing to MSSQL Database: tables not recognized by geometry_columns

Hello :)Here is what I am doing: I am working with FME-Flow and FME Workbench 2023.1.1. I have geodata in the form of .csv-files and am using a simple CSV-Reader ➡ MSSQL-Writer to create my datatable in my MSSQL-database (see attached image for FME-script). The datasets are saved to the ‘dbo’-schema and work without problem in GIS (QGIS in this case). This is done for a number of CSV-files. As an example I have shown the summary annotation for the dataset ‘ov_boende_aldre’.What’s the issue? When FME writes to the MSSQL database, the resulting datatable(s) is/are not recognized by the systems view ‘geometry_columns’ (se image below). Why not? When creating a dataset directly in MSSQL or in GIS (QGIS, for example), the dataset appears in said systems view. What can be the issue here? Any secret sauce to make FME create the dataset correctly that I am just missing?All help and hints are appreciated. Also, if you are aware that there is a similar post with a solution then do please direct

272

1 month ago

i_miss_the_chatContributor

asked in Data

loop through the newly created buffers and associate them with subsequent points that fall within the new buffer.

Hello,I am creating buffer polygons for points, but I would like to loop through the newly created buffers and associate them with subsequent points that fall within the new buffer.If 2 points are next to each other, the first point would be buffered and the second point would be associated with that buffer.Thank you.

361

1 month ago

gogopotter90Contributor

asked in Data

How to create a Block diagram of Geodata

Hello ,How could i convert Geospatial Electrical Network Data (includes stations, cables , houses) are connecting to each other to convert these data as compact Block diagram .it is not important the coordinates ,I need to create a simple block diagram of electrical data .I would like to adjust the positions of elements by bringing them closer together without overlapping and create the suitable connections between them .Is there any idea that I should to test ?FME 2021thanks in advance .

593

1 month ago

pcarrerabalsellContributor

asked in Data

creating shapes.txt from stop_times.txt, stops.txt amb kml line (GTFS) any ideas?

Hi, I'm trying to dynamically create the line drawings in a GTFS, that is, I want to create the shapes table, which is empty by default. I thought that it must be possible to do from the stop_times table, the stops table and the 'mother' lines in kml that contain the total theoretical route of each line. With stop_times I can know all the patterns that each line has (starts and ends) identified with stop code. patterns from stop_times.txtstops.txt, with coords The spicy part starts here. I should cut the parent line by start and end, keep the correct segment - I don't know how to identify it - and reorient it in case the 0 vertex of the line doesn't match the stop coords origin The thing is, I don't quite know how to do it. Any help would be appreciated. Thank you very much!Pol PD: I cannot attach KML on this thread. ‘Mother’ lines comes from there:R11: https://www.google.com/maps/d/u/0/viewer?mid=1Oovi7a-WCKVFI_h4IigcGDBCXPo&femb=1&ll=41.91271293489377%2C2.6520825662537018&amp

874

1 month ago

cochdpContributor

asked in Data

Read a new zipped shapefile (with any title) from a specified folder

Hello,I have built a workspace in Workbench 2022.2.4 that reads a specified shapefile (e.g. 10_18_24_line_review.shp) from a folder titled C:\New Shapefiles, performs some basic transformations (datetime stamping, attribute formatting, spatial relation, filtering, etc.) writes out two new resulting shapefiles (with naming conventions based on the new ‘_timestamp’ field) to a folder titled C:\Outputs, then moves the original shapefile being read into a C:\Completed folder using the FILECOPY writer.What I would like to do now is:Configure the workspace to read in any single zipped shapefile that I place in the “New Shapefiles” folder. Scilicet, the naming conventions of the zipped shapefiles I receive will not be consistently predictable, but only one will be in the “New Shapefiles” folder at a time.To clarify: my vision is to drag and drop a new zip of a shapefile (containing concomitant files [e.g. .dbf, .prj, .shp, and .shx]) into the empty C:\New Shapefiles folder, open my workspace

846

2 months ago

mr_fmeEnthusiast

asked in Data

Expose value Python Caller

Hi,How can I expose the result (print), like attribute in python caller: from_value = min_iten[0]['from'] to_value = min_iten[0]['to'] print(f"{from_value[:-10]}") Thank´s

352

2 months ago

areyiContributor

asked in Data

Append data iterative to a FGDB

Hi together We have a problem by writing mulriple cable data into a fgdb. The process runs once per cable and writes their information into a fgdb, the process is repeated for each cable, stored in a csv list. Now we have the problem, that the next cable in the list is overwriting the previous one in the fgdb, instead of getting appended. We assume, that the problem lies in the writer configuration, but we don’t find a solution to our problem. Is it possible to append data to existing data in a fgdb or do you have any ideas (like Python skripts, other transformers that could append the data of each run etc...)? Thanks for your support

191

2 months ago

ed-progresContributor

asked in Data

How to sort column names in Table View (Visual Preview)

Hi FME folks,Is it possible to sort the column names in the Table View in the Visual Preview alphabetically?It's frustrating to have to scroll all the way to the right to find a column that starts with an “a”... Thanks!

1116

2 months ago

philippebEnthusiast

asked in Data

FileGeodatabase - Coded Domain into Coded Domain ?

Not sure why this is possible, but in the FGDB Writer it is possible to create a coded domain into unlimited coded domain.No idea what is the point of being able to do this, because in ArcGIS this is not supported at all. Well, maybe there is something new I missed from ESRI. But yet I can’t see in ArcGIS Pro the SECOND_CD I created.Anyone could tell me what is the purpose of doing this ? What am I missing?I’m on FME 2024.1.Thanks!

394

2 months ago

mr_fmeEnthusiast

asked in Data

PolygonToCoercer very slow

Hi,I have a workspace that uses some RasterToPolygonCoercer, but the performance is very low.I need to improve the performance. I'm trying to convert some clouds detected in my raster image into polygons.How can I improve that? Thank´s

572

2 months ago

fei2000Observer

asked in Data

Failed to add CSV to ESRI ArcGIS Portal in one process

I am new to FME Workbench and I am using the latest 2024.1 version.When I tested to add CSV to ESRI ArcGIS Portal as a new point feature layer, it succeeded to create a new feature service in portal but failed to insert CSV data into the newly created feature service with following error message: “ArcGIS Portal Feature Service Writer: UPSERT failed for feature service 'Sample_Test': Append job failure: 'Operation rolled back.'”Then I have to re-run my workbench again, and this time it will insert all CSV data into the portal feature service without any issue.BTW, I am using “Create If Needed” as Feature Service Handling parameter.Could someone help me out to make this workbench adding CSV data to ArcGIS Portal in one process? Thanks.

100

2 months ago

Page 4 / 273

Badge winners

kenortey8has earned the badge Teacher (Grey)
edwardhas earned the badge Integrate Data with the FME Platform Training
kailinatsafehas earned the badge Enchanted by Attributes
gazmanhas earned the badge Welcome Back
miguelhacarhas earned the badge Spanish Speaker

Show all badges

30,351: Posts

Terms & Conditions Cookie settings

Sign up

Already have an account? Login

An FME Account is required to contribute

Login to the community

An FME Account is required to contribute

Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.

Enter your e-mail address

Back to overview

Scanning file for viruses.

Sorry, we're still checking this file's contents to make sure it's safe to download. Please try again in a few minutes.

This file cannot be downloaded

Sorry, our virus scanner detected that this file isn't safe to download.

A product of

Company Products Giving Back Careers Contact Privacy Feedback

Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.