- - Community Resources
Ideas
Product Updates
Events
Blog

Home
Forums
FME Form
Data

Data

Using readers and writers to extract and load data.

6,812 Posts
25,087 Replies

When you subscribe we will email you when there is a new topic in this category

6812 Posts

Edit <core:CityModel.....> of CityGML writer

CityGML writer created the following line automatically. <core:CityModel xmlns:brid="http://www.opengis.net/citygml/bridge/2.0" ..... I would like to change the text after <core:CityModel to other text.Could you tell me the procedure?

4 years ago

davebarter

asked in Data

S3Connector - Upload dynamic filename

I am creating a CSV file in a Featurewriter with a filename created as follows:-litmap-@DateTimeFormat(@DateTimeNow(), %B-%d-%Y)This is dropped into a directory on a fixed pathHow can I pass this filename to the "File To Upload" parameter in the S3 uploader?

Extract attribute value type from ESRI shapefile

Hi there,I'm trying to read attributes from a shapefile and write them into a SQL database. I need to write also the data type for every attribute, exactly as written in shape (not based on their values). How can I extract data types from data source? Thank you for any help!

770

4 years ago

asked in Data

How to set feature color transforming from ESRI shapefile to DGN?

I transformed SHP to DGN, but all features are in one color. I need to apply features in color set by level. Even with seed file set or DGN file with right colored features as a color source, features from SHP data are in the right levels, but have no individual color by level. Seems like transformer is putting SHP feature in right DGN level (by attribute) but with default color, not color by level.

240

4 years ago

dms2Contributor

asked in Data

Best option for catching DB related errors when writing to Oracle spatial tables

My translation writes features to spatial and non spatial Oracle tables in a given order. The problem is that any eventual DB related error should be reported to a web service and also logged to an xls file.I'm using Featurewriters to control the writing order but the Featurewriter transformer has no Rejected port. How do I catch DB errors then?SQLExecutor is fine for non spatial tables but not for spatial tables with complex geometries (due to the character number limitation for sql statements).Should I keep the Featurewriter approach and use a shutdown python script to catch DB errors and do the Excel and web service part? Is there any other option?Thanks for the help.

Hello, can someone help me with FME convert .GDB to ESRI Shape format much appreciated

4 years ago

mhabContributor

asked in Data

FME_TEMP size exploding with fts files

So I learned that *.fts files are part of the ongoing tuning efforts in FME and are written to FMETable subfolder in FME_TEMP directory.I was literally shocked yesterday when my FME_TEMP drive (256Gb SSD) suddenly ran full because FMETable seemingly "exploded" without limit from reading and processing 5Gb of ffs files.My impression is that this is a first in FME 2020 (maybe 2019 too ?) as the FME processing active at the time was little more than reading 5Gb (190 different ffs files) and writing them out as a single set of ffs. This process up to now never was in any way critical but now is a blocking point in my environment. Is it in any way explainable why 60 million features consume 250Gb+ as *.fts files - when they only need 5Gb as ffs files ?I expect not more than 40-50Gb in the FileGeoDB which will receive this whole dataset !B.T.W.: Memory use was next to nothing during this whole adventure ...I came across excessive disk space use when FME writes raster data to disk i

Reading in Oracle Long Raw data

Hi all, I wondered whether anyone had come across any issues reading in data from an Oracle DB with type LONG RAW. It seems FME cannot read in data of this type and 'skips' that column when reading in an Oracle DB table. Does anyone have any pearls of wisdom on how I could load this data in to my FME workflow? Thanks all, Amber

351

4 years ago

tim_woodContributor

asked in Data

Can FME be launched from PowerShell?

I've been trying to run an FME Workspace using PowerShell but I'm getting errors. At the moment, it's just a test Workspace with a Creator and an Emailer.When the FMW is on a different server in a different domain, I get an error saying the FMW file could not be opened. I am mapping a drive to the other server in the PowerShell console before I try to run the Workspace.When the FMW is on the same server as the EXE and PowerShell window, I get "A fatal error has occurred. Check the logfile..." although the Workspace did seem to have completed.I've tried the same things in a Command Prompt window with no errors so I'm going to use a Batch File for now.

17315

4 years ago

freddy17Contributor

asked in Data

Excel Multi language dictionary

Hi everyone I have an interesting question, and looking for some inspiration or advice to a problem I would like to solve. I thought I would ask the question before the FME Webinar on Excel that will be this Thursday (24.09.2020) There are a few workspaces that I need to adapt for multiple languages (English, German, French). The result is that all my outputs should be adapted for the language, meaning all texts and even the header in documents should be edited. My workflow is as follows: GoogleForms csv Input > FME Desktop Transformers > HTML Output As an exampleIn JSON one would build a multiple language dictionary like this: var arrLang = { "en-gb": { "HOME": "Home", "ABOUT": "About Us", "CONTACT": "Contact Us", }, "zh-tw": { "HOME": "首頁", "ABOUT": "關於我們", "CONTACT": "聯絡我們", }}; I am pondering what the best way to do this for my FME Workspaces. In my attached excel, I have used Lists to achieve a similar result, but how c

IMPORT KML AND/OR EDIT GOOGLE MY MAPS.

I tried with google fusion Table, but the service is down. if anyone has alternative to mymaps for creating an editable map on field, i am all ears

280

4 years ago

havmoejbvContributor

asked in Data

Reading LandXML CgPoints Feature info

I have some LandXML files containing points (CgPoints) and lines (PlanFeature) with some attributes. As per the LandXML schema, the <CgPoints> are defined with the coordinate geometry as a <CgPoint>, and the attribute information as <Feature> with properties: <CgPoints> <CgPoint name="TRAU201" code="" oID="TRAU201">1256922.890706 84309.502725 165.818283</CgPoint> <Feature code="Gemini attributes"> <Property label="S_OBJID" value="TRAU201"/> <Property label="S_FIELDCODE" value="41050"/> </Feature> </CgPoints>What I am struggling with is to read the Feature information. The LandXML reader gives me an option read both CgPoints and CgPoint, but not Feature. CgPoint will give me the name, code and oID above, but my Feature attributes are not there... I need to extract the "S_FIELDCODE" value along with the geometry, but how?

Autocad reader: get the version of a dwg file

Hello, I'm reading multi dwg files using dwg reader and need to get the version chossed when the user saved dwg file. i can see it in translation log when i run the reader but i couldn't get it as an attribute. any ideas or maybe transformator that i didn't find ´?

REST to FGDB error - "GEODATABASE_FILE writer: An unknown exception occurred"

I am experiencing an error when reading AGOL feature services that are publicly available as REST services and writing them locally to a file geodatabase. I am using the Esri ArcGIS Server Feature Service reader and the Esri Geodatabase (File Geodb) writer. However, I am having an issue where the translation fails when writing to the file geodatabase.To clarify, I have no issues reading the data into FME however when writing to file geodatabase I get a single error message in the translation:2020-06-16 12:02:03| 4.0| 0.0|FATAL |GEODATABASE_FILE writer: An unknown exception occurred Attached is the workbench: Data source: the AGOL REST service URL I am reading is here: https://services1.arcgis.com/8CpMUd3fdw6aXef7/ArcGIS/rest/services/Parcel_Service/FeatureServer Trouble shooting this issue found:I am able to write to shape file with success for the most partHowever I've observed that when writing to shapefile in some cases some fields are written with no values and in small numb

How to convert AIXM 5.1 file to ESRI geodatabase or shapefile? What kind of Transformers I should use?

I need to get geometry and attributes out of AIXM 5.1 into ESRI geodatabase.

730

4 years ago

lifalin2016Contributor

asked in Data

Paging in WFS 2.0.0 ?

Hi list.Does the FeatureReader reading WFS 2.0.0 data handle paging ?If a service is known to be limited to return a maximum number of features per call, paging, i.e. fetching the complete dataset in chunks, is a way to handle this.However, the only options I see are "Max Features" and "Count", but they don't incur paging, as far as I can tell.So how does FME handle WFS 2.0.0 paging ?CheersLars I

Cannot write multiple specific rows and columns for excel table

For some reason, I am unable to write to two consecutive specified columns in an excel spreadsheet. I have specified the Column B and 3 rows of that for one set of data. I have also specified Column D and 3 rows of that for the other set of data. It doesn't matter if I use the featurewriter or the xlsx writer. I am unable to get Columns B and D to populate together. My export comes from a template that has calculations in it. There is a ROW attribute that works correctly with fixed values of 37, 38, and 40. The column is specified as the field B and D for each data stream which is correct. They are both going to the same sheet and I have been unable to make the other column that is not writing write to another column or sheet somewhere. I do this often with other reports and have not yet had this issue. Driving me crazy. Any suggestions would be appreciated :) Thanks.

420

4 years ago

jhu_20Contributor

asked in Data

Need help to read Excel header info and hourly data and output to SQL Server tables

I have over 100 Excel files with standard format: each Excel file has multiple sheets corresponding to multiple days in one month on recorded hourly well flow data, and the format for each sheet is fixed. The top part of sheet has well header info while the lower part are recorded hourly well flow data. I need to combine the well header info and hourly flow data into standard table format and load into SQL Server database table. It is easy to read the hourly flow data into. My challenge is to also read well header info from the same sheet , combine with hourly flow data, and write all these data into a table. I am looking for examples and help on similar Excel data loading. Thanks in advance.

Hello, I would like to know how ca I read the Python data. I need to build a Workbench which as reader will be a Pyton script.py Is there any transformator? Thanks Tereza

4 years ago

tim_woodContributor

asked in Data

Error using RasterGeoreferencer on JPG with point and angle from CSV

I have a JPG image and a CSV file with data that can be used to geo-reference it (attached).I've built a Workspace to apply the geo-referencing data to the JPG. AttributeManager makes FactorY a negative number as I found I needed to do this when working out the process using ArcGIS. In the RasterGeoferenencer, I've selected the relevant fields from the CSV file as shown in the screenshot. OutX and OutY are the Ordnance Survey Eastings and Northings.When I run the Workspace in FME 2017.0.1.1 (64-bit), I get the following error:2017-06-08 15:39:10| 0.8| 0.0|ERROR |@GeoreferenceRaster: Failed to retrieve X Upper Left Coordinate parameterI've tried setting the coordinate system on the JPG Reader and/or the CSV Reader, and setting OutX and OutY to x_coordinate and y_coordinate in the CSV Reader and then as real32. None of these make a difference.Any ideas?

470

4 years ago

habbuContributor

asked in Data

Problems with Property Sets when writing IFC 4 instead of IFC 2x3

Hello I've created a workspace, writing an IFX 2x3 file with two Property Sets per feature, which works well.Changing the writer format to IFC 4 the two Property Sets are not written. I've no Idea, what the problem could be and hope, that somebody could tell me, what I'm doing wrong.You'll find the workspace and the Data attached. Regards, Hartmut

110

4 years ago

davtorghContributor

asked in Data

SQLite database: how to enforce foreign_keys.

Hi everybody, I need to write to a sqlite database containing tables with foreign keys constraints, but FME seems not to enforce the foreign_keys pragma directive when connecting to the database. I discovered this by running a pragma foreign_keys; query against the database with an SQLCreator (it returned 0).This is obviously a problem, because in this way the SQLite Writer can insert records that violate referential integrity.Is there a way to make the SQLite Writer execute thepragma foreign_keys = 1;directive before start writing? Any suggestion is welcome! Thanks

Group by and write to database

120

4 years ago

fikusasContributor

asked in Data

HTMLStripper wipes out accents

I have some text with HTML tags and properties. I need to strip out the HTML parts and leave plain text. I'm using the HTMLStripper which does the job pretty good, but replaces accented letters with non-accented ones plus adds question marks.How this behaviour can be fixed? Text example (before and after):Iš liet. šveñtas, -à &lsquo;pagarbiai saugomas, laikomas; iš pagarbos neliečiamas&rsquo;.Iš liet. šven?tas, -a? ‘pagarbiai saugomas, laikomas; iš pagarbos neliečiamas’.

4 years ago

theodore

asked in Data

Use LenghtToPointCalculator on a SHP file

HelloI would like to put points on a single line to use the named transformer and get the length as a new attribute, but it seems that the feature merger used in this examplehttps://gis.stackexchange.com/questions/30518/fme-calculate-distance-of-point-along-a-linedoes not work on Esri Shapefile format, I don't know why.Any idea on how to fix points on a curve, while keeping each point as a line/different entity in the visual preview ?Thanks Edit: MeasureGenerator does not work as the curve has much more vertex than needed and they can't really be sorted

120

4 years ago

Page 61 / 273

Badge winners

indrajeetsharmahas earned the badge Expert (Grey)
indrajeetsharmahas earned the badge Teacher (Grey)
indrajeetsharmahas earned the badge Student (Grey)
sunilgtefhas earned the badge Expert (Grey)
sunilgtefhas earned the badge Teacher (Grey)

Show all badges

30,671: Posts

Terms & Conditions Cookie settings

Sign up

Already have an account? Login

An FME Account is required to contribute

Login to the community

An FME Account is required to contribute

Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.

Enter your e-mail address

Back to overview

Scanning file for viruses.

Sorry, we're still checking this file's contents to make sure it's safe to download. Please try again in a few minutes.

This file cannot be downloaded

Sorry, our virus scanner detected that this file isn't safe to download.

A product of

Company Products Giving Back Careers Contact Privacy Feedback

Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.