Skip to main content

- - Community Resources
Ideas
Product Updates
Events
Blog

EN

Home
Forums
FME Form
Data

Data

Using readers and writers to extract and load data.

6,669 Posts
24,613 Replies

When you subscribe we will email you when there is a new topic in this category

6669 Posts

Recently active

Most replies Most views Newest first

featuremichaelSupporter

IFC3DObject Transformation from Local to X,Y,Z!

Hello FME members,I have an IFC file with no coordinate information in the site feature.it is a 3dObject of a parcel that is supposed to represent the surface - but it is just a (3dObject) flat surface with an extrusion of 1]What I have done: I have at least 4 coordinates to transform, I tried to use:the 3daffinewarper, but the transformer showed me an error [because the transformation from 0,0,0 to x,x,444.1 is coplanar - and I only have 4 known points that went from z:1 to z:443] (I also tried 0,0,1 to x,x,444.1, but it didn't work) so I used the affinewraper to send the surface to the correct coordinate system)that worked!i need to know how to get the 3d object to the correct z value to display the surface (3d object) at the correct height.I hope someone can help me.Greetings Michael

lambertusEnthusiast

Generalizing centerlines without 'side branches'

Hi,I have a dataset with Centerlines shown in blue, some lines contain too much detail which is not accepted as input for the LineDivider I use in the downstream pipeline.Is there a way I can smooth the blue centerline into something like the red line shown on the right?The Generalizer should be the way to go I guess but did not find the right settings yet.

emerson_stylesContributor

Converting Ordnance Survey MasterMap data to DWG format

Has anyone successfully created a workspace that can convert Ordnance Survey MasterMap data to dwg format? I understand the process would be quite complex, but it would be good to get an idea of what’s involved from anyone who may have done this.

Always create excel file even if data is empty

I'm creating a new excel file nightly and some nights there will be no data to populate the excel file, but I need the file to be created anyways. Is there a step in the Excel writer that I'm missing to do this?

FeatureWriter silently dropping bad features and total features written count is wrong

Hi,I'm using a FeatureWriter to insert data into a PostgreSQL database. When certain features can't be converted into the desired data types, FME drops them without terminating the translation.The logger gives a message in blue:"Value of attribute 'test' could not be converted to type 'bool'. Feature will be logged and skipped" However, the issue lies in trying to capture this scenario and terminate the translation.Despite dropping some features, the FeatureWriter when outputting the summary feature still contains the FULL COUNT against the '_total_features_written' attribute. I was intending on comparing this number to a count of incoming features to check for a difference. Now the only other way I can think of is querying the database immediately after and getting the written feature count that way. I guess I have two questions:1. Is there an inbuilt way to halt the translation when a feature is dropped like this?2. Is the _total_features_wri

zentnerfabianContributor

Compare FileGeodatabase in FME?

Hi guys,is it possible to compare two filegeodatabases in FME? In each fgdb are 443 feature classes (points, lines and area feature classes). After some testing with the first vesion of the fgdb, we had to update the fgdb in some feature classes and changed some attribute fields and add some new domains and values. Is there any transformer which can show me the differences between both fgdb’s?Thanks for answering in advance.Cheers,Fabian

stevenjhContributor

WFS reader truncates response and fails

I'm using a FeatureReader to read a wfs service and getting strange failures.The following error is being loggedHTTP transfer error: 'Failure when receiving data from the peer'XML Parser error: 'Error in input dataset:'https://data.linz.govt.nz:443/services;key=xxxxxxxxxxxxxx/wfs/layer-53382?SERVICE=WFS&VERSION=2.0.0&REQUEST=GetFeature&TYPENAMES=data.linz.govt.nz%3Alayer-53382&NAMESPACES=xmlns%28data.linz.govt.nz%2Chttp%3A%2F%2Fdata.linz.govt.nz%29&BBOX=-36.939824852066955%2C174.70954193086246%2C-36.91494916200109%2C174.76090622708065' line:1 column:121683 message:unexpected end of input'The XML Module halted on error, see the logfile for details<WFS> An error occurred while reading a GetFeature response, to ignore the error and avoid losing the previously read features, set the WFS "Ignore GetFeature Error" parameter. Total features downloaded: '0'A fatal error has occurred. Check the logfile above for detailsAccessing the link and checking the file manually

proteveContributor

after exporting shp to gdb features are missing from gdb

I have manage to export shp in file geodatabase with datasets but now it seems that i have a strange problem (or bug).every shp that i have exported is missing 1st entry in geodatabase. lets say a shp file has 10 points then i’ll get only 9 points in gdb and the one is missing is always the 1st pont in shp file. every shp file is missing that 1st entry.the strange thing is that fme shows in information that it wrote all entries that the shp files has but somehow 1 is missing.there is absolutly no error in logs. i get some info about some fields that i don’t export anyway. i really have no idea why this is happening.the shps are also exported with FME from othe types of data.for writer i use Esri Geodatabase (File Geodb Open API) with some schemas that i read from a template database with featurereader and join it on shps (no schema was readed).so what im doing wrong?why there is no error about the missing entries?

ienesaglam60Contributor

Cut the fme polygon from fme aggregate

How ı Cut the fme polygon from fme aggregate?You can find the photo attachedturquoise color object is fme aggregate .purple colored object is fme polygon .I want to cut below the yellow line.

goldsteinContributor

Add Timestamp to text file, written using FeatureWriter

Hi,I am trying to add a timestamp to a file, which is being written using feature writer. The workbench is on a shared data directory with a Flow Environment.I have seen other options of using fanout option but seems like they are limited to writers only.I just need a simple timestamp on my text-file when its written so that I can keep records of what is being written.I also have made a timestamp attribute using DateTimeStamper and DateTimeConvertor but don't know how to add that to the file name.Any pointers will be appreciated.Thank you

robinso1Contributor

Creating header lines

I’m attempting to automatically add three lines to the start of a processed data set, in order to create a batch file.There is probably a very simple solution but any help would be appreciatedThanks

jennychuContributor

gml_id and geometry issues in extending citygml 3

Hello everyone,I am a beginner with CityGML and am currently trying to convert a dataset to CityGML 3 with generics extensions using FME Workbench. However, I am encountering a couple of issues:Warning about inheritance relationship: The output gives a warning indicating no inheritance relationship. I have used AttributeMerge to add gml_parent_id, but the system does not seem to recognize it.<GML_2 Writer> - '12482' orphaned features found, XML comments representing these features will be present in the output file: Geometry export issue: It does not export correctly after setting up the geometry. In the Inspector, the data appears as a table without geometry.Additionally, the original dataset complies with the CityGML 3 standard, and its geometry's encoding is Name (encoded: UTF-16LE). Could this encoding be affecting the process?My settings and the transformer export are shown in the attached image.I would g

proteveContributor

Is it posible to create esri file geodatabe with DATASETS with Esri Geodatabase (File Geodb Open API)

Title!i’m trying to import shp files into a gdb database in separate dataset based on the folders that the shape files are located. i know i’ve read somewhere in the help to set dataset/feature name for feature name but its not workig.i get an error that its not possible without coordinate system. so i’ve added coordinate system to writer but i still get the same error.the error isFileGDB Writer: Cannot currently create feature datasets with unknown coordinate systems. Please specify a known coordinate system, or do not write to a feature dataset. Failed to create feature class 'B10/Shape_Z'so is it possible to write datasets in esri geodatabae with Esri Geodatabase (File Geodb Open API) writer? if so can you post an example.

Export FME workspace to a readable text or graphic format

Hello, I would like to export my FME workspace (see image below) in order to integrate it in the appendices of a technical report. Is there a way to do so apart the "Print"? Printing the workspace sadly does not work since it is quite large and would then become very small and unreadable. Any alternative? Can be either exported in a text format or a reduce graphic format...

crscrivenContributor

Migrating Sewer Utility Data from AutoCAD Civil 3D to Esri Utility Network

Hello everyone,I am currently working on migrating sewer utility data from an AutoCAD Civil 3D (2022) DWG file into the Esri Utility Network Foundation (Sewer) feature class/file geodatabase.Issue: The attribution for the DWG sewer layers is stored in tables that I need to join to my DWG/Feature class during the FME workflow by matching an ID field. However, the ID field in my DWG is not appearing in the reader I am using (Autodesk AutoCAD Civil 3D), not even as an “unexposed attribute”.I discovered a legacy .fmw file that successfully reads this field using a different reader (Autodesk AutoCAD Map 3D Object Data), which I do not have a license for.Questions: What is the necessary license for Autodesk AutoCAD Map 3D Object Data (aka AUTOCAD_OD)? Is it an FME-based license, or does it require an active AutoCAD Civil 3D license on my computer (which I currently don't have)? As an alternative, can anyone guide me on how to copy-paste the ID field that I can’t access in FME into a differ

mmccartContributor

Does FME 2024.0 Support Esri's New Big Integer Field Types When Creating New Dynamic Feature Classes?

I have been using Dynamic writers to create feature classes and tables in an Esri File Geodatabase for years. I normally use the FME field type “fme_int32” to create Long Integer field types. However, I need to create attributes using Esri’s new ‘Big Integer’ field type. I assumed I could use “fme_int64” but when I run my workspace in FME 2024.0, it runs without errors but it doesn’t create the new feature class. When I change the 2 fields from fme_int64 back to fme_int32, it runs without errors and creates the new feature class. I have ArcPro 3.2.2 installed which also supports the new Big Integer (64 Bit) field types.My question is: Should I be using a different FME Field type besides ‘fme_int64’ to generate Big Integer field types via a Dynamic FGDB Feature Class Writer?Thanks in advance.

fbastian73Contributor

FME Form to output KMZ in multiple folder format

Hi,I have an FME workbench that outputs a KMZ but i want to create a folder tree style for ease of view. Currently it outputs as below: I have been unsuccessfully trying to create a tree which will output something like this:Proper name instead of doc (folder) → operating_voltage (folder) → feeder_name (folder) → actual feeder spatial data .Screenshot of workbench below: The operating_voltage and the feeder_name coming from the attribute_renamers at the right end of the workbench.I tried using the instructions in below link, but it is not exactly the same as the attributes are in the data already and not being created for the purpose of the multi folder.Creating KML Folders Through KML IDs – FME Support Center (safe.com)Any ideas how this can be done?thank you,fb

stuarmitagehomeContributor

Extracting data from CLOB (cell/value) to create table of columns and values from that CLOB

Hi,I have a table (that started life as a JSON table, lists exposed etc) containing multiple attribute fields and rows. One of the fields contains CLOB data, like this:<cmAstLocSpId>148606829</cmAstLocSpId><criticalityDetails><criticalityReasons><criticalityReasonsList><criticalityReason>CMEH</criticalityReason></criticalityReasonsList></criticalityReasons></criticalityDetails><cmOsgridref>NZ30272359</cmOsgridref><cmOsnorthing>523598</cmOsnorthing><cmOseasting>430279</cmOseasting><cmRegion>TEESSIDE</cmRegion><cmRepairRegion>TEESSIDE</cmRepairRegion><cmDnoArea>NP (NE)</cmDnoArea><cmCreatedByUser>DAVID</cmCreatedByUser><cmCreationDate>2023-11-23</cmCreationDate> I want to turn the normal text into column / field headings, and those highlighted in bold into values in the relevant column. eg the first column will be “cmAstLocSpld” and t

neaContributor

IS MY 12 DAY OLD POST EVER GOING TO BE ANSWERED WITH SOMETHING THAT WORKS!

See 12 day old post. I appreciate the gamification of the user base but it is not a game when I have a critical, work FMW that that isn’t working. There has got to be something better than this support. Am I not entitled to some level of solution based, tech support for the software fees paid?

jfisch25Contributor

Read ArcGIS Online attachments and Maintain GlobalID's for both features and attachments

I'm trying to write AGOL features and attachments coming from a feature service tied to a Survey123 survey into a SQL Server geodatabase. I've looked through most of the tutorials and questions about this, but in my use case I need to maintain Global ID's on both the features and the attachments. The feature GlobalID's are being used to name photos that get stored in S3, so I can't use the examples that use a counter to create a new relational ID. I have set the "Maintain Global ID's" parameter on the AGOL reader and have exposed the AGOL attachment attributes. The attachments have a Global ID and Parent Global ID, and I figured it would be simple enough to use both of those to create the relationship class in the GDB since that's what the AGOL service is doing. When I query the service, I can see a feature's Global ID and the attachment's Global ID and its Parent Global ID (the GID of the feature it's attached to). Feature popup: Quer

virtualcitymattEvangelist

Overwriting EPSG definition in FME

So I’ve been trying to overwrite a coordinate system in FME, usually I work with EPSG codes. In FME most (if not all ) projections has an EPSG associated with them. When searching for a coordinate system you can just throw in the EPSG code. It’s great.I discovered that one of FME’s coordinate systems (EPSG:3004) by default uses a pretty poor translation to WGS84. I wanted to update the definition so the EPSG code would work better. I defined a new coordinate system in the hopes that when picking the EPSG code it would take my user defined system over the FME one. I found that the only way I could get the EPSG number to link to my new definition was if I actually gave it the same name as the one in FME Any other name and using the EPSG would always revert back to the old unprecise definition. In this case “MonteMario_1.Italy-2”. Of course now I get what looks to be duplicates in the list. Interestingly, no matter which you pick you always end up with the same coordinate system getting u

mr_fmeEnthusiast

Stop Workspace Execution

Hi,How can I stop the translation if one Tester in my workspace pass 0 features in the ‘Passed’ port? Thank´s

kyp4810Contributor

What version of citygml can I upload to fme?

Can fme only run up to citygml 2.0?I don't know the part about uploading citygml 3.0.

monitotierContributor

FME Errors when opening xml

I have FME v2024 with a Floating License. I have Windows 10 Enterprise installed. I am almost out of space on my disk where my "Temp" folder is located.The persistent problem I have with FME is that every time I try to open an XML file, although I do have the reader for it, I consistently get the same 6 errors. I have tried opening other files and there is no problem with those, only with XML files.[1]: HTTP transfer error: Timeout was reached[2]: Please ensure that your network connection is properly set up[3]: No proxy settings have been entered. If you require a proxy to access external URLs, please ensure the appropriate information has been entered.[4]:Failed to obtain any schemas from reader 'CITYGML' from 1 datasets. This may be due to invalid datasets or format accessibility issues due to licensing, dependencies, or module loading. See logfile for more information[5]:Workspace generation failed. Parameter file was C:\Users\User1\AppData\Local\Temp\FME_1717067304960_7084.Login

anteboy65Contributor

Tryging to read Postgres 14 with FME 2023 but getting "Failed to retrieve feature types"

I’m running FME Form 2023 and creating a new script with a reader wich fails when looking up tables. Message given: Failed to retrieve feature typesCould not open the enterprise geodatabase because the local version of arcgis is not compatible with the server's SDE version.On the laptop I have both Arcmap 10.7 and ArcGIS PRO 3.1 (AGP) installed. When creating the reader I use a connection file created in AGP. I also have FME 2018 but this is 2023. The reader is defined as “Esri Geodatabase (ArcSDE Geodb) / Embed Connection Parameters” as the geometry format in the database is ST_GEOMETRY. To me it looks like files from ArcGIS 10.7 is found and used which doesn’t work. Is there any way to force FME 2023 to use “library files” from AGP 3.1 or is there another way to solve the problem?

1
...
5
6
7
8
9
10
11
...
267

Page 8 / 267

Badge winners

marcusthas earned the badge Socializer (Narrator)
andybarkerhas earned the badge Student (Grey)
fdwhas earned the badge Student (Grey)
fdwhas earned the badge Expert (Grey)
fdwhas earned the badge Teacher (Bronze)

Show all badges

29,635: Posts

Powered by Gainsight

Terms & Conditions Cookie settings

Sign up

Already have an account? Login

An FME Account is required to contribute

Login

Login to the community

An FME Account is required to contribute

Login

Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.

Enter your e-mail address

Back to overview

Scanning file for viruses.

Sorry, we're still checking this file's contents to make sure it's safe to download. Please try again in a few minutes.

OK

This file cannot be downloaded

Sorry, our virus scanner detected that this file isn't safe to download.

OK