Skip to main content

- - Community Resources
Ideas
Product Updates
Events
Blog

EN

Home
Forums
FME Form
Data

Data

Using readers and writers to extract and load data.

6,719 Posts
24,748 Replies

When you subscribe we will email you when there is a new topic in this category

6719 Posts

Recently active

Most replies Most views Newest first

liseContributor

Joining two tables based on comparison of values

Hi, I am trying to do check if data is correct in one of my tables compared to another one. To do that, I thought about merging/joining the two tables based on columns and conditions. The common column between the two tables is the City Code (postcode). Then I need to compare landmark values, which represent the location of objects in space. I managed to join them based on the City Code column but I don’t manage to do the landmark values comparison yet. More context:My actual data is fields of grass along roads. Each section of road belongs to a city (City Code) and is separated into categories of landmarks. For example a road within city 1 is separated into groups of landmarks ranging from 1 to 1.5; 2 to 3; etc.; and the field X is situated at a specific landmark value 2.5.So the field should be in the landmark range 2 to 3 because its landmark value is higher than 2. The tables: The first table has 3 columns and links city code and landmarks to a unique ID: City code: the city pos

arash_hokmContributor

Write attributes to the DWG file

Hello FME community,When creating a DWG file I would like to write some attributes to it. It is important that the written attributes appear in the properties of the object.The following picture shows more precisely what I mean. I want Attribute 1 and Attribute 2 to be appended to the DWG file.Does anyone know how I can attach attributes to the DWG file.Thank you

gogopotter90Contributor

how to provide points/lines with unique constant id for each time of running FME?

Hello , I would like to generate constant unique id for lots of points or lines that has not get the id yet and keep the same constant unique for each time running FME (no change)i do not want any changing of id for each running Fme .lines or points should get constant unique id and after one month when I run Fme workspace and there are new lines or points .FME should provide the same constant unique id for old points or lines that provided them before and provide new constant unique id for new lines or points .The problem I am generating points that they are not exist before at GIS data and I am not able to save the points with id in Gis database but I want to keep the id for each point without changing in Fme thanks in advance fme 2021

marjan_nikoukarContributor

spatial join points based on one attribute in FME

Hi, I want to find duplicate addresses in the data based on duplicate address numbers.The way that I found in ArcGIS Pro is using spatial join of the layer with itself using attribute matching “address number”. This way if a point has a duplicate with the same address number in the same location, is joining with the point with the same address number and if a join count is more than 1 display, there is a duplicate. I am looking for a way to do the same step in FME. I used SpatialFilter but did not get a result. I really appreciate any suggestion for doing this analysis.Also, in ArcGIS Pro, it is possible to join spatially "within a distance". Is this option available in FME?

gogopotter90Contributor

Geometry validator can not detect the self intersection?

Hello ,i have self intersection at specific line .but Geometry validator did not catch it .I have read at other article that it is better to use chopper vertex 2 and snapper segment to let geometry validator to detect self intersection issue ?I have tried it but it does not work .is there other suggestion that I have to try ?FME 2021thanks in advance .

featuremichaelSupporter

IFC3DObject Transformation from Local to X,Y,Z!

Hello FME members,I have an IFC file with no coordinate information in the site feature.it is a 3dObject of a parcel that is supposed to represent the surface - but it is just a (3dObject) flat surface with an extrusion of 1]What I have done: I have at least 4 coordinates to transform, I tried to use:the 3daffinewarper, but the transformer showed me an error [because the transformation from 0,0,0 to x,x,444.1 is coplanar - and I only have 4 known points that went from z:1 to z:443] (I also tried 0,0,1 to x,x,444.1, but it didn't work) so I used the affinewraper to send the surface to the correct coordinate system)that worked!i need to know how to get the 3d object to the correct z value to display the surface (3d object) at the correct height.I hope someone can help me.Greetings Michael

lambertusEnthusiast

Generalizing centerlines without 'side branches'

Hi,I have a dataset with Centerlines shown in blue, some lines contain too much detail which is not accepted as input for the LineDivider I use in the downstream pipeline.Is there a way I can smooth the blue centerline into something like the red line shown on the right?The Generalizer should be the way to go I guess but did not find the right settings yet.

emerson_stylesContributor

Converting Ordnance Survey MasterMap data to DWG format

Has anyone successfully created a workspace that can convert Ordnance Survey MasterMap data to dwg format? I understand the process would be quite complex, but it would be good to get an idea of what’s involved from anyone who may have done this.

Always create excel file even if data is empty

I'm creating a new excel file nightly and some nights there will be no data to populate the excel file, but I need the file to be created anyways. Is there a step in the Excel writer that I'm missing to do this?

FeatureWriter silently dropping bad features and total features written count is wrong

Hi,I'm using a FeatureWriter to insert data into a PostgreSQL database. When certain features can't be converted into the desired data types, FME drops them without terminating the translation.The logger gives a message in blue:"Value of attribute 'test' could not be converted to type 'bool'. Feature will be logged and skipped" However, the issue lies in trying to capture this scenario and terminate the translation.Despite dropping some features, the FeatureWriter when outputting the summary feature still contains the FULL COUNT against the '_total_features_written' attribute. I was intending on comparing this number to a count of incoming features to check for a difference. Now the only other way I can think of is querying the database immediately after and getting the written feature count that way. I guess I have two questions:1. Is there an inbuilt way to halt the translation when a feature is dropped like this?2. Is the _total_features_wri

zentnerfabianContributor

Compare FileGeodatabase in FME?

Hi guys,is it possible to compare two filegeodatabases in FME? In each fgdb are 443 feature classes (points, lines and area feature classes). After some testing with the first vesion of the fgdb, we had to update the fgdb in some feature classes and changed some attribute fields and add some new domains and values. Is there any transformer which can show me the differences between both fgdb’s?Thanks for answering in advance.Cheers,Fabian

stevenjhContributor

WFS reader truncates response and fails

I'm using a FeatureReader to read a wfs service and getting strange failures.The following error is being loggedHTTP transfer error: 'Failure when receiving data from the peer'XML Parser error: 'Error in input dataset:'https://data.linz.govt.nz:443/services;key=xxxxxxxxxxxxxx/wfs/layer-53382?SERVICE=WFS&VERSION=2.0.0&REQUEST=GetFeature&TYPENAMES=data.linz.govt.nz%3Alayer-53382&NAMESPACES=xmlns%28data.linz.govt.nz%2Chttp%3A%2F%2Fdata.linz.govt.nz%29&BBOX=-36.939824852066955%2C174.70954193086246%2C-36.91494916200109%2C174.76090622708065' line:1 column:121683 message:unexpected end of input'The XML Module halted on error, see the logfile for details<WFS> An error occurred while reading a GetFeature response, to ignore the error and avoid losing the previously read features, set the WFS "Ignore GetFeature Error" parameter. Total features downloaded: '0'A fatal error has occurred. Check the logfile above for detailsAccessing the link and checking the file manually

proteveContributor

after exporting shp to gdb features are missing from gdb

I have manage to export shp in file geodatabase with datasets but now it seems that i have a strange problem (or bug).every shp that i have exported is missing 1st entry in geodatabase. lets say a shp file has 10 points then i’ll get only 9 points in gdb and the one is missing is always the 1st pont in shp file. every shp file is missing that 1st entry.the strange thing is that fme shows in information that it wrote all entries that the shp files has but somehow 1 is missing.there is absolutly no error in logs. i get some info about some fields that i don’t export anyway. i really have no idea why this is happening.the shps are also exported with FME from othe types of data.for writer i use Esri Geodatabase (File Geodb Open API) with some schemas that i read from a template database with featurereader and join it on shps (no schema was readed).so what im doing wrong?why there is no error about the missing entries?

ienesaglam60Contributor

Cut the fme polygon from fme aggregate

How ı Cut the fme polygon from fme aggregate?You can find the photo attachedturquoise color object is fme aggregate .purple colored object is fme polygon .I want to cut below the yellow line.

goldsteinContributor

Add Timestamp to text file, written using FeatureWriter

Hi,I am trying to add a timestamp to a file, which is being written using feature writer. The workbench is on a shared data directory with a Flow Environment.I have seen other options of using fanout option but seems like they are limited to writers only.I just need a simple timestamp on my text-file when its written so that I can keep records of what is being written.I also have made a timestamp attribute using DateTimeStamper and DateTimeConvertor but don't know how to add that to the file name.Any pointers will be appreciated.Thank you

robinso1Contributor

Creating header lines

I’m attempting to automatically add three lines to the start of a processed data set, in order to create a batch file.There is probably a very simple solution but any help would be appreciatedThanks

jennychuContributor

gml_id and geometry issues in extending citygml 3

Hello everyone,I am a beginner with CityGML and am currently trying to convert a dataset to CityGML 3 with generics extensions using FME Workbench. However, I am encountering a couple of issues:Warning about inheritance relationship: The output gives a warning indicating no inheritance relationship. I have used AttributeMerge to add gml_parent_id, but the system does not seem to recognize it.<GML_2 Writer> - '12482' orphaned features found, XML comments representing these features will be present in the output file: Geometry export issue: It does not export correctly after setting up the geometry. In the Inspector, the data appears as a table without geometry.Additionally, the original dataset complies with the CityGML 3 standard, and its geometry's encoding is Name (encoded: UTF-16LE). Could this encoding be affecting the process?My settings and the transformer export are shown in the attached image.I would g

proteveContributor

Is it posible to create esri file geodatabe with DATASETS with Esri Geodatabase (File Geodb Open API)

Title!i’m trying to import shp files into a gdb database in separate dataset based on the folders that the shape files are located. i know i’ve read somewhere in the help to set dataset/feature name for feature name but its not workig.i get an error that its not possible without coordinate system. so i’ve added coordinate system to writer but i still get the same error.the error isFileGDB Writer: Cannot currently create feature datasets with unknown coordinate systems. Please specify a known coordinate system, or do not write to a feature dataset. Failed to create feature class 'B10/Shape_Z'so is it possible to write datasets in esri geodatabae with Esri Geodatabase (File Geodb Open API) writer? if so can you post an example.

Export FME workspace to a readable text or graphic format

Hello, I would like to export my FME workspace (see image below) in order to integrate it in the appendices of a technical report. Is there a way to do so apart the "Print"? Printing the workspace sadly does not work since it is quite large and would then become very small and unreadable. Any alternative? Can be either exported in a text format or a reduce graphic format...

crscrivenContributor

Migrating Sewer Utility Data from AutoCAD Civil 3D to Esri Utility Network

Hello everyone,I am currently working on migrating sewer utility data from an AutoCAD Civil 3D (2022) DWG file into the Esri Utility Network Foundation (Sewer) feature class/file geodatabase.Issue: The attribution for the DWG sewer layers is stored in tables that I need to join to my DWG/Feature class during the FME workflow by matching an ID field. However, the ID field in my DWG is not appearing in the reader I am using (Autodesk AutoCAD Civil 3D), not even as an “unexposed attribute”.I discovered a legacy .fmw file that successfully reads this field using a different reader (Autodesk AutoCAD Map 3D Object Data), which I do not have a license for.Questions: What is the necessary license for Autodesk AutoCAD Map 3D Object Data (aka AUTOCAD_OD)? Is it an FME-based license, or does it require an active AutoCAD Civil 3D license on my computer (which I currently don't have)? As an alternative, can anyone guide me on how to copy-paste the ID field that I can’t access in FME into a differ

mmccartContributor

Does FME 2024.0 Support Esri's New Big Integer Field Types When Creating New Dynamic Feature Classes?

I have been using Dynamic writers to create feature classes and tables in an Esri File Geodatabase for years. I normally use the FME field type “fme_int32” to create Long Integer field types. However, I need to create attributes using Esri’s new ‘Big Integer’ field type. I assumed I could use “fme_int64” but when I run my workspace in FME 2024.0, it runs without errors but it doesn’t create the new feature class. When I change the 2 fields from fme_int64 back to fme_int32, it runs without errors and creates the new feature class. I have ArcPro 3.2.2 installed which also supports the new Big Integer (64 Bit) field types.My question is: Should I be using a different FME Field type besides ‘fme_int64’ to generate Big Integer field types via a Dynamic FGDB Feature Class Writer?Thanks in advance.

fbastian73Contributor

FME Form to output KMZ in multiple folder format

Hi,I have an FME workbench that outputs a KMZ but i want to create a folder tree style for ease of view. Currently it outputs as below: I have been unsuccessfully trying to create a tree which will output something like this:Proper name instead of doc (folder) → operating_voltage (folder) → feeder_name (folder) → actual feeder spatial data .Screenshot of workbench below: The operating_voltage and the feeder_name coming from the attribute_renamers at the right end of the workbench.I tried using the instructions in below link, but it is not exactly the same as the attributes are in the data already and not being created for the purpose of the multi folder.Creating KML Folders Through KML IDs – FME Support Center (safe.com)Any ideas how this can be done?thank you,fb

stuarmitagehomeContributor

Extracting data from CLOB (cell/value) to create table of columns and values from that CLOB

Hi,I have a table (that started life as a JSON table, lists exposed etc) containing multiple attribute fields and rows. One of the fields contains CLOB data, like this:<cmAstLocSpId>148606829</cmAstLocSpId><criticalityDetails><criticalityReasons><criticalityReasonsList><criticalityReason>CMEH</criticalityReason></criticalityReasonsList></criticalityReasons></criticalityDetails><cmOsgridref>NZ30272359</cmOsgridref><cmOsnorthing>523598</cmOsnorthing><cmOseasting>430279</cmOseasting><cmRegion>TEESSIDE</cmRegion><cmRepairRegion>TEESSIDE</cmRepairRegion><cmDnoArea>NP (NE)</cmDnoArea><cmCreatedByUser>DAVID</cmCreatedByUser><cmCreationDate>2023-11-23</cmCreationDate> I want to turn the normal text into column / field headings, and those highlighted in bold into values in the relevant column. eg the first column will be “cmAstLocSpld” and t

neaContributor

IS MY 12 DAY OLD POST EVER GOING TO BE ANSWERED WITH SOMETHING THAT WORKS!

See 12 day old post. I appreciate the gamification of the user base but it is not a game when I have a critical, work FMW that that isn’t working. There has got to be something better than this support. Am I not entitled to some level of solution based, tech support for the software fees paid?

jfisch25Contributor

Read ArcGIS Online attachments and Maintain GlobalID's for both features and attachments

I'm trying to write AGOL features and attachments coming from a feature service tied to a Survey123 survey into a SQL Server geodatabase. I've looked through most of the tutorials and questions about this, but in my use case I need to maintain Global ID's on both the features and the attachments. The feature GlobalID's are being used to name photos that get stored in S3, so I can't use the examples that use a counter to create a new relational ID. I have set the "Maintain Global ID's" parameter on the AGOL reader and have exposed the AGOL attachment attributes. The attachments have a Global ID and Parent Global ID, and I figured it would be simple enough to use both of those to create the relationship class in the GDB since that's what the AGOL service is doing. When I query the service, I can see a feature's Global ID and the attachment's Global ID and its Parent Global ID (the GID of the feature it's attached to). Feature popup: Quer

1
...
7
8
9
10
11
12
13
...
269

Page 10 / 269

Badge winners

kanjaniheena59has earned the badge FME Flow Authoring Training
daraghmccarthyhas earned the badge Student (Grey)
akohas earned the badge Welcome Back
ericwhuhas earned the badge English Speaker
ericwhuhas earned the badge Chinese Speaker

Show all badges

29,836: Posts

Powered by Gainsight

Terms & Conditions Cookie settings

Sign up

Already have an account? Login

An FME Account is required to contribute

Login

Login to the community

An FME Account is required to contribute

Login

Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.

Enter your e-mail address

Back to overview

Scanning file for viruses.

Sorry, we're still checking this file's contents to make sure it's safe to download. Please try again in a few minutes.

OK

This file cannot be downloaded

Sorry, our virus scanner detected that this file isn't safe to download.

OK