- - Community Resources
Ideas
Product Updates
Events
Blog

Home
Forums
FME Form
Data

Data

Using readers and writers to extract and load data.

6,701 Posts
24,707 Replies

When you subscribe we will email you when there is a new topic in this category

6701 Posts

Recently active

Most replies Most views Newest first

ebedmar

asked in Data

Get snapping tolerance of a Microstation file v.8

I am reading a Microstation v.8 des file from FME. The file contains lines and I am trying to generate polygons with AreaBuilder. Sometimes I can't generate the areas since the lines don't touch. I think it is a snapping problem. How can I get the snapping tolerance defined in the Microstation v.8 file that I am reading in FME? Thank you.

4 years ago

adrian_farrellContributor

asked in Data

Oracle Non-Spatial Reader 8000 character limit

Hi Trying to read an Oracle table containing one (Oracle limit) "Long" datatype field The field only brings through the first 8000 characters into FME (these are typically upwards of 9800 characters in this DB) As an aside into Access (ODBC) it reads more (8135) but not all of the field Does anyone have any idea how to bypass this limit and bring in the huge field? Thanks Adrian

4 years ago

jasperwisEnthusiast

asked in Data

IFC to Cesium keeping all individual elements

Hi,I would like to convert an IFC model to a format which is accepted by Cesium, but with respect to alloriginal individual elements. I am not looking to write everything to one single OBJ or DAE. The goal is to have every element selectable in Cesium.I know there is this documented workflow: https://www.mdpi.com/2220-9964/7/10/393/htm But I was wondering if there is a solution in FME. Anyone? Jasper

4 years ago

jpvo

asked in Data

Revit export IFC reader not working on fmecloud

The reader IFC with data view for Revit greturns an error message when used on fme cloud:Could not load module '/opt/fme-engine-2019/plugins/writer_keyword.so' : /opt/fme-engine-2019/plugins/writer_keyword.so: cannot open shared object file: No such file or directoryIt's working just fine on desktop FME(R) 2019.1.2.0 (20190829 - Build 19630 - WIN64)But not on FME Server 2019.1.2 Build 19630 - linux-x64Any idea?Cheers

4 years ago

aarongeppertContributor

asked in Data

Failed to parse SQL Server version

Our IT recently upgraded our Microsoft SQL Server to version 13.0.5026.0; previous version was 11.0.7462.6. Once in place, FME Workbench is reporting the following error in the logs during read of the new SQL Server. Unfortunately, we are still running FME Workbench 2013 and cannot install a newer build at this time. "MS SQL Server Reader: Failed to parse SQL Server version `13.0.5026.0'. FME may mishandle geography, dates, and times"Is there any documentation as to which MS SQL Server versions are compatible with each FME Workbench build or readers?

4 years ago

scarterContributor

asked in Data

Convert OS Mastermap GMZ to 500m2 .dwg tile files

I can see how I can covert OS Mastermap GMZ files (zipped GML) to .dwg. However is it possble to split the .dwg output into clean cut 500 metre squre tiles?

4 years ago

arnold_bijlsmaContributor

asked in Data

CSV file with comma-space as separator -> attribute name starting with space

My input CSV file has comma-space as separator (see attached). On the CSV reader, there doesn't seem to be an option to separate two characters, so I read it in by specifying just a comma. For the data this is not an issue as I can just trim the values. However, for the header line containing the attribute names it is a problem, because all my attribute names (bar the first one) are now prefixed by a space, and hardly any transformer can handle such attribute names. I can't use the AttributeManager or AttributeCopier or AttributeRenamer or ExpressionEvaluator, as they all automatically strip away the starting space from the name, and then they turn red for being an unrecognised attribute name. The only exception is the BulkAttributeRenamer, which allows me to strip away the prefixed space character, so I do have a workaround.Is there a way of setting comma-space as separator?Should I report it as a bug that all these commonly used transformers are unable to handle an attribute

4 years ago

jafrullahkklr

asked in Data

How to I extract ground co-ordinates from a raster ?

How to I extract ground co-ordinates from a raster ? I am able to extract only Row and column using co-ordinate extractor. I wanted to write ground co-ordinates in DMS format.

4 years ago

vanawful

asked in Data

Non-consistant text to json

Hello, came across Safe Software while researching options to convert text files into json. Some of the files I have are non-consistant. That is, beyond the date/time, each entry can vary from one line to n number of lines before the next entry. Only constant is the first line of each entry has date/time in this format:2019-06-05 07:31:49.968There may be more data after the date/time on the same line. A delimiter is used to separate data fields, however, there are cases in which the delimiter is not written to the file after a line return. In this case the next line starts with no delimiter so the first field could be wrongly seen as belonging to the previous.Anyhow, any one have experience with converting text files like this using any of the Safe Software tools?

4 years ago

cheffing

asked in Data

How to flatten nested JSON?

Hi,I tried several times to get a flat table of this JSON-File: . It comes out of an API-Request and has this structure:Target is a flat table with this columns to be able to join it with the corresponding shapefile:stundebstidheighttimesliceWIND_6HRISK_WINDBREAKAGE_BEECH_MAX_6HRISK_WINDBREAKAGE_SPRUCE_MAX_6HPROB_GUST_100kmh_MAX_6HPROB_GUST_75kmh_MAX_6HPROB_GUST_60kmh_MAX_6HThought about something like this within the JSON-Extractor:But can't get it to work like I expected.Looking forward to your help.Best regards, Christoph

4 years ago

vajnorcan

asked in Data

GEODATABASE_SDE reader: datetime2 field values converted as date

Hi all, I'm trying to detect the changes between the on premise data (SQL Server 2017) and AGOL (ArcGIS Online) data. The schema is the same, of course the field types are slightly different. I've noticed that ChangeDetector is constantly reporting a large number of changes, and closer inspection highlighted the problem, that the GEODATABASE_SDE reader into the sql server is reporting that the datime2 field type (CreatedAt) is of type date in FME reader, and is actually truncating the data into the date only. So if let's consider a single record:FME reads it from AGOL: 20190101000000 from DB: 20190101SQL Server reads it as 2019-01-01 00:00:00.0000000 Hence reporting is as a change. How can I force the reader to use datetime2 instead of just date?

4 years ago

kimburrows1242

asked in Data

Convert GDB to Mapinfo

I'm trying to write a large Geodatabase into TAB format for a client. I can see a direct read and write but it doesn't hold attributes. There doesn't seem to be many parameters on the Mapinfo writer and weirdly hardly any info online. It seems a lot of info on conversions FROM Mapinfo but not the other way around. Data is confidential so I can't share it, but it's pretty basic bunch of feature classes.

4 years ago

weskerjiang

asked in Data

How to convert 2D pipeline (shapefile or Esri File Geodatabase) to 3D pipeline

Hello, I want to convert 2D pipeline(shapefile or Esri File Geodatabase) to 3D pipleline to put it into cesium,Do you have any idea?Thanks

4 years ago

runnealsContributor

asked in Data

Truncate ArcGIS Online View

I have a hosted feature view (that I set the view definition for) on ArcGIS Online that is linked to a hosted feature service. When I add the hosted feature view as the writer in FME and set Truncate First to YES, it truncates/deletes all the data in the hosted feature service and not just the data that is in the view. Is there a way to get around this, as I expect when I truncate a view, it would only truncate the data that is visible in the view. @carsonlam

4 years ago

iverson

asked in Data

FME Desktop convert IFC to Cesium 3D Tiles

Hi, I'm using the FME Desktop to convert IFC to Cesium 3D Tiles, and the convertion was successful, but when I used the Cesium Client to present the 3D Tiles, it was not showing, and always show the message on the console:p.s. my IFC file is in EPSG:4978 Coordinate System

4 years ago

cshir065

asked in Data

Is there a way to keep the symbology from a DWG when writing to AGOL?

Hi all,I'm trying to write a few layers from a .DWG to ArcGIS Online. Is there a way to keep the symbology of the DWG when writing to AGOL? Or, is there a transformer that can change the symbology of a line/point and then write to AGOL?Thanks!

4 years ago

bubblebebContributor

asked in Data

Name Access output with Attribute

I am writing out to Access (JDBC) and want to use an attribute created in the routine for the filename. I have tried entering @Value(Filename) in the Database Path of the writer but it just creates a database called @Value(Filename).mdb Any ideas how I can get the attribute as the filename. The only option I have found is to use a user parameter to do it but I want a date stamp as part of the filename which is created in the routine. Thanks

4 years ago

mukhtargContributor

asked in Data

Excel Formula Not Read in Dynamic Report

Hello,I have a shapefile whose report I am trying to generate by pivoting. The result, I am able to get on different sheets. But the output is not calculating the formulas and hence I am not able to calculate summary for all the sheets. The template has only one sheet as a reference. It also has some formulas which calculate the summary. When the reporting is done for the single file it reads the formula and generates the summary as well.Attached herewith is the sample workbench and the template file to refer to. Any help in this regard is highly appreciated.Thanks.

4 years ago

mehar

asked in Data

how to set google sheet reader automatically to import fresh data

i am using google sheet reader and writing it to the database.. but the problem is there are more than 10 readers for google sheets which are are updating each and every hour .so if i just run the workspace it is not writing the fresh (updated google sheet) instead the same old one so i have to do update reader for every time for each reader to write the updated sheet please is there any solution to do it automatic after i run the workspace it will read the fresh data updated in google sheet..

4 years ago

stu787

asked in Data

FME desktop 2018 Proxy NTLM Authentication

Hi,I'm attempting to connect to an AWS S3 bucket from FME desktop (2018.0). (Colleagues have informed me that the situation is no different in FME 2019). The system proxy settings are controlled via a PAC file as part of group policy. I know that the proxy is NTLM, and as far as I can tell there is no way that FME can pass on authentication credentials correctly and so connections are always refused. I have tried the various settings within tools>FMEoptions>Network Proxy> but I always receive the same connection error -'Unable to execute HTTP request: 'I can navigate to the bucket via the browser with no problems. As at here https://knowledge.safe.com/questions/31838/s3uploader-and-network-proxy.html is it simply the case that the S3 tools do not work with an NTLM proxy? Is the only alternative (in my case to pay for) a proxy authentication bypass?

4 years ago

gauripearl

asked in Data

Which are the transformers for migrating data from Oracle to MongoDB?

While migrating data from oracle to mongodb, I am trying to use JSON reader and writer. I need to know if instead of using json reader and writer both is there any simpler way to do this.

Hi,I do not know what happend to my schema, but I need to update a reader all the time when I want to run the workspace. Before I did not need to do this. The schema and Data for reader are always the same ( just name od dataset-[not feature type name], and content are different).I had to upload a new dataset into a reader and that was all.Now, I need to Update reader AND feature types every single operation. Otherwise read the reader only a few feature types.I need to automate this schema in FME Server, but it will not work when I need to update a reader all the time. Does anybody has any idea why this happened?I did not make any changes in my workspace. Thanks

4 years ago

scarecrowContributor

asked in Data

Creating a DEM from points

Hello,I have a dgn file containing a 3D model of a ground surface. From the dgn I created a 3D polygon shapefile, extracted the vertices to points and assigned Z values to the points. I would like to interpolate between the points I extracted to create a DEM. This process works well in ArcGIS, using the Create TIN tool and I'm able to create a DEM raster that closely resembles the original 3D model. I've been trying to replicate this process in FME but I can't seem to generate a TIN similar to that of the original model. I've tried using TINgenerator, DEMgenerator, RasterDEMGenerator and Surfacemodeller. With all of these methods it appears to interpolate a surface but the faces are missing (looks like a greenhouse with the glass taken out). Or the interpolation is so course that it creates huge triangles, which is something that the Create TIN tool in ArcGIS appears to overcome. Is there away that I can generate a TIN with solid faces and a higher resolution of interp

4 years ago

reidm35

asked in Data

Problems with converting a fgdb polygon feature class to a KML

Hello everyone, As said in the title, I'm having trouble converting an SDE Polygon feature class to a KML file in FME 2017. When I do the translation on FME, I can see the geometry and attribute in the inspector, but when I add the KML to Google earth, there is no geometry! I was also using a KML Property Setter and KML Styler, but that shouldn't have any effect. I've made sure to check the coordinate system, and I've re-run the workspace In many different variations, and even tried doing the conversion in Arc, but no results for getting the KML polygon. Please help! One thing I've noticed is that the polygon KML is a very large file when compared to the other KML's generated in the workspace, both of which have many, many more features. The data that is not 'working' is the writer ending in '_Area'

4 years ago

garnet

asked in Data

Layers in DWG file. Create a list of them and UTF-8 issue.

Hi all, I have a dwg file with a lot of layers. Some of the layers are named in Latin alphabet some are not. The ones that are not are shown like this \\U+041A\\U+043E...............I have two questions:1. How can I see the layer names properly?2. How can I create a list of all layers, in the format dwgname,layername and write out as csv file?

4 years ago

Page 94 / 269

Badge winners

hermanlouiehas earned the badge Welcome Back
guygurhas earned the badge Student (Grey)
jerhas earned the badge Welcome Back
niall_mchas earned the badge FME Form Basic Training
will.jones4has earned the badge Welcome Back

Show all badges

29,730: Posts

Terms & Conditions Cookie settings

Sign up

Already have an account? Login

An FME Account is required to contribute

Login to the community

An FME Account is required to contribute

Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.

Enter your e-mail address

Back to overview

Scanning file for viruses.

Sorry, we're still checking this file's contents to make sure it's safe to download. Please try again in a few minutes.

This file cannot be downloaded

Sorry, our virus scanner detected that this file isn't safe to download.