- - Community Resources
Ideas
Product Updates
Events
Blog

Home
Forums
FME Form
Data

Data

Using readers and writers to extract and load data.

6,810 Posts
25,084 Replies

When you subscribe we will email you when there is a new topic in this category

6810 Posts

Read all tables from POSTGIS schema

I need to read all tables from Postgis scehma. With reader I got only tables which has geometry or which has tables related to geometry tables with foreign keys.Is there some option to read them all, or some Python script which can help me with this?

GUID Generator assigning Primary key automatically

When using the GUID generator i would like to assign that column as a primary key without opening the writer and assigning it? Is there a transformer i can use for this? (Would be a great addition to that transformer.)

overwrite existing attribute

I have a .dxf with points and lines within different layers and blocks. For one layer of data, I need to delete the attribute value within the block and insert in it the geometric value z. Which transformer is best to use?

Converting KML generated from Google Earth to Cesium 3D Tiles

I follow step 1 of https://knowledge.safe.com/questions/101352/write-b3dm-from-kmz-with-textures-to-cesium-3d-til.html, selected 'Placemark' feature and run.It generated 'tileset.json' and 'data/data0.b3dm' and I tried loading it in Cesium but I couldn't see anything. What is the right way to do to convert KML to 3D tiles?

Converting AGS Files as a GIS Layer

Hi all,Just been on working on converting AGS Files to GIS format.This is the first time I have gone through this and it is a difficult file to convert.Just wondered if anyone has any experience with this and are there any steps that have to be done before using FME to convert.@daveatsafe - I have seen a similar thread where you have done some fantastic work building the AGS FME file. I have played around with it but struggle with the output. Below is the FME file:I have also used the AGS transformer which FME now has but not getting the output I was after.

Add to Multiple Feature Writers at once?

I'm going from ESRI Shape Files to PostGIS and trying to add a primary key to all my tables so that Mapserver and other queries are happier with the unique Primary Key. I see from https://knowledge.safe.com/idea/74374/postgres-writer-add-unique-primary-option-for-inde.html that I can add a Serial and that works well but I'd like to add that to 20, 30, or even 50 shape files at a time. Is it possible to do this as a batch?Thanks!

140

4 years ago

geo-xParticipant

asked in Data

Compare and update Json into DB

Dear FME community.I would like to compare to database who contains json (with subobjects).The aim is to UPDATE, DELETE or INSERT json into the target DBInforamtion : the base 2 may be to json format or "classical" attributes format.What is the better way to do this ?Thanks.

470

4 years ago

jorge_vidinhaContributor

asked in Data

POSTGIS Reader different geom column

Is there any possibility to change the default geometry column 'geom' on postgis readers to some other column with a different geometry like geom2 ?Thanks

Hi Guys, this might be a simple question - I am writing data to a TAB file. every time I run the workspace it over writes the TAB file with the new data. How do I append data being written to the TAB file.Dave

I I am trying to convert the attached shape file to topojson. it is a shapefile of points. It seems some of the points are causing a problem when converting because I can take a sample and create the topojson file. I can't seem to figure out the problem with the offending points. Appreciate any help. Thanks.David

4 years ago

mamaieron

asked in Data

Unexpected SurfaceOnSurfaceOverlayer ouput

Hello everyone, I am converting CAD data to CityGML. To create the LoD2 class of buildings with hierarchical integrity, I used GeometryCoercer to transform the extruded solids into surfaces, followed by the Deaggregator, separating the roofs, walls and grounds. I tried to use SurfaceOnSurfaceOverlayer to identify shared walls between buildings but the transformer is not identifying all the cases. When I set the tolerance, it identify more cases but it ends up disturbing the geometry of the surfaces, generating non-flat surfaces, with duplicate points, etc ... all verified by GeometryValidator and by CityGML validation tools like Val3dity. As you can see in the prints, a case of shared surface is identified (Surface 1 and 2) and another one with the same characteristics (Surface 3 and 4) no. As I used the vertical direction in the Extruder and have the same X and Y coordinates, they should be coplanar- I also used the PlanarityFilter before the process and the value of the _surfaceNorma

120

4 years ago

dms2Contributor

asked in Data

GML reader and 'cannot convert null string into an integer' error

We recently upgraded FME Desktop 2007 to 2020. Now I'm upgrading all of our old workspaces and I've having an issue after upgrading GML reader to the latest version.GML files are read without any error with the old reader but with the new version I get the following error for every feature in every GML file:Cannot convert null string into an integer class StatusInfo *__cdecl STF_stringToUInt(const class ObsoleteString &,unsigned long &)(C:\\code\\fme\\foundation\\framework\\util\\stf\\stfutil.cpp:358) - class StatusInfo *__cdecl STF_stringToUInt(const class ObsoleteString &,unsigned long &)(C:\\code\\fme\\foundation\\framework\\util\\stf\\stfutil.cpp:358)I checked the GML version for all my files in the xsd file and they all say it's GML 3.1.1. Didn't see any limitation on this in the reader's help.Any idea of what's going on?Edit: the old GML reader works as expected before saving my workspace with Desktop 2020, but the error is there after sav

692

4 years ago

rudystricklan1Contributor

asked in Data

Read AutoCAD files to be processed from a text file

I have a text file that contains an AutoCAD pathname on each line, e.g., D:\\project4033\\dwg\\p1234.dwg D:\\project515\\dwg\\p642.dwg X:\\dwg\\a556.dwgHow can I set up a Reader to process each file (the output format will be Adobe GeospatialPDF)? I see answers like a "TextFileReader", but on my 2020.1 system there is no such transformer.

4 years ago

groschialeux

asked in Data

How to order tags in OSM Writer

I am working on a workspace that reads Tourism Information Office data from an XML file, converts the attributes to OSM tags and writes it to an OSM file. This OSM file is then processed by a program called OsmAndMapCreator to create an obf (OsmAnd Binary Format) file for the OsmAnd mobile application. Each office has a collection of characteristics and services so there is more than 1 tag for each node. The order in which those tags are written to the OSM file is important for how the feature will appear on the map. A tourism information office that has parking or picnic tables still needs to appear as a tourism information office no matter what else they offer. The OSM writer does not maintain the order of its reader or User Attributes list. In my reader and writer the tourism tag is at the top but for some reason, it is always the last one in my OSM file. Here's 2 nodes where it creates a problem.This node shows up as a picnic table on my map but it should appear has a tou

How to convert OpenDrive (.xml) to 3D SHP

Is it possible for FME to convert OpenDrive (.xml) to 3D SHP?

BIM (revit) to Cesium3dTiles, Reprojection Failed

Hi everyone,I have a BIM (revit file) that i would like to transform to Cesium3Dtiles. I used the "Revit Geolocation" Tutorial of @daveatsafe and exported the model into IFC. When I look in the FME Data Inspector the IFC file looks fine (I think). But when I add a Cesium3Dtiles writer I get an reprojection error. The coordinate system of the IFC file is EPSG:3857 and the coordinate system of the Cesium3Dtiles schould also be EPSG:3857. So I don't understand why there is a problem with the reprojector.

478

4 years ago

dms2Contributor

asked in Data

How to handle a geopackage received as base64 binary

I need to publish a workspace to fme server that will receive a geopackage file as base64 binary and will do some schema and data validation before inserting geopackage data into a number of Oracle tables. Actually an API will call the FME service providing that base64 binary and some other information.Now my question is how to read geopackage schema and data into fme. Any idea? I don't see how to get the binary back into a geopackage file.Thanks!

GoogleBigQueryConnector Query returns no results

I've installed the Google BigQuery package into 2020.1 and I've got a web connection to a BigQuery project. The connection seems to work since I can see the dataset and tables listed. I am sending a simple SQL 'SELECT * FROM...' and I've tried 'SELECT col1, col2 FROM ...' I want the query to return records from a non-spatial BigQuery table to use in FME. I don't get any results and the log doesn't seem to give anything helpful. We've opened all the permissions on the account. Does anyone have any advice on how to get this to work?

Ignore old files with ProjectWise

Hello! I'm reaching out because I'm in aa bit of a pickle. I have a task to pull a large number of photos and folders from a ProjectWise folder. I've successfully been able to do so through the ProjectWiseWSGConnector and FeatureReader, and then a number of other transformers that modifies the images for what we need. However, because the number of photos is quite large (GB's of data), it's quite the task to run it and to re-download all of the files. Is there a workflow that will skip over previously downloaded files and just download the newly added or updated ones? Thanks for any help!

ProjectWise: Version and Status not supported?

Hello,I am trying to get the information on files stored in ProjectWise. I am using FME 2020.1.However, the ProjectWiseConnector, seems to be unable to get all attributes, i.e. Version and Status.Also tried custom ProjectWiseNavigationHierarchyRetriver, which looks to be able to work with all attributes, but only works with API v.1.2 and 1.3 and we are on 2.4.Are there any other transformers or a newer version of the above that could read all file attributes?Many thanksMagda

250

4 years ago

jackydContributor

asked in Data

remove feature dataset from data when writing to FGDB Open API dynamically

Hi, I am trying the write out using the to a file geodatabase using a feature writer but am unable to remove the feature datasets from the data. I am using OPEN API as i am reading in all the fgdbs in one directory and writing them to a new location and splitting the data back to the original fgdb names, but don't want the feature data sets to be used (one of the main reasons for doing this) and i can not get the feature writer to writer to different fgdbs.Below is the basic work space so far, the schema reader reads all the feature classes in the fgdbs then gets the feature reader to read those tables and send them to be saved in the new location by the feature writer. This will be the first stage of a data update process, eventually the workbench will read new data that has come in, compare it with a previous version and then update the file gdb with the changes (along with details of the change user, date and source). Each time data is delivered there could all or just some of t

340

4 years ago

Anonymous

asked in Data

Build surface/solid from lines

I've been given a sample from a CityGML dataset that I want to eventually convert into an FBX 3d model. Now, before receiving this sample, I had built a workflow that did the entire conversion from a CityGML dataset into FBX models, tested it out with different CityGML datasets from open data repositories and it worked just fine. But in all of those datasets, the geometry was made up of surfaces, while on this sample there are just lines.I've been messing around with the AreaBuilder transformer but even though some of the surfaces are generated, I can't seem to get it nowhere near what it should be like.Any ideas?The sample: https://drive.google.com/drive/folders/1JAKSgxgjPMRfwJoDgCaKPValeaZRmO88

550

4 years ago

brandonguoContributor

asked in Data

SQL table generate by FME not visible in QGIS

I uploaded a table into MSSQL server, but the table is not visible in QGIS.There is no issue viewing the table in SQL Server Management Studio, and the primary key is generated by counter.So far, I tried turn Spatial Index on and off; Change the collection type handling; Change Insert into Identify Column on/off. None of those steps work.Any idea how to modify the writer to let the table visible in QGIS? Update:In ArcGIS Pro, the table is visible, but can't add to map, gives an error message.

Important: Does the Geocoder (ArcGIS Online) consume Credits?

Is it free to use or does it consume credits?

Trouble converting a 2019 DWG to File Geodatabase with Annotations

I have cad file I am trying to convert to a File Geodatabase. The CAD file has labels that I would like to convert to geodatabase annotations. When I generate a workspace specifying the DWG and output as File geodatabase, I get a few features that are empty and a blank gdb table. Any ideas on how to do this? Thanks! Attached is the cad file and a screenshot of the workspace I have.

4 years ago

Page 65 / 273

Badge winners

corentin_bergehas earned the badge Welcome Back
corentin_bergehas earned the badge FME Form Basic Training
vjanardhanhas earned the badge Welcome Back
candertonhas earned the badge Student (Grey)
edwardhas earned the badge FME Form Basic Training

Show all badges

30,363: Posts

Terms & Conditions Cookie settings

Sign up

Already have an account? Login

An FME Account is required to contribute

Login to the community

An FME Account is required to contribute

Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.

Enter your e-mail address

Back to overview

Scanning file for viruses.

Sorry, we're still checking this file's contents to make sure it's safe to download. Please try again in a few minutes.

This file cannot be downloaded

Sorry, our virus scanner detected that this file isn't safe to download.

A product of

Company Products Giving Back Careers Contact Privacy Feedback

Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.