- - Community Resources
Ideas
Product Updates
Events
Blog

Home
Forums
FME Form
Data

Data

Using readers and writers to extract and load data.

6,779 Posts
24,927 Replies

When you subscribe we will email you when there is a new topic in this category

6779 Posts

Recently active

Most replies Most views Newest first

jorge_vidinhaContributor

asked in Data

POSTGIS Reader different geom column

Is there any possibility to change the default geometry column 'geom' on postgis readers to some other column with a different geometry like geom2 ?Thanks

Hi Guys, this might be a simple question - I am writing data to a TAB file. every time I run the workspace it over writes the TAB file with the new data. How do I append data being written to the TAB file.Dave

I I am trying to convert the attached shape file to topojson. it is a shapefile of points. It seems some of the points are causing a problem when converting because I can take a sample and create the topojson file. I can't seem to figure out the problem with the offending points. Appreciate any help. Thanks.David

4 years ago

mamaieron

asked in Data

Unexpected SurfaceOnSurfaceOverlayer ouput

Hello everyone, I am converting CAD data to CityGML. To create the LoD2 class of buildings with hierarchical integrity, I used GeometryCoercer to transform the extruded solids into surfaces, followed by the Deaggregator, separating the roofs, walls and grounds. I tried to use SurfaceOnSurfaceOverlayer to identify shared walls between buildings but the transformer is not identifying all the cases. When I set the tolerance, it identify more cases but it ends up disturbing the geometry of the surfaces, generating non-flat surfaces, with duplicate points, etc ... all verified by GeometryValidator and by CityGML validation tools like Val3dity. As you can see in the prints, a case of shared surface is identified (Surface 1 and 2) and another one with the same characteristics (Surface 3 and 4) no. As I used the vertical direction in the Extruder and have the same X and Y coordinates, they should be coplanar- I also used the PlanarityFilter before the process and the value of the _surfaceNorma

122

4 years ago

dms2Contributor

asked in Data

GML reader and 'cannot convert null string into an integer' error

We recently upgraded FME Desktop 2007 to 2020. Now I'm upgrading all of our old workspaces and I've having an issue after upgrading GML reader to the latest version.GML files are read without any error with the old reader but with the new version I get the following error for every feature in every GML file:Cannot convert null string into an integer class StatusInfo *__cdecl STF_stringToUInt(const class ObsoleteString &,unsigned long &)(C:\\code\\fme\\foundation\\framework\\util\\stf\\stfutil.cpp:358) - class StatusInfo *__cdecl STF_stringToUInt(const class ObsoleteString &,unsigned long &)(C:\\code\\fme\\foundation\\framework\\util\\stf\\stfutil.cpp:358)I checked the GML version for all my files in the xsd file and they all say it's GML 3.1.1. Didn't see any limitation on this in the reader's help.Any idea of what's going on?Edit: the old GML reader works as expected before saving my workspace with Desktop 2020, but the error is there after sav

602

4 years ago

rudystricklan1Contributor

asked in Data

Read AutoCAD files to be processed from a text file

I have a text file that contains an AutoCAD pathname on each line, e.g., D:\\project4033\\dwg\\p1234.dwg D:\\project515\\dwg\\p642.dwg X:\\dwg\\a556.dwgHow can I set up a Reader to process each file (the output format will be Adobe GeospatialPDF)? I see answers like a "TextFileReader", but on my 2020.1 system there is no such transformer.

4 years ago

groschialeux

asked in Data

How to order tags in OSM Writer

I am working on a workspace that reads Tourism Information Office data from an XML file, converts the attributes to OSM tags and writes it to an OSM file. This OSM file is then processed by a program called OsmAndMapCreator to create an obf (OsmAnd Binary Format) file for the OsmAnd mobile application. Each office has a collection of characteristics and services so there is more than 1 tag for each node. The order in which those tags are written to the OSM file is important for how the feature will appear on the map. A tourism information office that has parking or picnic tables still needs to appear as a tourism information office no matter what else they offer. The OSM writer does not maintain the order of its reader or User Attributes list. In my reader and writer the tourism tag is at the top but for some reason, it is always the last one in my OSM file. Here's 2 nodes where it creates a problem.This node shows up as a picnic table on my map but it should appear has a tou

How to convert OpenDrive (.xml) to 3D SHP

Is it possible for FME to convert OpenDrive (.xml) to 3D SHP?

BIM (revit) to Cesium3dTiles, Reprojection Failed

Hi everyone,I have a BIM (revit file) that i would like to transform to Cesium3Dtiles. I used the "Revit Geolocation" Tutorial of @daveatsafe and exported the model into IFC. When I look in the FME Data Inspector the IFC file looks fine (I think). But when I add a Cesium3Dtiles writer I get an reprojection error. The coordinate system of the IFC file is EPSG:3857 and the coordinate system of the Cesium3Dtiles schould also be EPSG:3857. So I don't understand why there is a problem with the reprojector.

468

4 years ago

dms2Contributor

asked in Data

How to handle a geopackage received as base64 binary

I need to publish a workspace to fme server that will receive a geopackage file as base64 binary and will do some schema and data validation before inserting geopackage data into a number of Oracle tables. Actually an API will call the FME service providing that base64 binary and some other information.Now my question is how to read geopackage schema and data into fme. Any idea? I don't see how to get the binary back into a geopackage file.Thanks!

GoogleBigQueryConnector Query returns no results

I've installed the Google BigQuery package into 2020.1 and I've got a web connection to a BigQuery project. The connection seems to work since I can see the dataset and tables listed. I am sending a simple SQL 'SELECT * FROM...' and I've tried 'SELECT col1, col2 FROM ...' I want the query to return records from a non-spatial BigQuery table to use in FME. I don't get any results and the log doesn't seem to give anything helpful. We've opened all the permissions on the account. Does anyone have any advice on how to get this to work?

Ignore old files with ProjectWise

Hello! I'm reaching out because I'm in aa bit of a pickle. I have a task to pull a large number of photos and folders from a ProjectWise folder. I've successfully been able to do so through the ProjectWiseWSGConnector and FeatureReader, and then a number of other transformers that modifies the images for what we need. However, because the number of photos is quite large (GB's of data), it's quite the task to run it and to re-download all of the files. Is there a workflow that will skip over previously downloaded files and just download the newly added or updated ones? Thanks for any help!

ProjectWise: Version and Status not supported?

Hello,I am trying to get the information on files stored in ProjectWise. I am using FME 2020.1.However, the ProjectWiseConnector, seems to be unable to get all attributes, i.e. Version and Status.Also tried custom ProjectWiseNavigationHierarchyRetriver, which looks to be able to work with all attributes, but only works with API v.1.2 and 1.3 and we are on 2.4.Are there any other transformers or a newer version of the above that could read all file attributes?Many thanksMagda

222

4 years ago

jackydContributor

asked in Data

remove feature dataset from data when writing to FGDB Open API dynamically

Hi, I am trying the write out using the to a file geodatabase using a feature writer but am unable to remove the feature datasets from the data. I am using OPEN API as i am reading in all the fgdbs in one directory and writing them to a new location and splitting the data back to the original fgdb names, but don't want the feature data sets to be used (one of the main reasons for doing this) and i can not get the feature writer to writer to different fgdbs.Below is the basic work space so far, the schema reader reads all the feature classes in the fgdbs then gets the feature reader to read those tables and send them to be saved in the new location by the feature writer. This will be the first stage of a data update process, eventually the workbench will read new data that has come in, compare it with a previous version and then update the file gdb with the changes (along with details of the change user, date and source). Each time data is delivered there could all or just some of t

322

4 years ago

Anonymous

asked in Data

Build surface/solid from lines

I've been given a sample from a CityGML dataset that I want to eventually convert into an FBX 3d model. Now, before receiving this sample, I had built a workflow that did the entire conversion from a CityGML dataset into FBX models, tested it out with different CityGML datasets from open data repositories and it worked just fine. But in all of those datasets, the geometry was made up of surfaces, while on this sample there are just lines.I've been messing around with the AreaBuilder transformer but even though some of the surfaces are generated, I can't seem to get it nowhere near what it should be like.Any ideas?The sample: https://drive.google.com/drive/folders/1JAKSgxgjPMRfwJoDgCaKPValeaZRmO88

541

4 years ago

brandonguoContributor

asked in Data

SQL table generate by FME not visible in QGIS

I uploaded a table into MSSQL server, but the table is not visible in QGIS.There is no issue viewing the table in SQL Server Management Studio, and the primary key is generated by counter.So far, I tried turn Spatial Index on and off; Change the collection type handling; Change Insert into Identify Column on/off. None of those steps work.Any idea how to modify the writer to let the table visible in QGIS? Update:In ArcGIS Pro, the table is visible, but can't add to map, gives an error message.

Important: Does the Geocoder (ArcGIS Online) consume Credits?

Is it free to use or does it consume credits?

Trouble converting a 2019 DWG to File Geodatabase with Annotations

I have cad file I am trying to convert to a File Geodatabase. The CAD file has labels that I would like to convert to geodatabase annotations. When I generate a workspace specifying the DWG and output as File geodatabase, I get a few features that are empty and a blank gdb table. Any ideas on how to do this? Thanks! Attached is the cad file and a screenshot of the workspace I have.

4 years ago

jeffjamesjudyck

asked in Data

Raster Map Service Spatial Analysis

I am working on a FME script that is pulling from a raster in a map service: https://maps.nccs.nasa.gov/server/rest/services/global_landslide_catalog/landslide_susceptibility/MapServer When I do so using a Esri ArcGIS Server Feature Service I get a unsupported error. How can I get the raster into a workbench to do spatial queries against? Thank you in Advance!

4 years ago

bubblebebContributor

asked in Data

Excel Writer unlocking protected worksheets

Has anyone found that FME unprotects excel worksheets when it writes to them? I have a worksheet that I need to keep protected with certain cells being written to only. The rest of the sheet has loads of complex formulae which need to be kept protected.I have protected the sheet but when FME writes it unprotects it without the password. Does anyone know how I can stop FME from unprotecting the worksheet?

553

4 years ago

tnarladniEnthusiast

asked in Data

Using merged feature types in Excel reader and also get the formatting

I have created an excel reader with the merged feature type because the source dataset as well as feature type (sheet name) changes with each run. But when I am using merged feature types, I cannot get the .formatting. THey are all coming over as "missing". I would have to specify a specific feature type (sheet name), but because it changes each time (not named Sheet 1), I cannot specify that at creation time.Is there not a way to do this? Am I missing something?

4 years ago

frcoud

asked in Data

How to check existing files in a folder from a transformer ? (not a Path reader)

Hello everyone,I am currently trying to get a list of existing file names inside a folder. Here are some explanations :- I have a Path reader which reads existing files in a folder (let's call it folder 1). In this folder, there should be one text file;- If the text file exists, I use a FeatureReader to read it. This text file contains several paths to other directories (folders 2, 3, …, n).- Once I extracted those new path, I want to do the same as with the reader : get the names of the existing files inside folders 2,...,n. For now, I know how to read files from within the Workbench, with a FeatureReader. But how can I do the exact same thing as the Path reader without using one ? I do not need to read the files right away, I just want to get the names of the files Inside folders 2,...,n. The idea is to do all those operations in the same fmw script.Any idea ?

1085

4 years ago

armando_amadorContributor

asked in Data

How to write an HTTP CALLER response body to a CSV writter?

I'm running a workspace with a creator→hhtpcaller and I would like to have the response saved into a new csv (excel) file as a regular table format. This is the API as reference: https://datos.cdmx.gob.mx/api/records/1.0/search/?dataset=casos-asociados-a-covid-19&q=&rows=10000&facet=fecha_actualizacion&facet=id_registro&facet=origen&facet=sector&facet=entidad_um&facet=sexo&facet=entidad_nac&facet=entidad_res&facet=municipio_res&facet=tipo_paciente&facet=fecha_ingreso&facet=fecha_def&facet=edad&facet=nacionalidad&facet=embarazo&facet=habla_lengua_indi&facet=diabetes&facet=epoc&facet=asma&facet=inmusupr&facet=hipertension&facet=cardiovascular&facet=obesidad&facet=renal_cronica&facet=tabaquismo&facet=negativo&facet=migrante&facet=pais_nacionalidad&facet=rango_edad&facet=positivo&facet=pendiente&refine.entidad_res=GUANAJUATO Do you guys know what can I do?

The WFS reader always chooses the default srs

The WFS reader automatically chooses the default srs of a WFS layer. Is there any way to make it read the features in a different srs which is supplied by the WFS supplier (and so also listed as option in the WFS capabilities). Changing the coordinate system of the reader doesn't work. It will still request the features in the WFS default srs and then interpret it (wrongly) as the coordinate system set at reader level.

162

4 years ago

supaustinContributor

asked in Data

Has anyone successfully written photos to an Access DB?

I am attempting to read photos from attachment tables in a FGDB and write them to an Access DB, where we have a workflow for report generation. I can get the photos out easily enough and write them to a folder, but I can't find the proper settings in the Access writer to add the photos.

4 years ago

Page 64 / 272

Badge winners

ahmed.shahinhas earned the badge Integrate Data with the FME Platform Training
sjakohas earned the badge Welcome Back
jcvanlingehas earned the badge Expert (Grey)
ienesaglam60has earned the badge Student (Bronze)
jcvanlingehas earned the badge Welcome Back

Show all badges

30,117: Posts

Terms & Conditions Cookie settings

Sign up

Already have an account? Login

An FME Account is required to contribute

Login to the community

An FME Account is required to contribute

Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.

Enter your e-mail address

Back to overview

Scanning file for viruses.

Sorry, we're still checking this file's contents to make sure it's safe to download. Please try again in a few minutes.

This file cannot be downloaded

Sorry, our virus scanner detected that this file isn't safe to download.

A product of

Company Products Giving Back Careers Contact Privacy Feedback

Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.