Skip to main content
  • EN

Data

Using readers and writers to extract and load data.

  • 7,092 Posts
  • 26,102 Replies
7092 Posts
arnovananrooij
Contributor
arnovananrooijContributor
asked in Data
How to replace a part of GeoTiff with new values

Hello,I'm working with over a hundred huge non overlapping geotiff's (512 Mb each) where the band 0 value represents the height. I also have some polygons, which sometimes overlap multiple geotiffs. I would like to compute the maximum height from the geotiffs within that polygon and I would like to replace that part of the geotiff with this maximum height.I managed to compute the maximum height so I created a shapefile with the polygons and the maximum height as an attribute and I also managed to create a Geotiff for eacht polygon with the maximum height as the band value.Now I would like to get the last part of the job done: replace parts the original Geotiff whith the new geotiffs. I'm not sure how this could be done so any help is appreciated. Arno

840
takashi
Celebrity
p.jeremie
Contributor
p.jeremieContributor
asked in Data
Excel writer loosing headers when writing to named range

Hello,I try to understand a weird behavior with ExcelWriter in one of my workspaces.I achieved to reproduce the behavior in a much much simpler version of my workspace :My destination file has 2 sheets with named ranged on each :Behavior :When I run the workspace and the TEST/test writer is executed, everything's fine (=> NewData are written to Sheet/NamedRange and appended to TEST/test as there is no truncate of named range for this one).When I run the workspace and the TEST/test writer is NOT executed (if the link is from <Unfiltered> for example), the NewData are written to Sheet/NamedRanged but I loose the headers :  The comments are still there, though... Note : If i have two writers instead of one with two feature types, everything's fine. Any idea ? A zip with workspace and xlsx files :

2300
p.jeremie
Contributor
simonhume
Contributor
simonhumeContributor
asked in Data
Tesseract extract text from Image

Good morning I'm trying to extract text from a simple jpg image using Tesseractcaller  When i run Tesseract-OCR (version 3.05.02 64bit) on my PC (Windows 10) from Windows Powershell to extract the data from the .jpg, it works fine  If I run the attached workspace in FME, the text export to .csv only gives the first line ie hello there but not the second line. I get no output for the line option  I found another post about the Tesseract transformer, https://knowledge.safe.com/questions/80141/how-can-i-get-the-tesseractcaller-to-output-to-the.html, and have made the suggested change, but this didn't seem to work  Any suggestions as to how i can resolve this?  Simon Hume 

2490
jovitaatsafe
Safer
dmerrick
Contributor
dmerrickContributor
asked in Data
Export to Multiple formats

Hello, I am working on revamping a workspace that is used as an open data platform to export all available data into 5 different formats (Geopackage, FGDB, DWG, CSV, SHP). Each dataset (ie. cadastral, development, engineeering etc.) have varying featureclasses that need to be exported into one zipped folder. Currently my set up has one writer per dataset per format (10 datasets, 50 writers).Is there some way to loop through the formats on a single writer, while writing out into a zipped folder per format? Thanks!

160
itay
Supporter
R
richsnyder0
asked in Data
Issue publishing Apache Kafka package to FME Server

We're getting a generic "Package access failed" message in the log when we try to publish a workspace with the Apache Kafka package. Any assistance would be much appreciated.

270
nathanatsafe
Safer
jbradfor
Contributor
jbradforContributor
asked in Data
AWS Aurora for PostgreSQL

We are interested in creating an Aurora for PostgreSQL Spatial DB. However, it is not clear if FME has a Reader/Writer. Do you, or would we use the normal RDS PostGIS/PostgreSQL reader/writer?

180
jbradfor
Contributor
LizAtSafe
Safer
LizAtSafeSafer
asked in Data
Tutorial Series Update: Update a Database Using an FME Server App

Hi Partners,  The tutorial series Tutorial: Push data from the database to an application in real-time has been updated in anticipation of the Google Fusion Tables deprecation in December. The tutorial now features FME Server Apps to trigger the databases. With this new change, the series has been renamed Tutorial: Update a Database Using an FME Server App.   

80
LizAtSafe
Safer
L
luckym
asked in Data
KML to GPX

Hi, I want to translate some KML to GPX. Never done with this types of files so I am using quick translator. I am getting some results but i need to get a timestamp for my points. How can I get this? In my KML data time stamp is located in Name and Snippet section (Picture 1). GPX that I need to get is  GPX that i get from quick translation is:   

570
L
sigtill
Supporter
sigtillSupporter
asked in Data
ZDA-format

Has anyone worked with the *.zda-format? I have some files that should contain some drill hole data but struggle to figure out which software produced these files and which software can read them. They are approx 0,5MB large.When you unzip the *.zda you get these files. and the *.dat files are  And the data in the *.dat-files are like this - not sure how to decipher these. Any help appreciated!

240
sigtill
Supporter
hth
Participant
hthParticipant
asked in Data
citygml cityfurniture

I am looking for an example to write CityGML-cityfurniture with FME-Desktop. I've created a 3D-modell of a sign within FME which I now want to export as citygml-cityfurniture. While creating it I've assigned different apperances to parts of the sign (sign=white, text on sign=black) and aggregated all faces to that sign. Looks great, if I export it to 3D-PDF. But exporting it to citygml-cityfurniture lacks the colors of the sign. I am not quiet sure, if I doing it right. After aggregating I am using the transformers as shown in the attached jpg-file.    Any suggestions or a workbench that creates cityfurniture the write way would be nice. :) 

160
harmen_kampinga
R
reto_meier
asked in Data
Writing PostGIS - CompoundCurve

HelloI try to write Polyline-Geometry from ESRI Personal Geodatabase to PostGIS (Version 2.5) with geometry typ CompoundCurve. With PathBuilder I am successful to build Paths with Segments with only IFMELine or mixed IFMELine/IFMEArc and to import them.But I am not able to write Paths with only Arcs to PostGIS. The import fails with following message:Error executing SQL command ('INSERT INTO ... Geometry type (CircularString) does not match column type (CompoundCurve) Any Ideas? Thank you. 

1580
R
P
pelwinderpal
asked in Data
Problems when converting from FBX to GLTF

I have a model in .FBX format and I want to convert it to .GLTF format. When I do this in FME under generate workspace, I can go straight from ,fbx to .gltf. However, the final .gltf model loses some texture. In my case, the vehicle structure is there but details such as the windows are all missing. Would appreciate some advice. Thank you.

280
virtualcitymatt
Celebrity
H
hrinze
asked in Data
Offset error when referencing autocad drawing with template in FME

Hi, I am generating dwg files in FME with the symbols from a DWG template. The file opens correct in Autocad but when using x ref they ends up with an offset.Do I have set coordinate sytem, origo etc in the template or is it something I can do in FME? Thanks Henrik

120
daveatsafe
Safer
clemensb
Contributor
clemensbContributor
asked in Data
Database Updater - Format:ArcSDE deletes Geometry with Server 2019.1.1 even when Value is NoAction

We used FME Server 2017.1 and this week we upgraded to FME 2019.1.1Today i found out the DatabaseUpdater with ArcSDE Geodb deletes the Geometry when you want to update a Table Column with a value from an non Spatial source.We had some Workspaces with 2017.1 and it worked fine there. For example one of productive workspace is a non-spatial-oracle database table, which updates a ArcGIS Featureclass in a MSSQL SDE.I restored the Backup for Server 2019.1. The workspace ran over night and the next day i found out, the data is broken because the geometry is removed.I tested it and found out it is the DatabaseUpdater Transformer with ArcSDE Geodb Parameter.I changed the format to mssql in the DatabaseUpdater, this works fine.I also can reproduce this with a brandnew Workspace, wich was made with FME2019.1.It doesnt matter where the non-spatial-value comes from, i made workspace with different readers(excel, text,csv, and so on) it will delete the geometry when you use a DatabaseUpdater with A

370
nampreetatsafe
Safer
kathyjbryce
Contributor
kathyjbryceContributor
asked in Data
SharePoint read display datetime (not UTC)

I have a SharePoint list that displays local datetime from many different local time zones, this is the datetime required. The reader returns datetime fields in UTC only. The time offset could be 6 to 8 hours. I have no way to know what local timezone the records was entered in, thus I cannot determine what the time offset should be. Is there a way to read the displayed datetime, not UTC?For example, two records with different time offsetsSharepointValue read by FMEFormattedOffset9/14/2019 21:0020190915030000+00 9/15/2019 3:306 hours2/14/2019 21:5520190215045500+002019/02/15 4:557 hoursThanks in advance,Kathy

351
jovitaatsafe
Safer
A
abelc
asked in Data
How to Pass input to ODBC Reader

I am building a process flow that moves data from point A to B. I noticed the ODBC reader has faster speed over SQL Executor. The challenge is the ODBC reader doesn't have inputs as incoming initiator like SQL Executor. Is there a way that I can use tow ODBC readers one execute after the other receiving inputs?

70
jovitaatsafe
Safer
kathir
kathir
asked in Data
dwg conversion

I Have a dwg file. I need split that into multiple dwg based on the used autocad layer.

390
jovitaatsafe
Safer
michael_m
Contributor
michael_mContributor
asked in Data
Writing to an SDE dramatically slower than writing to a file geodatabase?

I have a workbench that I regularly use to read from an Oracle Locator database and write to an ESRI file geodatabase. It writes about 55 feature classes to this geodatabase (approximately 8 million features total) and takes about 90 minutes to complete. I have just had an SDE made available to me so I made a copy of my original file gdb workbench and have been changing it to write to the SDE instead. The problem I'm having is the workbench now writes extremely slowly; it took 90 minutes to write a single feature class with 168,000 features in it (at that rate I estimate it would take two or three days to write all 8 million features to the SDE). Has anybody experienced anything like this or have any idea what I can do to speed it up? If it makes a difference I believe the SDE is running on some version of SQL Server. Thanks,Mike

3170
michael_m
Contributor
M
mari
asked in Data
point cloud: extract 3D objects / solids "Best Practice" ?

Is there a practicable way in FME to extract 3D objects or solids (indoors and/or outdoors) from point clouds?I found posts back from 2015, lots of feature requests and ideas. What's "State of the Art"?Inside / outside of FME... (conc. Revit but also other data models)Bye, Marion

860
jovitaatsafe
Safer
M
milossedlacek
asked in Data
FME integration to ArcGIS PRO

Hi, I have a question about FME desktop 64 bit. Will be possible in the future integrate FME desktop to ArcGIS PRO like for 32 bit version into ArcGIS desktop? The reason is that ArcGIS desktop is stopping slowly supported and if you would use ArcGIS interoperability you have to pay another licence. The basic functions which allows FME integration in to ArcGIS desktop are great and I would have them also in ArcGIS PRO.Thanks,Miloš

220
arnold_bijlsma
Enthusiast
deanhowell
Influencer
deanhowellInfluencer
asked in Data
FME Server (Local) Install Database

We have just installed FME Server on a local server using the express option but did not see any settings for the Postgres database?Is there a way to find out the login / connection details for the Postgres Database?Is it automatically set-up for PostGIS?

160
itay
Supporter
tomfriedl
Contributor
tomfriedlContributor
asked in Data
FME 2017: Writer FileGDB handle feature_type wrong

I'm insert:Writer FileGDB (ArcObjects)Set Feature Class to fme_feature_typeDuplicate the OutputIn the navigator is only one Feature Type visible. In FME 2016 there are two. FME 2017 ignore the second output.It's a bug?

110
V
boubcher
Contributor
boubcherContributor
asked in Data
start a work space as soon as a record is added to google sheet

Hello there is there a way we could trigger a workspace as soon as a record is added to google sheetThanks

50
boubcher
Contributor
C
crutan
asked in Data
File geodatabase reader reading extra features

I have set up a file geodatabase reader (ArcObjects) and pointed it to read one feature class in a file geodatabase. The feature class has 50 features. When I run the process, it reads 100 features. I tried deleting one feature from the feature class, and now it reads 99 features, while ArcCatalog shows only 49 features in the feature class. This issue has not been consistent -- sometimes if I change the settings of another file geodatabase reader (pointed at a different file geodatabase) in the workspace, the problematic reader will only read 50 features. I am trying to use dynamic inputs, passed to the reader from a scripted parameter. Why is this happening, and is there an easy solution? I am using FME through ArcGIS's Data Interoperability extension, version 10.5. It is FME version 2017.0.0.1 (20170316 - Build 17271 - WIN32) and at this time upgrading is not an option for me.

700
C
K
katiem
asked in Data
WFS returning 0 features

I'm having difficulty accessing data with a WFS reader. I can read the feature types, and the translation is 'successful', but returns 0 features. I can load the WFS in QGIS too. I've tried amending various parameters without success, including: connection timeout length, max features, ignore application schema (this worked with the previous version of the WFS), and using a search envelope. The WFS is: https://environment.data.gov.uk/spatialdata/historic-flood-map/wfsWould really appreciate if anyone can nudge me in the right direction!  

15713
U

Badge Winners

Show all badges

Community Stats

31,807
Posts
120,820
Replies
39,435
Members

Latest FME

Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.