Skip to main content

Data

Using readers and writers to extract and load data.

  • 6,970 Posts
  • 25,684 Replies
6970 Posts
D
dewan
asked in Data
Putting in the Second Reader once the First Output is Written

Hello. Can we control the flow of the in[ut gdb feature classes so that the first feature class is read first, goes to the complete workspace and the output is written? Then the second feature class is read, goes to the complete workspace and the output is written and so on! Thanks.

2750
mark2atsafe
Safer
fmelizard
Contributor
fmelizardContributor
asked in Data
Esri's ArcGIS Online is removing support for TLS 1.1 and 1.0, Is there anything I need to be aware of as an FME user?

For context see this Esri FAQ: https://support.esri.com/en/technical-article/000019305 In the article, Esri plans to require TLS 1.2 connections for ArcGIS Online services starting in February 2019.

120
brianatsafe
Safer
gcarmich
Participant
gcarmichParticipant
asked in Data
Binning a large area

I want to create contiguous 30 meter hex grids for binning spatial point data for on-going summary analysis. Ideally, I'd like for these grids to cover the entire earth. Creating and working with this volume of spatial data would be difficult if not impossible. I've thought about creating large rectangular grids as reference with the hex grids overlayed and subdivided and stored into groups (files) referenced by the rectangular grids. When evaluating new point data, a workflow could reference the rectangular grids that intersect the points and open only the corresponding group of hex grids. Would this approach work with FME? Is there another approach to binning and summarizing point data on a very large scale?

280
mark2atsafe
Safer
T
tsurveyor
asked in Data
Reading google sheets that contain pull down list data

My import was working when I was using Excel with data validation but I put all the data in google sheets, where it needs to be, and now the sheets won't read. Both sheets have attributes whose values come from a pull down list. There is then a third attribute with a rate that I need. The rate is then joined to another table based on the first two attributes matching. I am using a featurejoiner to join the google sheet to the other table. Is there a problem reading this type of file? What is the solution?

50
T
A
andreandrade3
asked in Data
How to update state machine

Hi there, I'm copying a table to other table in different oracle database. When the copy finishes, I need update a state field in source table (reader). How a reader could be transformed to a writer?

190
mark2atsafe
Safer
A
andreandrade3
asked in Data
How to store a select result in a constant

I'm new on FME and want to know if is there a way to store an oracle select as constant? This value will be used to update some oracle tables.

180
gio
Contributor
P
purnima1298
asked in Data
how to extract 2d building foot print from lidar point cloud

how can i get 2d building foot print from lidar point cloud data?

910
C
B
bazooka
asked in Data
Updating AGOL data

I have Feature layer which is made with FME at AGOL. Now I would like to update that layer, but translation fails every time. I am using Esri ArcGIS Online (AGOL) Feature Service writer and Format Parameters are Writer Mode = INSERT, Truncate Layer First = YES and Rollback On Failure = YES. I think that it should work with those parameters. If I delete layer from AGOL, then it works fine, but if I don't delete layer, then translation fails. I would not like to delete layer manually. How to update AGOL Feature layer with FME?

2890
T
L
luckym
asked in Data
How to determine position of one feature in relation to another?

Hello, I have 2 Shape files, one point, other line. I need to determine on which side of the line point feature is located. I need to take only points that are within 5m of line. For every point I need to add new attribute that will allow me later to separate right and left. Orientation of the line is determined by start and end point of the line.This is way out of my league so I need Your help.

1580
L
J
jerez
asked in Data
How to define material/appearance in the cesium 3d tiles writer?

Hi Is there any convenient way to modify the material definiton for the batched 3d models when using cesium 3d tiles writer? I'm creating b3dm from CityGML and viewing them in the Cesium app in a browser. My problem is that some of the facades look too dark because of the shading according to the position of the sun. I read that setting some of the material settings of the gLTF, so that any shading from light source is flat, should solve the problem and make the building equally bright on each surface. It would be awesome if one could define these material/appearance settings as parameter values in the cesium 3d tiles writer.

570
virtualcitymatt
Celebrity
R
robertlilley
asked in Data
Incremental extract of Intergraph data to Smallworld

I am looking to do an initial extract of a utility customer's Intergraph data to Smallworld using FME workbench. Is there a way to do an incremental extract on a nightly basis, without completely reloading the entire database? In other words, is there a way to identify daily changes in Intergraph data (ie time stamp, versions, etc) ? @MarkAtSafe

90
R
G
greenstone1564
asked in Data
Output buildings into individual .obj files

So I have a large CityGML file, which is the LoD2 model of New York City. I now would like to export all buildings into individual .obj files. Currently I am facing two challenges: The file is too large (30G) to be loaded into FME data inspector directly. I do not know how to extract individual buildings in a programmatic way. I already loaded the data file into a 3D City Database using the importer / exporter. But still confused how to solve the two technical challenges above. Any advice would be helpful. Thanks!

1560
G
spatialphil
spatialphil
asked in Data
Issue authorizing FMEServer for ArcGIS Online

Hi @Mark2AtSafe, @RylanAtSafe Having trouble authorizing FME Server (FME Server 2017.0 - Build 17259 - win64) for use with ArcGIS Online. The FMEServer instance is part of a private network, therefore I have followed examples including those at: https://knowledge.safe.com/questions/43567/arcgis-online-oauth-authorize-issue-on-fme-server.html https://knowledge.safe.com/questions/65076/connecting-to-arcgis-online-via-fme-server.html https://knowledge.safe.com/articles/40215/configuring-fme-server-to-use-oauth-based-web-conn.html I have done the following: The FMEServer has been enabled for HTTPS The service account under which FMEServer runs on the server has necessary proxy whitelisting to hit the AGOL domains Since the FMEServer is private, it is not accessible publicly, nor has a public DNS entry. So as per the example in (3) I have set up a nip.io redirect to the IP address of the 'private' FMEServer I've got a registered application in AGOL (so got a clientId, clientSecret etc). The

150
philippeb
Enthusiast
G
granty555
asked in Data
Highlight Cells in Existing Worksheet

Hi Everyone, I have a workflow that takes a submitted Excel workbook and validates all of the input. The validations are not simple and often require checking data against databases, checking against other rows/tabs in the workbook etc. At the moment, I am writing out any issues/validation failures to a text file e.g. The value: 'XYZ' supplied on Tab: 'Tab1' in Row:5 Column:'C' is invalid for reason 'ABC'. I would like to highlight Cell C5 in red on the original file and send it back. I don't want to create a new file or write all the data back out, just simply locate specific cells in the file I read in at the start and change their formatting. Is this possible? Thanks! J

90
P
K
kmeinert
asked in Data
Writing to an ArcSDE Versioned table - where did my edits go?

So I'm simply trying to update a versioned table (happens to be non-spatial) in my ArcSDE database. The table exists now in SDE, I just want to truncate it and replace it with a whole new set of records every so often. Everything seems to run correctly in the log, but when I go view the table, even under my version (the one I used in the writer), all the old records are there like nothing ever happened. It says it wrote them, where did they go? So not only did it not truncate the table like I told it to, it didn't write the new records either. If it matters, I've tried using both SDE writers with the same result. Tried using the Insert vs Update parameter, same result. Tried using the SDE default version instead of my own, same result. This is a 10.0 SQL database, but I have 10.1 loaded on my desktop.

380
friesewoudloper
I
idrispeiren
asked in Data
Search string in pdf

Hi, I have a table containing Id's and url's referring to online pdf's. For each record in the table I want to search a specific string in the associated pdf. If the string has a match in the pdf, I want to retrieve the Id from the record containing the url to the pdf. I tried using feature reader en pdf reader but didn't get far. So any help would be welcome. Idris

650
I
F
fazsnatch
asked in Data
Generating X & Y points for Chainage

Hi guys, Hope you are well. I am just trying to figure out what would be the best way of doing this and if FME has a method. I understand Arcmap can do this but may take sometime. I have a shapefile generated which has a chainage of a road with various chainage parts/distances. I then have a spreadsheet which contains 40 records which has a start chainage and end chainage. What i require is to create X & Y coordinates for the start chainage and end chainage within the chainage of the shapefile for the spreadsheet. So basically this spreadsheet will then have another column with the X and Y coordinates.What would be the best process to do this? Is there a specific transformer that can be used to create these points? Start ChainageEnd Chainage153150154340

1020
T
A
ashishtwr
asked in Data
Delete existing records instead of Truncate in Oracle Spatial Writer before Loading new data

I am creating a Dynamic Data Uploading Workspace which will read all .mdb files present inside a folder & upload Data in the corresponding tables in Orace Spatial DB. The problem is the DB user which I have, to perform this Upload operation does'nt have Truncate rights on tables since its not the Table Owner & for policy reasons it cannot be given these rights although it has Insert,Update,Delete rights. How do I Delete records from Tables before inserting new ones? Apart from creating 2 seperate workbenches, First for deleting the records using SQLExecuter & second for inserting using Dynamic writer & connecting the 2 using Batch Processing.

450
erik_jan
Contributor
Z
zubairsm
asked in Data
Problem writing a dwg filename which contains arabic character

Hi, Using fme workbench I am writing(fan out) dwg files. Some of the filename has arabic character which fme is not able to write. following is the message ERROR |AutoCAD Writer: An error occurred for dataset 'D:\\Projects\\Filegdb_to_AutoCad\\Output-dwg\\?????? ????.dwg', in function RealDWGWriter::saveDatabaseAs: 'eFilerError' Note: fme is able to write arabic attribute as OD without any issue but the cannot create file. I am on windows 10 arabic enable OS. I can create arabic filename outside fme. Any solution or workaround. Regards

220
Z
ld
Participant
ldParticipant
asked in Data
Is it possible to write Geodatabase metadata using FME Cloud/Server?

Only the Open API Geodatabase writer can be used on FME Server/Cloud, which doesn't allow you to expose the 'geodb_type' and 'geodb_metadata_string' format attributes. Is anyone aware of a workaround for writing metadata to Geodatabase? This seems to be a major drawback of Server/Cloud if not.

70
virtualcitymatt
Celebrity
D
deckard
asked in Data
SENTINEL2 reader

Hi, I try to open an XML file downloaded with the SentinelDownloader but I get this error: Loaded module 'SENTINEL2' from file 'C:\\Program Files\\FME\\plugins/gdal/GDAL.dll' FME API version of module 'SENTINEL2' matches current internal version (3.8 20180604) SENTINEL2 reader: Opening dataset 'C:/4-Igual que 3 pero bajando ALL/all/S2A_MSIL2A_20181103T111221_N0209_R137_T30TVK_20181103T122857.SAFE/MTD_MSIL2A.xml' SENTINEL2 reader: Cannot find =Level-2A_User_Product.General_Info.L2A_Product_Info SENTINEL2 reader: Failed to open the dataset 'C:/4-Igual que 3 pero bajando ALL/all/S2A_MSIL2A_20181103T111221_N0209_R137_T30TVK_20181103T122857.SAFE/MTD_MSIL2A.xml'. Please ensure source data is valid and correct reader is selected A fatal error has occurred. Check the logfile above for details Failed to obtain any schemas from reader 'SENTINEL2' from 1 datasets. This may be due to invalid datasets or format accessibility issues due to licensing, dependencies, or module loading. See logfile for

140
D
G
granty555
asked in Data
Reading fme_connections.data

Hi all, I am trying to create a workspace to automatically review other workspaces to ensure they meet our development standards and guidelines. One of these standards is that all connections used should be in the master connections file we all share. Which reader do I use to read the fme_connections.data file and get a list of the connection names? A standard text reader returns a load of #$@#@!# so I assume it's encrypted somehow? Thanks in advance James

360
rahulsharma
Safer
S
spatial_aus
asked in Data
Read THREDDS Data Server Data Via OPeNDAP into FME Workbench

Is it possible to read in data into an fme workbench from a THREDDS data server via Opendap? I can see that this article (https://blog.safe.com/2014/10/netcdf-meets-gis-geospatial-architecture-scientific-data/) suggests that it is possible but a cannot find any useful resources that explains the process. Any help/direction to docs will be much appreciated.

140
C
A
andre
asked in Data
Global Mapper GMG Format

Hi, having problems Reading a Global Mapper raster format: *.GMG The *.gmg format is a grid format in binary form Anyone know how to read it with fme?

810
A
G
galensonet
asked in Data
How to create a KML file from a set of jpg geotagged

Hello, I am new in FME. I need to create a KML that contains a set of placemarks with every jpg geotagged photo. INPUT: Set of jpg geotagged OUPUT: KML with photo icons that I clic and show me the photo in a window. Thanks for your answers.

4470
srg

Helpful Members This Week

Badge Winners

Show all badges

Popular Tags

Community Stats

31,141
Posts
118,067
Replies
38,761
Members

Latest FME

Terms & ConditionsCookie settings

Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.