Using readers and writers to extract and load data.
Recently active
Hi all,I am using FME Desktop to convert some spatial DBF files to MapInfo Tab files. I'd like to ask if there are any restrictions for doing this operation. While the operation is running successfully, no geometry is being copied from the source files.Kind regards,
I'm trying to create a workspace to create a XML file from an excel-report (Sample) and an excelfile containg some other information (Blad1). However I'm running in to issues with the encoding of the created file. It seems like textwriter is not able to output UTF-8 encoded characters (input contains Swedish characters like åäö). Trying to understand where in the process this gets wierdly formatted but loggers seem to show the right characters. When I exchange these characters manually in the output XML file the validationservice I'm using is working. Anyone who can point out how to fix this? Kind regards.
Hi,Apparently (this is new to me too) the ESRI Shape definition has been expanded with a "code page file" (*.cpg), which contains the "code page" of the table structure, i.e. field names.I've just received such a data set, but FME doesn't seem to recognize this extension. My field names end up as un-decoded UTF-8 string, e.g. "ByggeÃ¥r" instead of "Byggeår".Is it possible to have FME (2017.1) recognize this extension, or if not, are there any plans to do so ?Cheers
Hi, I am trying to load features from ESRI file GDB to another ESRI File GDB schema. Geodatabase Error (-2147220987): The user does not have permission to execute the operation.FileGDB Writer: A feature could not be written. I am using ArcGIS version 10.14 and FME Desktop version 15.1 Thanks, Shahid
Hi,In regards to my previous question, I have tried finding how to read shapefile from AWS directly and read it (though seems to be quite slow). This possibility can neglect the use of multiple WorkspaceRunners(and I am still not sure how to use that)shp reader- Select file from web- Browse S3- Authenticate- Choose shapefile.my case:C:\PATH\ABC\ABC_Q1.SHPC:\PATH\ABC\ABC_Q2.SHPC:\PATH\ABC\DEF_Q.SHPC:\PATH\ABC\GHI_Q2.SHPSo, I can basically select 1 shapefile and process it at one go but really not sure about reading multiple filesin one go.S3 path: fme://amazon-s3/ABW_C_Q4.zip?id=prithvi-output%2FqualityLevel%2FABW%2FABW_C_Q4.zip&connection=parashas%20Amazon%20Web%20ServicesTried using * in place of ABW_C_Q4 and ABW but doesn't really help.
I am using FME 2014 in order to manipulate some shape files. When I load and inspect the content of the shapes file, I get the following problem: Although all the greek characters of the content is shown correctly (in UTF-8), the titles of each column (seeing in data inspector) are not recognized. A similar problem I had some time ago with an excel file which had content in greek. In that case also I could see the content of the excel cells but I could see the names of the tabs (sheets). Back then it was enough to rename the sheets to latin characters but now I don't have this luxury (some hundreds of shape files). What is the issue here? Is there some internal encoding properties in FME? I see I have the same problem in FME 2015. Moreover, I try to import a shapefile with greek characters and FME can not read it correctly. In the directory field it shows question marks instead of greek characters and its impossible to load it!
We have Sharepoint 2013 Enterprise and numerous excel tables I would like to access are stored there. I've tried and failed to manage to access any. I tried using an xls Reader and a SharePoint Reader. Is the SharePoint List Reader even meant for this? Or is the HTTP Caller the way to go? Is there any better documentation with real examples of what is required for the various SharePoint parameters (site, authentication). I am using FME 2017.1. Any insight would be appreciated.
I'm using FME 2018 with default python settings, I got the attached work space and it doesn't work because of the python problem , I don't know much about python or the coding of the transformers . @daleatsafe @boubcher @takashi
See the support roadmap from Oracle here: https://www.oracle.com/technetwork/java/java-se-support-roadmap.html
Hi guys, I am newbie to this forum and even FME. My simple question is that why IFC to GML export not allowing me the layer values? The layers values (like wall, floor etc in IFC) are missing in GML format.
Hello. Can we control the flow of the in[ut gdb feature classes so that the first feature class is read first, goes to the complete workspace and the output is written? Then the second feature class is read, goes to the complete workspace and the output is written and so on! Thanks.
For context see this Esri FAQ: https://support.esri.com/en/technical-article/000019305 In the article, Esri plans to require TLS 1.2 connections for ArcGIS Online services starting in February 2019.
I want to create contiguous 30 meter hex grids for binning spatial point data for on-going summary analysis. Ideally, I'd like for these grids to cover the entire earth. Creating and working with this volume of spatial data would be difficult if not impossible. I've thought about creating large rectangular grids as reference with the hex grids overlayed and subdivided and stored into groups (files) referenced by the rectangular grids. When evaluating new point data, a workflow could reference the rectangular grids that intersect the points and open only the corresponding group of hex grids.Would this approach work with FME? Is there another approach to binning and summarizing point data on a very large scale?
My import was working when I was using Excel with data validation but I put all the data in google sheets, where it needs to be, and now the sheets won't read. Both sheets have attributes whose values come from a pull down list. There is then a third attribute with a rate that I need. The rate is then joined to another table based on the first two attributes matching. I am using a featurejoiner to join the google sheet to the other table. Is there a problem reading this type of file? What is the solution?
Hi there,I'm copying a table to other table in different oracle database. When the copy finishes, I need update a state field in source table (reader). How a reader could be transformed to a writer?
I'm new on FME and want to know if is there a way to store an oracle select as constant? This value will be used to update some oracle tables.
how can i get 2d building foot print from lidar point cloud data?
I have Feature layer which is made with FME at AGOL. Now I would like to update that layer, but translation fails every time. I am using Esri ArcGIS Online (AGOL) Feature Service writer and Format Parameters are Writer Mode = INSERT, Truncate Layer First = YES and Rollback On Failure = YES. I think that it should work with those parameters. If I delete layer from AGOL, then it works fine, but if I don't delete layer, then translation fails. I would not like to delete layer manually. How to update AGOL Feature layer with FME?
Hello,I have 2 Shape files, one point, other line. I need to determine on which side of the line point feature is located. I need to take only points that are within 5m of line. For every point I need to add new attribute that will allow me later to separate right and left.Orientation of the line is determined by start and end point of the line.This is way out of my league so I need Your help.
HiIs there any convenient way to modify the material definiton for the batched 3d models when using cesium 3d tiles writer?I'm creating b3dm from CityGML and viewing them in the Cesium app in a browser. My problem is that some of the facades look too dark because of the shading according to the position of the sun. I read that setting some of the material settings of the gLTF, so that any shading from light source is flat, should solve the problem and make the building equally bright on each surface. It would be awesome if one could define these material/appearance settings as parameter values in the cesium 3d tiles writer.
I am looking to do an initial extract of a utility customer's Intergraph data to Smallworld using FME workbench. Is there a way to do an incremental extract on a nightly basis, without completely reloading the entire database?In other words, is there a way to identify daily changes in Intergraph data (ie time stamp, versions, etc) ?@MarkAtSafe
So I have a large CityGML file, which is the LoD2 model of New York City. I now would like to export all buildings into individual .obj files. Currently I am facing two challenges:The file is too large (30G) to be loaded into FME data inspector directly.I do not know how to extract individual buildings in a programmatic way.I already loaded the data file into a 3D City Database using the importer / exporter. But still confused how to solve the two technical challenges above. Any advice would be helpful. Thanks!
Hi @Mark2AtSafe, @RylanAtSafeHaving trouble authorizing FME Server (FME Server 2017.0 - Build 17259 - win64) for use with ArcGIS Online. The FMEServer instance is part of a private network, therefore I have followed examples including those at:https://knowledge.safe.com/questions/43567/arcgis-online-oauth-authorize-issue-on-fme-server.htmlhttps://knowledge.safe.com/questions/65076/connecting-to-arcgis-online-via-fme-server.htmlhttps://knowledge.safe.com/articles/40215/configuring-fme-server-to-use-oauth-based-web-conn.html I have done the following:The FMEServer has been enabled for HTTPSThe service account under which FMEServer runs on the server has necessary proxy whitelisting to hit the AGOL domainsSince the FMEServer is private, it is not accessible publicly, nor has a public DNS entry. So as per the example in (3) I have set up a nip.io redirect to the IP address of the 'private' FMEServerI've got a registered application in AGOL (so got a clientId, clientSecret etc).
Hi Everyone,I have a workflow that takes a submitted Excel workbook and validates all of the input. The validations are not simple and often require checking data against databases, checking against other rows/tabs in the workbook etc. At the moment, I am writing out any issues/validation failures to a text file e.g. The value: 'XYZ' supplied on Tab: 'Tab1' in Row:5 Column:'C' is invalid for reason 'ABC'. I would like to highlight Cell C5 in red on the original file and send it back. I don't want to create a new file or write all the data back out, just simply locate specific cells in the file I read in at the start and change their formatting. Is this possible? Thanks!J
So I'm simply trying to update a versioned table (happens to be non-spatial) in my ArcSDE database. The table exists now in SDE, I just want to truncate it and replace it with a whole new set of records every so often. Everything seems to run correctly in the log, but when I go view the table, even under my version (the one I used in the writer), all the old records are there like nothing ever happened. It says it wrote them, where did they go? So not only did it not truncate the table like I told it to, it didn't write the new records either. If it matters, I've tried using both SDE writers with the same result. Tried using the Insert vs Update parameter, same result. Tried using the SDE default version instead of my own, same result. This is a 10.0 SQL database, but I have 10.1 loaded on my desktop.