Using readers and writers to extract and load data.
Recently active
HI!I have an issue updating a feature in ArcGIS Online. I am using a change detector and FME confirm that agol is updated. I am generating a list to check the changes. But in AGOL the features are not updated. How can I solve this problem?
Hello folks,I have another question for you , which I'm having trouble solving. I have made an IFC 3d model which I try to import into Revit. Although my walls are redirected to the Walls family in Revit, they each make up an entire family with their own instance. So I figured I would use the example of the LOD 200 Ifc file from the FME blog here in the BIM examples to see if this would give the same result. Now when I import the sample data into Revit I get the same 'error' that I get with my own model. The only difference is that in the LOD 200 sample some of the walls are redirected to the basic walls family whilst others are getting redirected to their own family with their own instance. So my question: How do I get my walls in the basic wall family of Revit. At which point in the converting/importing process is this 'error' taking place and how can I solve it. Because now I can't alter the family of the walls when I import them into Revit. Thanks in adva
I'm constructing an XML query for a WFS service by using an XMLTemplater. The ROOT element is this:<Filter><Or>{fme:process-features("SUB")}</Or></Filter>And the SUB element, per feature, is this<PropertyIsEqualTo><PropertyName>identificatie</PropertyName><Literal>{fme:get-attribute("bag_vbo_id")}</Literal></PropertyIsEqualTo>Store in an attribute then use that feature to trigger a FeatureReader and use the query attribute. This works fine and results in a working query. Except when there's only one feature, in that case the WFS service seems to bork about the unnecessary <Or> </Or> tags.My workaround:StringSearcher to count the number of times </PropertyIsEqualTo> occurs (i.e. the number of sub elements)ListElementCounter on the list of matchesTestFilter to test for the number of elementsIf 1 StringReplacer to replace the <Or> and </Or> with empty stringsI can't help but think there's got to be a
Hi we run into the following issue when executing a number of jobs resulting from an FME Server Schedule on a time that our Oracle database was unavailable. The jobs that read data through an SQLCreator got the same error message ORA-00257 however the job was reported success in FME Server job log. The jobs using Oracle Reader were reported status failure. Other people having same experience? Is this behavior meant to be?
Hi, In my workflow, I am creating an .xls file with some sheets, there are two columns. One of them with the value is with the word sector and corresponding number. Eg: Sector 1, Sector 3 ... How do I enter these values in ascending order. Currently it looks like this: Sector 4Sector 4Sector 6Sector 6Sector 3Sector 3Sector 1Sector 1 Sorting is being done alphabetically, by another column. Thank´s
Having trouble updating attributes on a MySQL db table. We do not receive any errors. The workspace is doing what we want, but not writing an update.
Hi all, I have an excel file with several lines. I have a data field and countries, I need to validate which countries have less than 1 line with data between the last 7 days or the week before, How could I do that?Thank you
Hello, is there a way to import GTFS data into a PostGreSQL database with FME ? Thanks for your help.
Hello FME People,I am in the process of creating a workspace that reads data from geopackage and writes to SQL Server database, after performing a few transformations and checks.I am using Python to perform pre-population of some tables in SQL Server (otherwise FME wouldn't be able to write the new tables due to schema restrictions).At the end of the Python script I run the following statement that calls FME: os.chdir("Z:\\CurrentProjects\\school-uplift\\fme")os.system("""fme.exe geopackage2mssql_spatial.fmw" --DestDataset_MSSQL_SPATIAL_8 "SCHOOL_NAME" --SourceDataset_GEOPACKAGE_12 "..\\gpkg\\Service component type.gpkg" --SourceDataset_GEOPACKAGE_10 "..\\gpkg\\Service component link.gpkg" --DestDataset_MSSQL_SPATIAL_12 "SCHOOL_NAME" --FEATURE_TYPES_2 "ls_carp ls_contours ls_gate ls_landscaping ls_road ls_tree ls_bollard ls_embankment ls_fnce ls_gden ls_path ls_pond ls_steps ls_wall" --SourceDataset_OGCGEOPAC
Problem:I'm having trouble converting 3D Sketchup building data to a more simplified PostGIS 3D Polygon (ST_Polygon or ST_MultiPolygon). The SketchUp buildings data are not represented as 3D volume representations but made of series of parts representing the building which need to be recreated to polygon objects and output to POSTGIS DB.Aim:We need to reconstitute the building parts to solid polygon 3D representation losing the separate SketchUp parts of faces, edges and, vertices. Our aim is to simplify the building to show the outer most extent (removing detail/vertices within the building).The Process:We're using the GeometryPartExtractor to get the SketchUp data into a usable format and geometry coercer to fme_polygon. The complexity of the vertices and ordering of them has meant we get some unwanted results when re-constituting the geometry after partextraction and using PostGIS geometry conversion or geometry fix commands (ST_MakeValid). The FME Inspector seems to rende
I have created a new azure installation following this guidance:https://docs.safe.com/fme/html/FME_Server_Documentation/AdminGuide/Kubernetes/Kubernetes-Deploying-to-AKS.htm When I look at the logs to my fmeServer Postsql service from powershellkubectl logs fmeserver-2019-1590072608-postgresql-0I get the following with authentication errors:INFO Initializing postgresqlpostgre INFO ==> No injected postgresql.conf file found. Creating default postgresql.conf file...postgre INFO ==> No injected pg_hba.conf file found. Creating default pg_hba.conf file...postgre INFO ==> Deploying PostgreSQL from scratch...postgre INFO ==> Creating postgres user with unrestricted access...postgre INFO ==> Configuring PostgreSQL...postgre INFO ==> Configuring replication parameters...postgre INFO ==> Configuring permissions for config files...postgre INFO postgre INFO ########################################################################postgre INFO Installation parameters
Hi there - We're simply trying to push an SDE feature class into a zipped GDB using the "Insert" and "Truncate Existing" options. However, when we do this we receive a "Feature class must exist" message. The alternatives of using "Create if needed" and "Drop and create" aren't ideal because they recreate a Feature dataset that we don't want to see in the output. Are we doing something wrong? Or is it simply that FME cannot read a zipped geodatabase?Thanks!
Hi TeamI have been excited to use the dgn reader in 2020 that allows you to view Item Set Attributes. It does work in one regard however the issue we are facing is that the item name (attribute) gets prefixed with the set name and therefore you end up with 100's of attributes, even though there may only be really 10. Besides using an iterative Bulk Attribute rename has anyone got any other ideas?You can see in the screenshot below that there are about 15 different attributes (ie Asset, Count, Comments etc) however they come through as a separate attribute for each item set. Thanks very muchDominique
Hello,I'm implementing a workflow where I need to move data from a file geodatabase to a versioned SDE geodatabase. In this workflow, I need to keep GlobalID values while moving records from source to destination.I set the "Preserve GlobalID" to "Yes", but I'm getting the following error:"Could not unregister GlobalID column 'GlobalID' in table 'Table_Name'. The error number from ArcObjects is: '-2147212936'. The error message from ArcObjects is: {Unregister GlobalID column is not supported for versioned or replicated data.}" Appreciate if you guide me to any solution or workaround for that. Thanks
HiNew to FME. Familiar with ETL/ELT tools.I would like to read rows from a Postgres DB and load them into Snowflake DB.I want to do it incrementally, i.e. read from Postgres only the rows that have changed since the last time the Workspace run. I have some timestamp field in the table, named "last_updated".The query should look like: "select * from table where last_updated > /previous_max_last_updated",I thought maybe doing it using a special management table, that will hold the values of last_updated for each source table, but it will require some more reader and writer in the workspace. Is there a way to hold variables from session to session? I mean, from 1 workspace run to the next? Thanks!
I have a File Geodatabase (ArcObjects) Writer that points to a FileGDB I want to update. When the Writer Mode=UPDATE, the feature count from Reader to Writer is very slow, at a rate of ~1 feature per second. If I were to perform DELETE first and then INSERT as a way to update, then it's very fast. Why might UPDATE be so slow? There are 2 bottlenecks I think might be occurring:The field I'm updating on is not a unique field - This field represents a "tile" of records, by which I want to update on a tile-by-tile basis.Grid 1 is set to 100. When updating database tables, it's recommended that one removes spatial indexing, perform update, then recreate the index. Is there a way to do that with the FileGDB ArcObjects Writer? Might I have to use PythonCaller to call arcpy.RemoveSpatialIndex()?
I have a workspace that runs just fine on deskop, version 2019.2.I publish this workspace to FME Server, version 2020.2 and I have two problems:It is VERY slow. The whole workspace on desktop runs in 2.5 minutes. On Server, it runs for over 20 minutes. The part that failed, buffer and snipper, takes about 5 seconds to do on desktop with about 4000 records. When it fails on server, it has only processed 1000 records in the 20 minutes.It erred out with this cryptic error: 856 FFS writer: Failed to open file '' for writing. Please ensure that you have sufficient privileges to write this file. If the file already exists, please ensure it is not open in another application857 UnitSegmentCreator_Centerline_Input1589901888 Input Splitter (TeeFactory): Internal Error: Failed to build feature table858 Closing the Geodatabase reader859 Internal Error: Failed to build feature tableI'm not sure if the two are related. I would appreciate any help on how I can debug where the issue might
Hello all,I have been trying to convert a topjson/geojson file to a cesium 3d tileset. I have been able run the conversion but i cannot get it to display in cesium. I feel that this may be caused by the coordinates not being converted correctly. I cannot figure out how to fix it. I tried using no coordinate system selected and LL-WGS84, but neither worked. I did notice the area cesium zoomed to changed but i could still not see the shapes. Any help would be greatly appreciated. Below is a link to the geo and topo json files i tried. https://le-temp-public.s3.amazonaws.com/counties-us.topo.jsonhttps://le-temp-public.s3.amazonaws.com/counties-us.geo.json Again thanks
Hello.Can I read data from AGOL using the editor account? I see all the objects when reading and writing data (hosted) with the admin account. Objects are shared with the editor like for "organisation". But when I add a reader / writer and try to read the available objects, I get a notification Note. I use FME 2017.1.0.0 (win 64).
In my workspace, I've three ESRI GeoDB reader Feature Types with WHERE condition. Here values in the WHERE condition are passed from a private/published parameter input values.Few things to be considered -There are approximately around 100 values to be passed in sequential order from Excel sheet to Parameter values.Workspace has to run completely for every input parameter value Currently I was passing each value manually to the parameter and is time consuming.Any thoughts on doing this more efficiently and quickly? Thanks in advance!
Hi, We have implemented a FME script that an ESRI GIS database on several computer. However the script fails on one computer and succeed on all the other. Here is the error messge return in the FME log file.2020-03-25 14:45:29| 23.0| 0.0|ERROR |FFS writer: Failed to open file '' for writing. Please ensure that you have sufficient privileges to write this file. If the file already exists, please ensure it is not open in another application2020-03-25 14:45:29| 23.0| 0.0|ERROR |BADNEWS: Internal Error: Failed to build feature table class StatusInfo *__thiscall STFFeatureTableWrapper::WrapperFeature::FTFeature::initChangedValuesWriter(void)(c:\\code\\fme\\foundation\\framework\\engine\\stffeaturetablewrapperfeatureftfeature.cpp:764) - class StatusInfo *__thiscall STFFeatureTableWrapper::WrapperFeature::FTFeature::initChangedValuesWriter(void)(c:\\code\\fme\\foundation\\framework\\engine\\stffeaturetablewrapperfeatureftfeature.cpp:764)2020-03-25 14:45:29| 23.1| 0.1|INFORM|Cl
Hi,I have been trying to create a multi-component polygon consisting of line, linestring and arc. The arc parameter consists of Start/From, Centre, End/To, the Radius in metres as well as the arc direction, clockwise or anti-clockwise. Negative radius indicates anti-clockwise arc. Using the python script posted by @takashi in here, I was able to generate most arc except for some instances. In the sample data, I have included 3 polygons. While ID=47 works fine, the other two polygons (ID=407 and ID=482) failed. I tried the Arc Property Setter method in the same post, unfortunately it yields an undesired gap at end points. I have attached the csv file containing the 3 polygons records and the FME workbench. Any help will be much appreciated. Thanks in advance.Lawrence Correct output polygons for ID=407 and ID=482 as shown.
When publishing to ArcGIS Portal the layer has some default configurations that I would like to change. For example, share to a group/everyone, disable editing and prevent the layer from being deleted. Any ideas in how to do that using FME?
I am working on land xml coding in primary stage and I need some samples of basic geometry (point, line, polygon etc.)
It is possible to get the units of a Revit model with the Native Revit reader? I can't find it defined anywhere. Anybody know if this is anywhere? https://www.cad-notes.com/revit-units/