Using readers and writers to extract and load data.
Recently active
Currently using FME 2018 Desktop. I have a workbench using the AGOL Writer to write a large data set to an ArcGIS Online Feature Service layer as follows:Writer mode: INSERTTruncate First: YesFeatures per request set to 1000When executing the workbench, sometimes it works and sometimes it Fails and gives the output time out error 504. What's the best approach to optimize the AGOL Writer or the workbench to avoid timeout issues with AGOL? Thank you.
I have a feature layer in AGOL. What I try to do is when there is a new record added into the layer, the FME workspace published into the FME server can auto detect the changes, then trigger the workspace to run. FME Schedules can do the job, but not idea. Any ideas? Thank you.
I'm reading an Excel file with UUID fields looking like this {c52aff68-61fb-4004-bd58-84a3eb35e878}. How do I transfer them to an Oracle raw(32) fields in the Writer?
I am trying to use a WFS FeatureReader. I'm not getting any errors and it is outputting the schema and initiator but not the actual layer that I have selected It's almost as if it isn't in the correct coordinate system however I have cheked multiple times and with multiple WFS links and it's just not working. I am trying to select the "WFS Output Format" but there are no options in the drop down menu. I'm wondering if this is the issue? Or is anyone else having the issue of WFS FeatureReaders not wokring. I'm using FME 2019
I am writing data to an AGOL feature but when the process runs, one field of the data does not get written. Everything else writes properly except for the one field. The field types are correct as other similar data is written out during the process.Any ideas?I have included a screen grab of the data preview in the reader, and the data table in AGOL.Thanks,--gary
Hello !I am trying to import an IFC model into Minecraft, but some parts are causing a fatal error :PointCloudCombiner___VectorMerger(VectorToPointCloudFactory): The input geometry could not be converted to a pointcloudA fatal error has occurred. Check the logfile above for detailsI get this error with my own file, but I also get it with the file from this article.More specifically, these two readers are causing the problem : IfcCurtainWall and IfcWallStandardCase. When I disable them, I get the desired output, minus these two elements.this is what I got with my successfull export, you can see that the walls are missingI'm very new to FME, so I must be doing a basic mistake. Can someone help me figure out which one ? Thanks !
I have never used FME before and I am just trying to view a file I need for a report but I do not have the correct software to open an .las file. How can I convert it to a .pdf so I can open it?
Hello,I have an FME 2018 workbench with an AGOL Writer set up to write to a Feature Service layer in AGOL by Writer Mode: Insert, Truncate First: No (I want to keep appending different records to the table). (See screenshot below)Steps to reproduce the FME warning (error):1. First time run the FME workbench from a January data set csv file that gets geocoded to arcgisonline_polygonResults: All column fields get written to AGOL geodatabsae and feature service layer perfectly fine2. Second time I run the FME workbench from a February data set csv file that gets geocoded to arcgisonline_polygonResults: FME gives me a warning: "ArcGIS Online Feature Service Writer: Attribute 'PLAN' on feature type 'Land_Lots' does not match any field on the corresponding layer/table. The attribute will be ignored"The workbench translation is still successful. However, when I look at the table in AGOL, I now notice that the second time I had ran the FME workbench, the PLAN field go
In FME 2018 when reading out the SQL_STATEMENT from the Transformer Parameters reader, the SQL comes out formatted roughout like this example:SELECT<space><lf>SITE_ID_FS<comma><lf>LIFEFORM<comma><lf>PLANT_CODE<comma> <space><space><space><space><space><space><space>from<space>my_table<lf><space> where <space> LIFEFORM <space>= <space><apos>TR<apos> Is there an easy way to decode this back to a simple text that will run again in a SQL window ?It looks like FME 2019 handles this better, but I am limited to 2018 for now.
Hello,I tried to add attributes from Esri File (Points-at the end it were no Points, just attributes = sum of counts etc) to the Esri Polygone. It has everything worked, till the last result form aggregator. If I check result in Inspector, the result is there as I wish. But when I inspect the writer, the result is empty... I checked the Shape in ArcGIS Pro, and there is also nothing. Any ideas why? Thanks
Hello,I 've processed some data sets into one using FME 2018 and now I'm trying to write this data set to AGOL. On the AGOL account I don't have Content (Feature Service (Layers). When I insert a Writer (AGOL) I manage to 'Add a Web Connection'->Connection Name->Authenticate, which then opens a window to ArcGIS Online and I insert my credentials. Once this is complete, it takes me back to FME.I can't finalize the Writer (See screenshot below) because I need to fill the Feature Service. When I select the Feature Service it just loads for a long long time not doing anything (see screenshot below). Am I supposed to create an empty feature service in AGOL first, then attempt the Feature Service connection (See screenshot below)?I was under the impression once I finalize the Writer "If the Feature Service directive refers to a Feature Service that does not exist, and the writer mode for all feature types is INSERT, then the ArcGIS Online Feature Service Wr
Hello, I'm trying to write tags to a dgn-file on 3D solids which i have generated in FME but no tags appear. It works fine if I use other geometry types i.e. igds_shape but then I do get a lot of features which I would like to avoid. This message makes me want to ask if anybody out there has found a workaround: DESIGN WRITER: Tag writing is currently not supported for the associated element Best regards Olof
I have an AttributeFilter that creates multiple Esri shapfiles (filter by an attribute value) that use one esri shapefile writer to write out each shapefile. Is there a way to compress (zip) each of these shapefiles individually? I know there a shapefile compression option to create a .shz, but a .zip file is required.
I would like to transfer the fmesuperuser role privileges to an Active Directory role.In the FME Server documentation, there is an article about how to do this using the SECURITY_SUPERUSER_ROLE parameter in the fmeCommonConfig.txt configuration file. Unfortunately, this does not seem to work. The parameter line mentioned (SECURITY_SUPERUSER_ROLE=fmesuperuser) is not present in the configuration file. Also just adding the line as a parameter does not seem to work.The answer in the article already present in the knowledge base is not sufficient, because this only concerns assigning an Active Directory user to the role.I ruled out errors in the Active Directory distinguished name by testing with a test FME Server role (non-AD) and using this in the configuration file (e.g. SECURITY_SUPERUSER_ROLE=testsuperuser).How can I accomplish transfer of privileges on the AD role?
Hello we installed the new FME 2019 and when i try to use the LDAP reader i recieve the following error:Is there something required to ahve java work in the new version of FME? The workspace may run slower because features are being recorded on all output ports.The Java Virtual Machine could not be started. CLASSPATH = '-Djava.class.path=C:\\Program Files\\FME\\common-webservices.jar;C:\\Program Files\\FME\\plugins;C:\\Program Files\\FME\\plugins\\commons-compress-1.17.jar;C:\\Program Files\\FME\\plugins\\dropbox-core-sdk-3.0.11.jar;C:\\Program Files\\FME\\plugins\\hsqldb.jar;C:\\Program Files\\FME\\plugins\\ili2c.jar;C:\\Program Files\\FME\\plugins\\ili2fme.jar;C:\\Program Files\\FME\\plugins\\jackcess-2.1.11.jar;C:\\Program Files\\FME\\plugins\\jackson-annotations-2.9.8.jar;C:\\Program Files\\FME\\plugins\\jackson-core-2.9.8.jar;C:\\Program Files\\FME\\plugins\\jackson-databind-2.9.8.jar;C:\\Program Files\\FME\\plugins\\jackson-dataformat-cbor-2.9.8.jar;C:\\Program Files\\FME\\p
Working with esri and the developers for ArcGIS Online, we came up with this script that can be used on shutdown. You will need to write data out to a zipped file geodatabase (or other formats), then configure the script.Only works with 2019.x+ as it requires you to use ArcGIS Pro (tested in 2.3). It does take a while to run (as it has to use the ArcGIS Python library), but is SUPER efficient when trying to write LARGE datasets to ArcGIS Online. It is much preferred over using the FME writer because that one uses the traditional database insert for each record while this is similar to bulk inserts. Can probably be modified for writing to Portal as well.https://github.com/runneals/GISProjects/tree/master/agol/Appendcc @carsonlam @bruceharold @mmccart
Hi All, we are trying to export data from Smallworld to Mapinfo Tab format. In Smallwrold we have objects like Underground Route and Cables, which are having length fields with values available in millimeters, but these values are getting exported as meters to Mapinfo tab, how to keep the values as it is while exporting. Please help. @takashi please help in this regard. Thanks in advance.
I have a pdf that has blank boxes that are editable. Is there a way to write to them? Attaching sample pdf to show how the pdf would look.
I have geometric data that I want to join with non-geometric data. From what I read and noticed, a FeatureJoiner won't keep the geometry during a join. I would use a FeatureMerger, but the cardinality of my features won't make it work. I tried aggregating one of my feature to make it possible for me to use the FeatureMerger, but the aggregations are not supported by ArcGisOnline (that's where I want to write). I ran out of ideas to solve this problem, do you have any solutions? Thank you
One of our customers still uses v2007 of FME Workbench and I'm having a bad time trying to do what I find so easy with latter versions.Workbench help doesn't help much and I can't find examples on the internet so if anyone has a clue on how to do it I'll be very grateful.I want to use attribute values in sql statements but SQLExecutor transformer in FME v2007 looks too basic and I don't know if it's even possible to do such a thing. The image below shows what I tried and doesn't work at all. I get error: BADNEWS: Database Writer: Commit failed -- [NOTSUPPORTED] commitTransaction not supported by core library
Morning,Does anyone know if there is a way of taking monthly excel reports and adding them to a single google sheet with monthly tabs.ie Monthly calls for each month are produced in excel - what we would like to do is when a new report is put into a centralised location fme server will detect a new spreadsheet and then the workspace would run and add the new month to the google sheet. Thanks
Is there a way to retrieve the actual record that fails a database constraint when trying to insert in an oracle table (using Oracle Non-spatial writer) instead of the SQL statement? It seems as if FME only outputs the SQL statement that fails and the oracle error message, but can't seem to find a way to get the actual record that fails a constraint.
Hi @markatsafe, I am trying to read in a Microsoft SharePoint List (from a server not sharepoint online). I have tried in both FME 2017.1 and FME 2018.1 and am having no luck. Can you please review the error message and let me know what the issue may be? I have been told that I have read-only permissions to the list.Also, what do I put in the 'Site:' parameter? Is it the link to the actual list itself that I have read-only access to, or is to the 'Lists' folder or the base URL of the whole site? I am wondering if the issue is related to me only having access to one list on the site....Thank you in advance
Hi, I've a got point layer in shp format and a pointcloud file in las format and I'd like to find the nearest point in the las file for every point in shp file. I tried to use PointCloudCoercer to change las file into mutipoint, but z value of points won't be merged. Then I tried to change las file into individual points, but it was too slow, I got thousands of las files and each of them contains over 1 million points.Can anyone provide a right and fast solution to this matter? Thank you!@takashi @jdh @danatsafe @mark2atsafe
Hello!I have nested blocks on autocad because I need to have the same block with the insert location on a other place. So, on autocad, when you explode the block1 you obtain the block2. I cannot change this configuration because is used for a long time. The 2 blocks have the same name.Is it possible to extract the block nested (block2) without explode it? Or at least extract the insert location of the block 2? When I explode the block on FME reader parameters, FME explode all the elements and keep only the insert location of the block1. There is no option of "hierarchical" explosion.Thank you!