Using readers and writers to extract and load data.
Recently active
Hi, can FME write PDF/a format files? I don't see any options in the Geospatial PDF writer.
Can any one provide guidance on how to convert a .TML file to a ESRI shapefile for feature class? Unfortunately I'm not able to upload the .tml and schema to this forum. They can be accessed here: https://drive.google.com/drive/folders/1pnvv5YCRYUE9h5OOt1dNf3jAChWL686F?usp=sharing
Hello,I need to compare strings from two sets of data and to find strings which contains the strings from other set. For example, in first table I have strings: column1ABCABCDABBABBDACCACCD In second table I have stringscolumn1ABCAC I need to compare them and have the result something like this:column1; column2ABC; ABCABCD; ABCABBABBDACC; ACACCD; AC Rows without data in second column may also be omitted. What would be the best approach?
Hi Community!I'm trying to access and read/upload/download data from a S3 compatible cloud storage. Since the cloud storage isn't an S3Amazon i cannot get the S3Connector to work. The S3 compatible cloud storage is equivalent to S3Amazon storage and should be compatibleDoes anyone else know how to do this?
Hello all.Last year, I delivered a number of Shape files to a customer, which they had no problems in using.Now I've adapted another of our systems to create Shape files as a supplement to an email message.These latter Shape files cannot be read by their ArcGIS Pro. The error message is "Failed to add data: ... A user-supplied component or subscriber raised an exception (Exception from HRESULT: 0x80040208)". I've also tried to apply Geopackages, but they err the same way. Both output routines uses FME. The former used straight up fixed writers, and the latter uses FeatureWriter, but that shouldn't matter, right ?I cannot see any difference between the two Shape files, and QGIS reads them both without any qualms. Apparently they can read the offending Shape into QGIS, and have it save it as another Shape, that works in their ArcGIS Pro.I cannot see any difference between the two flavors (in QGIS), except that one fails when input to ArcGIS Pro.Does anyone have any
My last pair of FME socks from the 2017 UC are on their last legs. Probably the last day I'll be able to wear them. I'll have to put them in a safe place until I can put them on for the UC in August. Anyone else need some sweet new Safe swag?
Which FME requires to connect with SAP HANA ?i have FME Desktop , is there a method to connect with SAP Hanai wish that someone shares his experience to integrate FME with SAP HANA and also which ideas and problems that he faced .Thanks in advance .FME Desktop 2020
I am having trouble connecting to the API at https://developer.vasttrafik.se/portal/. I would like to get the public transport stops (https://developer.vasttrafik.se/portal/#/api/Geography/v1/admin), but I am stuck, I keep getting "HTTP/1.1 403 Forbidden" from the HTTPCaller. I have tried using "Authorization" as suggested at https://community.safe.com/s/question/0D54Q00008WImfCSAT/using-fme-to-call-power-bi-via-the-api-to-refresh-data-sets-dynamically-i-am-stuck-with-the-authentication-piece-for-the-bearer-token, but no success. Any suggestions on what I might be missing?
Hello Brains trust , hoping someone out there can help me. My workspace is as follows.2 arcsde feature readers. sde1 contains records and attributes that are pulled from a number of sources to create areas of interest.sde2 contains one of the source datasets that are used to update sde1. sde2 is also a source of truth dataset so the geometries are the most current.nb - the sde's have been maintained using ESRI not FME - trying to automate the process. My work flow checks the geometries of sde2 against sde1 and were they don't match I use sde2 geometries to UPDATE the sde1 records (i use the fme_db_operation attribute to identify that it is an UPDATE and i keep the attributes from sde1). Simple enough. However, some work and some don't when writing (have been using a process of elimination to narrow down the records that write and don't but nothing in particular is jumping out). The error FME is giving for the records when it fails to write is I have written the compl
I'm trying to speed up the processing speed. There might be a few different ways to do this. I'm trying to make incremental changes to a workspace and every time I run it, it takes about 40 seconds. I have two tables in ArcGIS and I use a FeatureJoiner to join them in FME. One of the tables has a data field. I'd like to only process a range of dates. I have about 8,000 records total and I'd like to limit it to 1,000 or so records. Once I get my workspace functioning correctly, I'd like to remove this date range limit. I'm very new to FME.
While connecting to MongoDB Atlas I'm getting error TLS Handshake failed. However, when connecting via MongoDB Compass, the connection is successful. Attached is the screenshot of the error. Could anyone please share MongoDB Atlas is supported via FME? If yes, how to connect to it? Best,Sarthak
I have a workspace which reads rows from an excel file and writes the same rows to an other excel file using dynamic schema definition. Gets the schema from the excel file which is read. My problem is that the first row that is read is skipped in the writer. In the attached example the output will only contain the second row of TestExcelWriter.xlsx
I'm running into a problem with the Google Sheets (V4) Writer. We have a Tableau Public Dashboard connected to a Google Sheets Document. Tableau Public will only daily update it's data when coming from Google Sheets. Other data formats are not supported.To keep our dashboard up to date I'm writing a selection of the last 24 months of data from our internal database to Sheets. About 1,5 million records. Using a 'truncate existing' mode. Only 1 column of data. All columns from B to Z have been removed from the sheet. Writer is set to end at column A . So no extra colums are created.When starting with an empty sheets this runs perfectly fine but after a week of daily truncates the sheet contains nearly 10 million rows. Only 1,5 million contain a value, the rest is empty. So not a real truncate, just a 'delete value' is performed. The next day my workspace crashed because the sheet will exceed 10 million cells...A drop and create won't work either because it
I am importing an AutoCAD DWG with multi-leader blocks for asset locations. The issue I am running into is that the value showing up in FME for the "TAGNUMBERS' is the Default value vs. the value that shows up in AutoCAD. From AutoCAD: I'm trying to extract the 'T9' text from the MLEADERs shown here: These are custom MLEADERS developed by the drafting department. After different attempts to explode blocks, expose attributes both in Model and Paper Space, different combinations of parameters, etc. I attempted 13 different combinations to import the DWG and now luck, I keep seeing this in FME: FME is only recognizing the 'Default' Tag Number. I've tried to bring in both Model and Paper Space in a single reader to match Entity Handles, but no luck. I also tried to leverage the multitude X/Y coordinates and came up with nothing. If I explode the MLEADERs in AutoCAD first, the 'T9' shows up in the 'autocad_text_string' attribute as ex
As the question suggests I cant seem to find instructions on how to convert LAS scans to GLTF format and retain the color_red, color_green, color_blue values. I've expereince with successful conversion of CAD models to GLTF but the resulting LAStoGLTF refuses to play ball. I've tried various combinations of reading LAS > use PointCloudPropertyExtractor to expose color data and even forced the issue with PointCloudLASClassColorSetter but all to no avail. The resulting GLB is created using default white colors leaving me with non-viewable model. Is this translation just too new a possibility for GLTF writer or am I missing something fundamental here (most probably I hope!). Thanks for any advice.Source LAS from Drone:Resulting GLB model:
module skp/skp is unavailable for use with this fme edition
Hi everyone,I have a series of regularly produced Excel files, each containing a number of sheets. The number and name of the sheets in each file isn't fixed.I can read data from all the sheets no problem by keeping the reader parameters blank - but annoyingly for these files the header starts on row 2 and in some cases junk is written into row 1. As such, FME picks up attribute names from row 1, so for one sheet the attribute for column B might end up named"995" and for the next sheet it might be "376", so I end up with a lot of attributes where there should be one - ideally just called "B".Is there any way to force the Excel reader to always use column letter as the attribute name regardless of how many sheets are in a file and what they are called?
https://www.railml.org/index.php/en/
Hi, I'm trying to figure out how to write nested attribute structure to a GEOJSON file. For that reason I've prepared some sample data for a Quick Translation to see how FME would set up the writer feature type properly. But surprisingly, my nested structure is lost during that translation! So in fact my test endet up in two questions instead of delivering a solution:1.) Why does the Quick Translation fail2.) How do I set up the GEOJSON Writer properly to build the desired nested structure.2.1) In case JSONTemplater is the solution to this task, how do I have to set it up properly? I spent some time playing around but never came up with some suiteable result, especially for proper GEOJSON (geometry handling, type defintion and so on). Find attached my sample file. Looking forward to any suggestions (and maybe solutions 😁 )
Hi,I would like to Read a XML and convert to csv file, but preserving the sequence of Attributes from XML.The Structure XML:When I create a just workspace Reader ( XML ) to Writer ( CSV ) respecting Dynamic Schema, the Attributes are'nt in order.Thanks,Danilo
I have had FME writing to an ArcGIS Online Feature Layer, working no worries. I then decided to 'tidy up' my field names and alias, so I dropped a few fields and recreated with more appropriate naming convention. So in my FME workbench I have removed and then re-imported my feature type, thinking it would pick up my changes.Note: the fields were dropped and recreated using the AGOL web interface. Since reconnecting/importing the feature type and renaming the fields accordingly in the writers User Attributes I am getting the below errors, and I cannot work out why the workbench is still trying to write to those old field names. Below is the current field names, these were originally slightly different, as shown in the errors below. Here is my updated User Attributes panel, these fields match the above updated AGOL fields (as far as I can tell) But when I run it, I get the below errors, these line up with the OLD field names I was using. 2020-04-08 19:32:06| 1.4| 0.0|INFORM|ArcGIS Onl
When using the ArcGIS Online Feature Writer, sometimes during truncation, AGOL will encounter an error on delete. It looks like this:2022-01-12 18:39:01| 25.9| 0.0|INFORM|ArcGIS Online Feature Service Writer: Truncating layer/table 'Meters'. 2578 features remaining2022-01-12 18:39:39| 26.1| 0.2|INFORM|ArcGIS Online Feature Service Writer: Truncating layer/table 'Meters'. 1578 features remaining2022-01-12 18:40:47| 26.2| 0.1|WARN |ArcGIS Online Feature Service Writer: 'deleteResults' error for a feature in 'Meters'. The error code from the server was '1018' and the message was: 'The specified feature could not be deleted or does not exist.'2022-01-12 18:40:47| 26.2| 0.0|WARN |... Last line repeated 315 times ...2022-01-12 18:40:47| 26.2| 0.0|ERROR |ArcGIS Online Feature Service Writer: 6000 features successfully deleted from layer/table 'Meters', but the server rejected the 1000 deletions in the last request due to errors. See warnings above. Aborting translation2022-01-12
Hello everyone, I should start by saying I'm an FME beginner user. I'm currently toying with an old ESRI coverage dataset that still has some importance. The files are split by index (grid) and so each layer is stored in a separate folder like shown below. My goal is to extract the data from only one theme ("batim_p" which is building point data) and to merge it into a unique layer. Although the schema is always the same, it is possible that some index folder do not hosts the data folder. The thing is that I have an arcpy script that does this within about 40 mins but the FME process I constructed (which basically is only a feature reader) takes almost 10 hours to do about the same thing. I believe my problem comes from the fact that the reader scans EVERYTHING, although I have only selected the feature type of interest in the initial parameters. So I tried to work around this by using a Path/Directory reader to extract the folder name and THEN use a featureReader t
I've been noticing something and having trouble resolving it. I have a workflow that runs every half hour, it runs fine however I noticed that it is creating a legend.tiff file in temp in a numbered folder within /tmp. This does not seem to clear and is about 1GB. I would like if this file was not created or at least removed when the process finishes. It will eventually create too many of them and lead to the disk getting full. I can remove them manually or with script on intervals, however I feel there must be a way for FME not to create them. If any one has any tips on what transformer is creating them that would be great.
I'm using an SQL Creator to return data from three tables (Microsoft SQL Server), one of which contains a geometry column. Initially, I was just joining two tables and this all worked as expected with the geometry being correctly displayed within FME. When i amended the SQL to include a third table, the geometry column seems to have stopped being interpreted as geometry and is now displayed as an attribute containing binary data. A bit more investigation, if I limit the fields being returned by the SQL creator, the geometry is interpreted correctly (bottom SQL creator). If i attempt to return all fields (explicitly naming or using *), the geometry field is returned as an attribute Additional information, only the table I'm attempting to get the geometry from has a geometry column. Am I missing something obvious? FME(R) 2020.2.2.0 (20210111 - Build 20817 )