Using readers and writers to extract and load data.
Recently active
Is there a way to publish a feature class service to ArcGIS server without having ArcGIS Portal set up?
I want to know the value of a raster at specific points using the Point on RasterValueExtractor. Inputs are a raster and a point geometry shapefile (generated in another FME workspace).I simply connect the raster and point shapefile in the PointOnRasterValueExtractor (they have the same coordinate system), where they should be connected. Yet it gives the following error:PointOnRasterValueExtractor_2(VectorOnRasterOverlayFactory): Invalid point feature or feature with non-point geometry encountered. Only vector features with point geometry are currently supportedAre point shapefiles not supported? Do I need to transform my point shapefile in some way before using it for the RasterValueExtractor?
Currently in the process of trying to upgrade from FME Server 2015 / ArcGIS 10.2 to FME Server 2017/ArcGIS 10.5. We have around 400 scheduled replications that mostly write to an Oracle SDE database using the SDE30 writer. About 150 of these FMW's crash when run on the new environment. The log writes some warning messages (that I do not believe are related to the crash as they appear in the 2015 replications as well) and then silently crash. I have tried running with FME workbench 2017, and this process does not provide any additional logging.The warning messages before the crash are:FME Configuration: Using FME Reprojection EngineCreating writer for format: Esri Legacy ArcSDEFME Configuration: Destination coordinate system set to `BCALB-83'Coordinate System `BCALB-83' parameters: CS_NAME=`BCALB-83' DESC_NM=`Albers Equal Area, Province of British Columbia, Canada' DT_NAME=`NAD83' GROUP=`CANADA' MAP_SCL=`1' ORG_LAT=`45' ORG_LNG=`-126' PARM1=
I recently updated Robyn's "Using the FME JDBC Format" article, adding a few more formats and a new category for the JDBC formats that FME now supports more completely, including documentation.It got me thinking... what have others attempted to do with the JDBC Reader/Writer? If you have any experiences - I'd be curious to hear about them and I could update the article.
I want to copy data from a source into a PostgreSQL table and I want the content of the target PG-table to change only if the whole operation is successful. My approach is to get the truncate and all inserts done in one single transaction. One way to do this is setting “Features Per Transaction” to >= number of features in the source (some hardcoded value, perhaps 9999999999999999999999). But is there a better way? In the documentation I’ve found info on setting “Features Per Transaction” to VARIABLE in combination with using a fme_db_transaction attribute. I’ve tried this, both setting fme_db_transaction to COMMIT_AFTER and to IGNORE. But the log file still says that there are 1000 features per commit (=the default number).
Hi,I'm new at FME so I'm asking for some help. I have autocad layers that contain a three letter code (ex: <Project>-<Beginning>-<End>-<CODE>-...) and I have an Excel sheet that contains a list of Codes.I'd like to check if the code in the cad layer exists in the excel list. Then fill a report (excel) with a new feature called "CodeCheck" and the value would be Y or N.I'm using an attributeSplitter to get the code but I can't find a way to check if the code is in my excel file. I must get the matched AND unmatched in my output (only the cad layers).I tried Testers, filters, change detector,... but nothing end up like I want.Please, how do I check the code and fill my report ?Louis.
How do I figure out how stops are ordered when I use ArcGISOnline Router? I get the optimized route, but I am trying to figure out how to show the sequence of stops in the route.The information is in the Nodes dataset, but that is generated only if the Return Directions feature is enabled. Also the information is more implied than directly available. It needs some processing. Is there a more direct way of getting to the sequence of stops?
I have some LIDAR data to work with, but need to removal artifacts, noise and add classification. Does FME deliver the appropriate tools to do this?
Hello,I would like my FME script to download some data directly from OSM (probably using Overpass) when executed. At first I wanted to get the result from a specific Overpass QL request (using HttpCaller ?), but I don't know how to do that.I have tried the OSMDownloader, but the resulting file has no geometry ... And even if it would work, how to wait for the file to be entirely written before reading it with a reader ? Or is there any other way to do that ?I also tried a solution from friesewoudloper, but I don't understand the XSL template (XSLProcessor) and how to use it the way I want ...Any help with that issue would be much appreciated :) Thanks in advance
I'm using FME Desktop 64-bit (v 2017.1.2.1) and trying to read a large CSV file (approx 1gb and 60 million records) into a PostgreSQL database table with no geometry involved. The only transformers I am using are an attributerenamer to map the fields and two expressionevaluators to calculate KM/hr and Metres/Sec from Miles/Hour (see below).My issue is that running the workbench consumes most of the 8GB RAM available on the VM and this appears to be due to the CSV reader reading in all the records prior to the first transformer (the attributerenamer). I ran the workbench overnight and it failed due to an FME app crash at approx. 52 million records processed and I'm assuming this was memory related as the last few lines of the log are: 2018-03-03 01:20:55|17816.4| 1.0|STATRP|AttributeRenamer(AttrSetFactory): Processed 52421236 of 59977315 features2018-03-03 01:20:56|17818.1| 2.8|STATRP|ResourceManager: Optimizing Memory Usage. Please wait...2018-03-03 01:20:56|17818.1| 2.8|IN
Hi advanced experts~Just reading of my kmz file generating error message in a row as follows;HTTP transfer summary - status code: 200, download size: '20 bytes', DNS lookup time: '1e-6 seconds', total transfer time: '0.266 seconds', url: 'http://cloudrf.com/API/ppa/rx.php?uid=20574&f;=0220003602_Dairy_Flat_1a.4326&'KML: The file `C:\\FME_TEMP\\FME_OGCKML_DLTEMP_1520219068411_14984\\FME_KML_1520219072179_14984\\rx.php' has an invalid or unsupported extension `php'KML: The url `http://cloudrf.com/API/ppa/rx.php?uid=20574&f;=0220003602_Dairy_Flat_1a.4326&' was downloaded, but the resulting file, with mime-type`text/html', does not appear to be a supported file typeany experience in this?please your experience..ThanksDavid
I want to read csv files with different schemas but with some attributs which are part of all csv files:file1in: attribute1, attribute2, attributeA, attributeB, attributeCfile2in: attributeD, attributeE, attribute1, attribute 2, attributeF, attributeGThen I want to do some processing and write the files with some additional attributes to new files:file1out: attribute1, attribute2, attributeA, attributeB, attributeC, attributeX, attributeYfile2out: attributeD, attributeE, attribute1, attribute 2, attributeF, attributeG, attributeX, attributeYFile names for output are the filenames of input with additonal postfix like "_transformed":<filename1in> => <filename1in>+"_transformed" (<filename1out>How can I do this? I have tried to use a csv writer with dynamic properties but then I get resulting filenames of "CSV". If I change the feature type names from "format names" to "file names" in the csv reader, then I get the fil
I have tried using the old SDE connection file and created new SDE connections. Both fail when using the FME reader Esri Geodatabase (ArcSDE Geodb). The following is the error message I receive:================================Opening the Geodatabase readerThe ArcObjects license 'Advanced' is being selected from the ESRILicenseInfo ArcObjects callFME has checked out an Esri license. The product checked out is 'Advanced'Installed ArcGIS version is '10.4'Could not open the Enterprise Geodatabase. The error number from ArcObjects is: '-2147209112'. The error message from ArcObjects is: {FDO error: -2147209112}Please ensure the connection parameters are correct (server=`gisnas1', instance=`sde:sqlserver:gisnas1', user=`***', password=`***', database=`Cadastral_SDE', version=`sde.DEFAULT')A fatal error has occurred. Check the logfile above for detailsFailed to obtain any schemas from reader 'GEODATABASE_SDE' from 1 datasets. Th
Hello , i got textured CityGML files and converted them to .b3dm format using FME.But when i visualize them on Cesium , its becoming amazingly slow.I dropped quality of textures and gzipped them but still same problem occuring.Here is link for CityGML files: https://www.dropbox.com/s/1dbf9gvmatbvem3/cesme_citygml.rar?dl=0 And for converted 3DTiles https://www.dropbox.com/s/7lh01wo25dqzarf/cesme_3dtiles.rar?dl=0I would be so appreciated if you can solve my problem.
Hi,My question is: how do I do this? I have a ECW file which is clipped using the clipper transformer. The output needs to be stored in a postgres database. It has postgis installed.When I inspect the clipper output I see the right aerial photo in the data inspector. Do I need to use the postgres writer, postgis or postgis raster? I tried all three, and I don't get any error messages when running the workbench. But when I try to read the data back out, I get a whole lot of nothing, and a weird raster of pixel without a photo.Has anyone done this? Does anyone know how to do this. Please let me know what info you need from me (I recently started working where I currently work so I'm still learning the ropes). Thanks!
Hey all,Just wondering if anyone has figured out how to write a brand new google sheet to a folder?I have the following file and folder directory structure:My Drive/My Drive/Statistics/My Drive/Statistics/2017 <---googlesheetI want fme to create the google sheet, '2018' in the "My Drive/Statistics/" folder automatically (without user intervention), but no matter if I include the string "My Drive/Statistics/" or the folder ID & "/" , it always writes the new sheet to the root of My Drive. If the Google Sheet already exists, this is not a problem.
Does FME support migrating Smallworld Dimensions to the ESRI Dimensioning Feature Class?
Is FME already capable of working with a Service Desk / Ticketing type application? Right now I am researching applications that I could use to manage emails & tickets (tasks). An idea (or hope) I have is for a user to email a zip file to a ticketing system, and then FME Server could be triggered to work along side the ticketing application to do some work with the zip file. We currently don't have a ticket management system, and so I am open to suggestions. Some cloud applications that I have seen online are: Zendesk, Freshdesk, or Freshservice.Thank you for your time,
I have 1.6TB of .tif files. Each file (11,500 of them) are named numerically and stored in individual .zip files. I need to convert all of the .tif files to JPEG2000. So far I ran the tif to Jpeg2000 reader/writer through the batch Deploy on small test runs of 10 files and it worked just fine. But when I run the full meal deal FME freezes. Please Advise
I have a FeatureReader for WFS. Because I want it to work with any WFS source I have parameterised the Dataset and "Feature types to read" fields and set the output to Single Output port.This means that I have to use the Generic output port.Further down the workspace I have some transformers that do logging. I have a ListBuilder followed by a ListElementCounter, so I can log the number of records returned by the WFS service. I also have an UpdateDetector with the same arrangement.Before I started using the generic output port, everything was OK. But since I have used the Generic port, some ListElementCounters are showing problems. (The one connected to the WFS reader and also the one connected to the Unchanged port of the UpdateDetector. Others are OK.) The ListAttribute fields are showing up as red and they don't seem to be able to read the _list{} attribute that should be supplied by the ListBuilder.If I run the workspace, it seems to still work, but of course I get a w
I am trying to create a workflow that would update an existing table in a file geodatabase. It works when i have feature operation set to "Insert" and Table Handling set to "Create If needed" under Format Parameters. When I try to change the Table Handling setting to "Truncate Existing", the workflows fails and errors outError:"Geodatabase Error (-2147220987): The user does not have permission to execute the operation.FileGDB Writer: A feature could not be written"
I am searching for a method to take an XML based array and insert each array value into a new record. Currently my workspace parses non-array XML tags\\variable and inserts them into one single SQL record, but now I need a one to many relationship. The number of array items is indefinite. The use case is that we get field inspections in the form of an XML document that gets inserted into our SQL database (various tables - all 1:1). We now want to start registering file\\photo attachments to a reference table, however the photos are stored in an array (not a simple 1:1 insert). Has anyone accomplished this? Idea would be much appreciated. Thanks!
I have a raster in which I need to change the vertical datum. I tried using the CSMapReprojector but it didn't work. It looks like FME just always sees the Raster at 2D when I view it in the Data Inspector. Is this a bug?
Is it possible to get AutoDesk BuzzSaw reader?
I may have encountered an issue with reading Excel sheets with attribute names that start with parts of existing FME format attribute names.My first experience with this was while using the SchemaMapper transformer, but when I investigated the Excel reader, I noticed similar behaviourThe attribute names I wanted to use were:fme_geometry_namefme_type_nameIf a sheet with these attribute names is loaded into the SchemaMapper transformer or the Excel reader, I notice the following behaviour:SchemaMapper Dataset can be loaded and Parameter window works as expected. When clicking Next, a popup with 'Reading Dataset' appears and disappears very quickly. I am not able to finish the SchemaMapper wizard.Excel Reader Dataset can be loaded, but after completing the Add Reader wizard, the FeatureType object on the canvas does not contain the added attributes.When repeating the process with the following attribute names, everything behaves as expected:geometry_name_fmetype_name_fmeMy impress