Using readers and writers to extract and load data.
Recently active
I have a large, relatively complex workspace that performs a few tasks - loads data, validates bearings, distances, areas, creates subsets of the data and then creates a number of reports, PDF, HTML and XML)The PostGIS database is located externally on a separate hosted server.On my laptop the workspace takes less than 2 minutes to complete. My laptop is a pretty stock standard HP, Windows 64 Bit with 8GB RAM.I have published to FME Server Cloud and had a starter configuration established with 2 virtual cores and 4GB RAM. This configuration caused the server to melt down and the jobs just kept failing with no specific errors. I upgraded to the standard configuration with 2 virtual cores and 8GB RAM. The workspace failed after about 30 minutes and what appeared to be multiple cycles of attempting to run but still no errors in the log.Below is the memory monitoring. The job was started at 8:06, memory peaked a couple of times and then finally failed at 8:36What I am seeking is some advic
I know about the feature service in FME to create and update Feature services in ArcGIS Online, but I want to create (copy to, host), for example a Word file, in ArcGIS Online. https://doc.arcgis.com/en/arcgis-online/reference/supported-items.htm
I have a number of tables in Oracle Spatial that need to be synced from Smallworld. The tables contain multiple geometry columns. We have a requirement for a nightly job that updates the ~80 tables, and are passing in only records that have been updated over the course of the day. In some cases there may be null geometries for some of the features (ex. in one of the two geometry columns), depending on the data changes. When using Feature Operation: Update and Table Handling: Use Existing, the translation fails with the error that one of the geometries cannot be found in the current feature: "Spatial Column 'FIBER_OPTIC_LOCATION' of type 'geometry' cannot be found in current feature."Is this the expected behaviour for performing updates to Oracle Spatial DBs with multiple geometries? Is there a way to get around this to update records that may contain multiple geometries?When inserting records and truncating the existing table, the translation completes succes
As far as I can see, SharePoint writer currently only supports Custom / Generic Lists. Are there any plans for adding the possibility to write a SharePoint calendar? Thanks in advance!
Hello all,I am seeking help to configure the JSON reader, which I am not familiar with.The JSON file is not a GeoJSON, (despite containing coordinates). Only the JSON (JavaScript Object Notation) Reader recognises it.There are two arrays containing objects. custom_markers and markers "custom_markers": [ { "rating": 3, "gallons": 0, "marker_type": "Other", "mileage": 0, "longitude": "152.690125", "latitude": "-27.976959", "price": 0, "date": "1563341193.242588", "pin_color_blue": 1, "pin_color_red": 0, "marker_description": "", "pin_color_green": 0.5019607843137255, "name": "Stop 1 " },there are about 17 custom markers and these should be rendered as point objects "markers": [ { "longitude":
I have a workspace (2018.1) that read a ArcSDE feature class into a Geojson file and then read the Geojson into MongoDB. The Geojson writer works fine in writing, but always gives an error at the end of writing regardless of the geometry type or the size of the feature class:-------------------------------------------------------------------------------------------------------Opening the GEOJSON reader with source dataset 'W:\\gis_data_admin\\Admin\\AllenG\\FME\\MongoDB\\Geojson\\IHS_INT.json'A JSON syntax error was found at line 1, column 0The JSON data is incomplete: Unexpectedly encountered the end of JSON dataA JSON syntax error was found at line 1, column 0The data does not contain any JSON text-------------------------------------------------------------------------------------------------------If I change the reader to shape files, then I don't have the above problem. So the problem seems to be related to ArcSDE feature class. On the other hand, the Geojson reader
Hello there.I'm having trouble with an Excel writer.Where the value being read in FME is completely numeric, the data value is output with an additional '.0' suffix. I've tried changing attribute data types to auto or string, and I tried choosing 'Treat as Text @' from the attribute formatting/'Edit...' button in the writer - no luck.Example...I've got the following values being written to an attribute in an Excel writer...TEST VALUE HERE 12341234567ABC1234And it is outputting the following values to Excel...TEST VALUE HERE 12341234567.0ABC1234I don't want the ".0" attached to the end of values that are fully numeric, rather I want all of the values in the attribute to be treated as a string.Perhaps this is just an issue with Excel itself. When I look at the cells with these particular values, Excel gives me a warning box that shows 'Value stored as text'; when I choose 'Convert to number' the formatting appears as expec
Hello, I just moved to fme 2019 and in this example I'm using workbench FME(R) 2019.0.1.0 (20190506 - Build 19253 - WIN64) I have very simple workspace: creator -> featurereader (postgis) -> gml writer When I run it I don't even get to the breakpoint. On reader there is log: 2019-07-18 21:46:06| 7.6| 7.1|FATAL |POSTGIS reader: An error occurred. FME will attempt to provide more information on the error, but this may cause the translation to be terminated2019-07-18 21:46:06| 7.6| 0.0|INFORM|Successfully closed POSTGIS database reader2019-07-18 21:46:06| 7.7| 0.1|INFORM|Successfully closed POSTGIS database reader2019-07-18 21:46:06| 7.7| 0.0|FATAL |POSTGIS reader: stk::ex::bad_expected_access I discovered that problem is with one row. In column of type:timestamp with time zone value is:-infinityIn version fme 2016 I don't have that problem. I executed that workspace with fme 2016 and 2019 and here I can see the difference in fme postgis reader version: I did
I'm experiencing an issue writing into file geodatabases using the FILEGDB writer. The issue occurs when eitherThe geodatabase has been created by ArcGIS Pro, orAn existing geodatabase has had a feature class pasted using Pro, thenTrying to create, or drop and create, a feature classAs with many things the problem is not always consistently apparent, but occurs much more frequently than not.The issue shows itself by either not being able to drop and create an existing feature class, or by not being able to write a new one. Instead, a non spatial table is created with the schema, but no data. Deleting this sometimes seems to leave behind ghost feature classes that appear if the geodatabase is repaired, or if a feature class is pasted in that has the same name.Using the GEODATABASE_FILE writer appears to be fine (but not an option for us when publishing to FME Server), and truncating and inserting into feature classes also appears to be fine (with the FILEGDB writer).I'm using de
When I post a new question on this forum, the attachment button does not work. Clicking on the button does nothing. When I hover over it, my cursor becomes a hand symbol.I would like it to work, as sometimes I need to attach a file to clarify my question.
Greetings.I am trying to export a Sqllite (geopackage) to SQL Server.Keep getting an error despite changing the options.Reader: Writer: Error:Transactions are being used by the Geodatabase WriterAn error occurred while attempting to open the table, feature class or relationship class 'main.airports'. The error number from ArcObjects is: '-2147467263'. The error message from ArcObjects is: {}An error occurred while attempting to open the table, feature class or relationship class 'main.airports'. The error number from ArcObjects is: '-2147467263'. The error message from ArcObjects is: {}A fatal error has occurred. Check the logfile above for detailsA fatal error has occurred. Check the logfile above for detailsStored 1 feature(s) to FME feature store file `C:\\Users\\User\\AppData\\Local\\Temp\\wb-cache-geopackage2mssql_spatial-NJVZFF\\Main_airports -1 10 -1 0 89fdbd78da335154a0ddf39d89823152eee47e98.ffsupdating' Any suggestions?? Thanks,Clive
I'm trying to take an RGB raster and convert it to Gray+Alpha. I do the following:Use RasterExpressionEvaluator to calculate linear luminance from the three RGB bands into a single band with interpretation Gray8Use RasterBandAdder to add a band with interpretation Alpha8Using RasterBandPropertyExtractor and Inspector in the workspace, I see that the result have two bands with the correct interpretations. However, once the result is written using the ECW writer, all GIS tools agree that the two bands in the file does not have an interpretation (I have checked the file with QGIS and FME Data Inspector and the bands come up as "Undefined" or "UInt8).I tried swapping the ECW writer for a GeoTIFF writer and in this case the file came out with correct band interpretations. Is this a bug in the ECW writer or is there another way to set the interpretation?
But i have Always the same message :Polygon feature must have at least 4 coordinates to be valid for the shape file. The feature will be skippedREJECTING BELOW FEATURE:And i dont understand what is really the problemCan you help me pleasethanks
The MS Access (JDBC) Reader gives an error reading table schema's in FME 2019. The workspace still works fine in FME 2018.1, but generates the following log in version 2019.0: MS Access (JDBC) Reader: Connected to 'UCanAccess driver for Microsoft Access databases using HSQLDB' version 'V2007 [VERSION_12]' using driver 'Ucanaccess' version '4.0.4' built against JDBC version '4.0'MS Access (JDBC) Reader: Error reading schema for table 'qry02_Energiesnoeiers'. Error: <null>java.lang.NullPointerException at COM.safe.fme.jdbc.msaccess.MsAccessBackendHelper.createColumnMetadata(MsAccessBackendHelper.java:343) at COM.safe.fme.jdbc.JdbcBackendHelper.getTableSchema(JdbcBackendHelper.java:129) at COM.safe.fme.jdbc.JdbcReader.readSchema(JdbcReader.java:948) at COM.safe.fme.jdbc.JdbcReader.readSchemaHelper(JdbcReader.java:872) at COM.safe.fme.jdbc.JdbcReader.readSchema(JdbcReader.java:830)java.lang.NullPointerExceptionFailed to obtain any schemas from reader 'MSACCESS_JDBC' f
We are trying to upgrade our SQL Server to 2016, does the latest version of FME Workspace compatible with SQL Server 2016?
I am suddenly receiving a pipe separated text file with imbedded pseudo-unicode characters that are supposed to be a macronated 'o'. Unfortunately it translates to a ^Z in the ASCII file.All the vowels can have macrons in Maori and there is a new policy that all government departments must add macrons. If all software was unicode aware then this might work.Some programs will handle this, reading the whole file regardless of the ^Z but many stop. The FME CSV2 reader stops. Oddly the FME Textfile reader does handle them with encoding set to DOS-Latin-1 (ibm-850)What can I do?The simplest idea is to translate the pair of characters ^Zo back to a plain ascii o.Surely tr could just strip the ^Z...nope.I have tried to use utf-8 encoding parameter on the CSV reader and other tricks with tr without success.I have attached a test sample.asc of two records.Unix wc -l returns a count of 1, not a good start since I can cat two records.
Hi I have a list of grid tiles on Esri shape format. Those tiles are not clustered, tiles can be anywhere in the country. I need to extract geometry within those Tiles, from an Oracle spatial DatabaseHow do I go by without reading all the features on the entire Database?thanks
Hi all,I have a set of 8000+ GML files that i wish to batch process into a geodatabase, i have found that putting them a workspace runner is effective but slows down massively past 50 GML files. Therefore, i want a way to write a new geodatabase file every 50 features, to reduce the 8000 GML files to 160 geodatabases to then combine afterwards.Has anyone any ideas? Apart from a huge conditional value in attribute manager!CheersJack
I am having some problems writing extended attributes to a dwg, I have attached the source dwg and my workflow.The source dwg contains a polyline that has several fields of extended data, my aim is to be able to change a few of the fields with a string replacer, for example in this case change “JSY” to “CWG”.I am able to inspect the extended data list in the reader and change the values, however when it comes to writing to dwg the attributes don’t come through. Running the command XDLIST, shows no extended entity data on the entity.Any help would be appreciated.
Hi there - I'm trying to make schema changes to an SDE feature class as well as an SDE table.... both have attachments and the obligatory relationship class. When I try and output to the writer using "Drop and create", the attachments (and relationship class) are deleted. But when I output using another option (e.g. truncate), they are not deleted. I'd like to use "drop and create" because we need to reorder fields and add new ones. Wondering if I'm doing something wrong, or not understanding the process, or not using a much better alternative. Or possibly all 3!
I have multiple FME workbenches that read from a single SQL Server Spatial Database. This is a global database so I often have more than one workbench trying to access the database. However, it seems like the first workbench is locking the table preventing others from reading from it. This seems more likely to be an issue if the SQL table has triggers enabled. Is there away to get around this problem?
Hello everybody, I need to create a kml file with 3D geometry in .dae format (SketchUp). The links are in the table of my source file with point geometry. I need to obtain this: Is there anybody who knows how to do?Thank you!
HiI juste use an AttributeManager after a reader which works fine. My value is changed.but the writer doesn't update my record in the table.I tried all options in the writer to match the row selection ... nothing works.Do you have an idea about this issue ?Thanks
Hello, I would like to write some data from my statisticscalculator to a sharepoint list. If the list with the autocad input name does not exist, it should create a new list with the same name as the autocad input. After Running my current workspace it tells me the following:Cannot find a matching schema named 'C:\Cloud\Sync\_Projecten\FME\Inwerken Progress Checker/Input\Fme Generated\Karteringen&Areas.dwg' in the known set of schemas: ''. Known schema sources are: 'SHAREPOINT_2'. Please verify writer feature type configuration.Python Exception <KeyError>: 'C:\\Cloud\\Sync\\_Projecten\\FME\\Inwerken Progress Checker/Input\\Fme Generated\\Karteringen&Areas.dwg'A fatal error has occurred. Check the logfile above for detailsA fatal error has occurred. Check the logfile above for detailsA fatal error has occurred. Check the logfile above for details I am confused as to why it it wont create a new one. My writer has been set up like this: my Current workspace looks like this T
I have a input file geodatabase that just has points. The workbench first uses the file name extractor transformer to get info on the input file. I use the results from the file name extractor to build out the output file path because I want it to be dynamic. For ExampleInput: C:/Temp/Input.gdbOutput: C:/Temp/Output.gdb (Dynamically created based on the file dir of the Input, name would also be generated)On the output file geodatabase destination fanout directory I put in the user param "OutputDir" that is blank to start but filled out by an attributeCreator that overwrites the value after the file name extractor transformer runs, same for the output filename but it does not seem to work dynamically and the value must be hardcored in the writer to work?