Using readers and writers to extract and load data.
Recently active
Hi,I am working on a workflow that can do some simple calculations on different data sets but within just one database of my choosing. Each project has its own database with tables(data sets) and i would like to connect to them by just giving FME the pXXX project number and data set number. I tried to use published parameters to set the same values that i normally use in a more static workflow. This ends up in an error when adding the reader.Undefined macro `database' dereferenced in file `'Program TerminatingTranslation FAILED.Workspace generation failed. Parameter file was C:\Users\Jeroen.GPA\AppData\Local\Temp\FME_1510569791750_7796.Is it not possible to set the database into a parameter or do i need to save it as something else than text? Or is there another way to get dynamic connections to different databases on the same server(login credentials are the same for all tables and databases on that host and port).edit: i, of course, filled in the host, port and other information tha
I can get my Feature Types, but when I connect them to an Inspector, I can't read more than 150000 entities. I get this message : Is this an error because of my parameter ou because of the WFS Data ? I'm using FME Desktop 2017 (build 17652, 32-bits and 64-bits version). My project use only about 300 Mo RAM. I keep the WFS Reader's default parameter, except : Max Feature : 150000 WFS Version : 1.1.0 WFS Output Format : GML 3.1.1 Connection & Transfert Timeout Length : 1000 s If I set "Max Features" to 100000, I can visualise my datas. Thanks for any help or advice ^_^ Patrick
I am trying to manually geocode some addresses, and am having a hard time developing the correct workflow to acheive this. I have a dataset coming from a SQL database that is full of addresses with unknown coordinates. I am geocoding with a CSV file that I have created that contains the addresses compiled from county auditor websites with the associated latitude and longitude fields for each address. I am starting with FeatureMerger because if both address fields are an exact match then this works perfect. -- address geocoded successfully. However some addresses will not match based on different suffix types. (i.e. ROAD vs. RD or HIGHWAY vs HWY) I have added a string replacer transformer to the workflow to trim off the suffixes that could eliminate the addresses from matching via a rubular expression. The part I am struggling with is how do I compare these fields again after I have trimmed off the various suffixes? I have attached a picture of my workflow.
Hello everyone,I've just started my PhD and referencing takes me forever so thought FME could do the heavy lifting. The idea is to have an excel chart (example attached) that I manually fill in with the referencing details and run it through a workspace and at the end I have a full bibliography written to a word document. The process for this is to take the excel file and sort it alphabetically based on authors surname and then filter the attributes based on the type of reference and number of authors. From there I use the MSWordParagrapher to list the Harvard reference format that I wrote in the text editor The problem I have run into is that for the life of me I cannot get more than two MSWordParagraphers working at once. If I link up attributes to a third one I get a failed workspace and I will ideally want around 20. Is this possible? Am I missing something simple? Or is there a better way of doing it?, I've attached the workspace and the source datasets if anyone can help
Is there a way of reading .pts files within FME
Hello everyone,I am back and I have 99% of this very important workbench completed (thanks to you all for helping with last question).My next "issue" is stumping me. My workbench is reading form SDE and writing to a GDB.One of the features I have is a Cutout object and within the SDE is the Cutout_tb (Sde table object) which carries the Child records of the Parent Cutout.I simply bring it in and connect it to feature merger to get the attribute I want (fuse size).Seems simple enough but the Workbench wont read records from the table in the workbench whether I put it in its own reader or not. Its puzzling because if I make separate workbench and simply write to access database it reads and writes the records fine.Is there any way to get the workbench to get the child records straight from SDE upon reading the cutout object (such as reading the SDE Relationship Class object in SDE)? Or is there a way to get the simpler way of simply having the table read in workbench and join
Hi,When we try to create a new feature class using the Esri Geodatabase (ArcSDE Geodb) writer, we get the following error: Geodatabase Writer: Creating feature class `OBJ.Duikers_2017'An error occurred while attempting to create the feature class 'OBJ.Duikers_2017'. The error number from ArcObjects is: '-2147216072'. The error message from ArcObjects is: {Underlying DBMS error [ORA-13224: geen tolerantie opgegeven voor laag in USER_SDO_GEOM_METADATAORA-06512: in "MDSYS.MD", regel 1723 ORA-06512: in "MDSYS.MDERR", regel 17ORA-06512: in "MDSYS.SDO_GEOM_TRIG_INS1", regel 39 ORA-04088: Fout bij uitvoering van trigger 'MDSYS.SDO_GEOM_TRIG_INS1'.]}A fatal error has occurred. Check the logfile above for detailsI think the correct translation for the ORA-13224 message is:ORA-13224: zero tolerance specified for layer in USER_SDO_GEOM_METADATAIn you can find the sql statements that are executed by the trigger MDSYS.SDO_GEOM_TRIG_INS1 that raises the error.The problem seems to be that no toleran
New to FME: need to convert excel csv x,y meter coordinates into dwg file format
I have a JSON file and need to extract the attributes from this file. How can I do it? Example:{ "id" : null, "revisionNumber" : "1.0.0", "periodTimeInterval" : { "start" : "2014-10-10T01:00Z", "end" : "2014-10-11T01:00Z" }, "timeSeries" : [ { "id" : "silo:67688:536653:raw", "name" : null, "tskeys" : null, "dimensions" : { "attribute" : [ { "value" : { "item" : [], "type" : "String", "value" : null }, "name" : "Datenquelle" }, { "value" : { "item" : [], "type" : "Integer", "value" : "0" }, "name" : "Netzbetriebsplanung" }, { "value" : { "item" : [ { "key" : "0", "name" :
A fellow FME user asked me if there are many people using Oracle Cloud and also they were wondering what formats are commonly used in combination with Oracle. I figure you all might be able to help answer this one.
Hi,I need to vallidate SHP (or DWG blocks with attributes) attribute value against Field Data Type and Domains specified in template GDB or XML Workspace Document. Reader: ESRI SHP Writer: ESRI GDB ISSUE: If the SHP attribute value doesn't match with GDB Field Data Type, the translation fails. I need to validate those values and write the fails.I wanted to use 'AttributeValidator' but the Validation Rules must be specified manually. Is there any way how to read the Validation Rules from the template (GDB or XML Workspace Document) and write them automatically?EXAMPLE:Reader SHP feature 'PIPE' has got field 'MATERIAL' with attribute value 'UPVC'. Writer GDB feature 'PIPE' has got field 'MATERIAL' with specified domain values: 'PVC-U, HDPE, PE' There is no match so the translation failsThanks.
Hi all,I'm very new to FME and GIS so sorry if I get any terminology wrong!To give you a run down, I've created a TIN surface from an ESRIASCIIGRID file. I have then extracted the boundary of this to get a footprint in a .shp file.With this, I am trying to download a satellite image from either google earth or google/bing maps (to drape over the surface later).Attempt 1:Import the .shp file into google earth, I then attempted to export an image from the extents of the polygon from the .shp file. I can't figure out a way to do this so I've just saved an image from google earth manually with 4K resolution, showing the extents of the polygon in red.Is there a way for FME to detect a red polygon and to clip the image around this? Would it then be possible to add geo-spatial data to this image to match the previously created surface?I have read through this post but am unable to get it to work, but thankyou @takashi as it's helping me to better understand FME.Attempt 2:U
I'm trying to read a feature layer and write to a geodatabase. I'm able to connect to my feature layers but I get the following error when running the processArcGIS Online Feature Service Reader: Encountered an unexpected error. The error code from the server was '400' and the message was: 'Cannot perform query. Invalid query parameters.'. Details: 'Unable to perform query. Please check your parameters.'
See this question, I suspect it may be a bug and would appreciate it if somebody from support could take a look at it.
For some reason when you change the "name" of an xref in a workbench the writer doesn't replace the old xref, it just adds a new one. Purging in the writer doesn't work. Does anyone know how to get rid of defunct xrefs?
HiI have a few thousand shape-files arranged in pairs of 5 in the form of a1.shp, a2.shp, a3.shp...... and b1.shp, b2.shp..... and so on. I would like to batch merge these into a.shp, b.shp and so on.Some guidance on how to create this batch would be much appreciated.Best regards, robin123
We are using FME to read DWG and write OBJ. OBJ only supports flat surfaces, so objects such as cylinders in the DWG are converted to flat surfaces in OBJ. Which DWG reader parameter I should look for to increase the number of flat surfaces written to the OBJ? I have an existing fme workbench made by someone else, and I'm new to fme. Thanks much for any quick help from the community!
We are attempting to standup FME server in an automated fashion using Docker to spin up Linux containers. Because of the linux containers, our workspaces can only run SQL Server JDBC reader/writers per FME. Plus, in order to spin up the containers quickly, we avoid using any ESRI dependencies. For one of the workspaces, we are using Spatial readers and Non-Spatial writers because we are just updating attributes based on the processing results. The JDBC writers work but are twice as slow as the native SQL Server writers. Do we have any other options for updating the attributes in these SQL Server tables that may give us a performance boost since we don't actually have to write any data?
Hi I have a CSV file with X,Y coordinates. I am loading this to a point feature class. However the XY coordinates sometimes come in quotes and have a comma, e.g.GLOUCESTERSHIRE,ST BRAIVELS COMMON,GL15 6SD,"355,280","203,644"Any suggestions what's the best way to handle this when using the CSV reader.
INSERT INTO `stations`.`observation` (`station_id`, `date`, `time`, `energy`, `power`, `average`, `temperature`, `voltage` ) VALUES ( 1, @Value(interval_start), @Value(_timeOutSubstring), @Value(_dataEnergyOutString), @Value(_dataPowerOutString), @Value(_dataPowerAvgString), @Value(_dataTempString), @Value(_dataVoltString) )
Hey, I'm getting a unique constraint error coming from an indexThe error number from ArcObjects is: '-2147216072'. The error message from ArcObjects is: {Underlying DBMS error [ORA-00001: unique constraint (FACILITY.A17846_IX1) violated]}Is there a switch in the writer I can set?? I have tried GRID 1 = Nothing and GRID 1 = 0 and both fail.Thx Brian
Hi, I'm trying to read data from this server:https://geodata.nationaalgeoregister.nl/bag/wfs?request=GetCapabilitiesbut only a 1000 features (the same ones it seems) come through. I've tried switching some parameters around, like what was suggested in previous questiosn about WFS on this forum (ignoring the schema etc. ). However, nothing has worked so far.
Hello,FME 2017.1.0.0 64-bit, FME 2017.1.1.1 64-bit / Windows 7, Windows 10I encountered this error when adding the Esri ArcGIS Online (AGOL) Feature Service Reader to the workspace.Logged Message:ArcGIS Online Feature Service Reader: Encountered an unexpected error. The error code from the server was '500' and the message was: 'Unable to get user.'. Details: 'Unable to get user.'The Feature Service list has been retrieved correctly, but I cannot get the Layer List from any service.The AGOL account has been provided by a client in order to test collaboration between their AGOL site and FME Server. I can access their AGOL site with the account using a web browser. The error occurs only when accessing the AGOL feature service via FME AGOL reader. (I also have my own AGOL account, and the reader can access the feature service in my AGOL site with no problem)Question: How can I resolve the error and read the layers from the feature services?Thanks.
Hello,I have a shp file with ZIP code level polygons. I need to combine all of them to give me polygon for the state, is it possible to do in FME?I tried to use the Aggregator, which took all my polygons and showed 1 output, but when inspecting the file, there was no geometry.Thank you!