Using readers and writers to extract and load data.
Recently active
I have multiple jpegs in multiple folders. I first read in using the Directory and File pathnames reader. I didn't want all jpegs to be read so I created a tester to negate some of these. Next I used a string searcher to isolate only the jpgs and not world files or .aux. Next is where my issue arises. I want to now read in the jpgs that I have isolated using a FeatureReader. But within the feature reader parameters for dataset I don't know how to link back to the jpeg files. Any ideas would be greatly appreciated.Thanks
Hi all,i'd like to read a large Smallworld dataset (millions of features) in several tranches in FME Workbench. Maybe in blocks of 500.000 records at a time, since my workspace takes a lot of time to complete cause the complexity of geometries.If i want to use "Start feature" and "Max feature to read" parameters, can i assume the reading order is always respected without skipping any features?Any other technics to read large datasets?Thanks in advance!!!
Is there a way to force a table custom primary key in SpatiaLite db generated from scratch? It seems that FME always add an OGC_FID field and there aren't options to change this behavior. Only if I use a db template I can set my custom primary key.
Hi! I'm trying to duplicate an Excel writer, I get a new writer box (see image with arrow 1) , but not actually a new writer (arrow 2).I've created a somewhat complex writer and creating a new writer from scratch and getting all the parameters in is not a very sensible option because there is room for error. Trying copy-pate or duplicate does not seem to do the trick. The rationale behind what I want to do is to have 2 different XLS files instead of a file with 2 sheets or tabs. Any help is appreciated.
perhaps I am overthinking this, but here is my workflowuser provides a CSV file with two columnstable nameappleorangebananaqueryquery 1 = select count(*) from apple where colour = 'red'query 2 = select count(*) from orange where colour = 'orange'query 3 = select count(*) from banana where source_location <> (select top 1 source_location from orange)read the csvuse feature reader to dynamically read that table from a GDB (table name attribute as input)here is where I am overthinking.here is my desired output tabletable namequeryresult countappleselect count(*) from apple where colour = 'red'2orangeselect count(*) from orange where colour = 'orange'5bananaselect count(*) from banana where source_location <> (select top 1 source_location from orange)1I want to be able to run each row from the CSV table, and return the count result of the query.keep in mind, there are queries that rely on other tables as well. so my thinking is throw all of them
Inputing an excel file and a shp file.Exporting a kml file and a shp file.I am taking an excel spreadsheet and joining it with a shp file to merge the attributes from the excel file into the shp file. I then take only the joined records and filter out the extra records (4 in this case) the joiner created in the database. I then remove attributes from the database that are not needed that originated from the shp file. Then I apply the kmlpropertysetter and styler. The results go to a kml writer and a shp file writer.The kml is set to fanout the data based on a TYPE attribute. (Rock or Cobble). So if there is Rock and Cobble in the dataset it will export two kmls, one for Rock and one for Cobble. Right now it is naming the kmls by the TYPE attribute, this is what it does by default when you select fanout and tell it to base it on the TYPE attribute.I need the resulting kmls to be named by the COUNTY attribute and then the TYPE attribute: for ex. "Williamson Rock.kml" ('Will
I am new to FME, and I am trying to extract data of specific lines of the XML, but once I import the XML, I am unable to find these lines referenced anywhere. Is there a transformer that I could use that I could define the required line (by attribute contained in that line if possible) that would then extract the data so I could consume it
Hi FME'ers, Mark from Safe Software here. I was just wondering if anyone out there is using FME Server on a Hyper-V virtual machine? We'd like to get an idea of how many folk are using that configuration, and if it's working out well for you (it's something we haven't yet used ourselves and we aren't sure if we need to commit more resources to testing it). Either reply to this "question" or you can ping me an email (mark.ireland@safe.com) Cheers Mark Product Evangelist Safe Software Inc.
Hello, Does anyone know if you can read a .hdr and associated .dat file in FME? This is hyperspectral imagery and I think it is an ENVI format. I've tried the envi hdr reader but it fails - is it expecting a .bil file not .dat? Thanks, Owen
There's a workspace which is writing to Netezza. Included GENSTATS query as End sql in Netezza writer, but it gives the below error.ERROR: GENERATE STATISTICS cannot run inside a BEGIN/COMMIT block or stored procedure.Any thoughts or workaround is appreciated.
Hello,For the updating of our topography files I'm trying to cut and merge a lot of .dgn tiles into certain logical areas and output them as dgn.For this task I have the tiles and areas as featureclasses to select where the tiles have to go and clip if needed.I'm running into problems at the point where I have to pass the dgn files to read into the dgn(v8) reader. At first I tried doing the selction in a separate workspace, writing the result to a txt file and reading them again in a new workspace. I could not get this to work with a scripted python parameter even though I was replicating the exact string used for reading multiple files. It was either reading only the first file or none at all.My 2nd attempt was more successful by using the FeatureReader. I merged the selecting and subsequent merging and cutting into a single workspace and it outputs the correct areas. But then I noticed my cells are missing because the FeatureReader is not exposing the IGDS attributes needed
Hi All, I have a highway network polyline and some point events. I have snapped these to the highway centreline. I have been asked to create sections of the highway centreline 1km downstream and 2km upstream of these points. note some of these point events are < 1km apart so the resulting section maybe less thank 1km if the line runs into another eventI have used the pointonline to cut the highway centreline at the points. then i have been trying using the snipper to cut the highway into the 1km section. Here is my work flow. Aggregate highway into 1 entire piece (is 2000 sections) ,Cut this 1 piece highway into two sections using the PointOnLine overlayer at the event point.use snipper to snip the resulting lines to 1km and 2km. issues im having are: how do i feed the aggregated highway back into the loop so each point event gets to cut the aggregated highway not the cut up one.how do i workout which section of the two highway pieces to make 1 or 2 kms (maybe _segment_id)any sugges
Hello all,Quick question. I have a Polygonal Feature (Flood Plain Boudaries) and am using Spatial Relator (its faster than Spatial Filter) and am exporting records to an Access Database.Problem I am having is if one of the features is a line and goes over two different Flood Plain boundaries its only capturing the data from a single polygon. I full expect the records exported to be MORE than the initial data as the line (Gas Main) may cross 3 different flood plain boundaries. I need a record for each one of those added to a record of the original main record.For instance if Gas Main 1234 is a single line and intersects 8 different Flood Plain Ploygons, Id want 8 different records exported to Access.Ive built some complex workbenches but this has me puzzled.Please help!!Thank you,Savas GiarenakisEversource EnergySavas.Giarenakis@Eversource.com
I have a shp of a tile grid and a folder than contains thousands of tif's. Each grid square in the shp has an attribute that contains the name of it's corresponding tif. How can I use this attribute to only select it's corresponding tif in the folder to either:1. copy the tif to a new folder (file copy writer)or2. use as input to the raster mosaickerI can concatenate a string to build up the full path of the tif in the folder. Do I pass this list to what?
I have tried to use xml transformers to update InfoPath forms. Does anyone has successfully use FME to fill out and update InfoPath forms?
Hi,I am trying to use a static map fetcher for OSM. There are a number of these on Github but they all focus on using a center point and zoom level where the output is always provided as an unreferenced PNG. I'm basically looking for a bounding box OSM StaticMapFetcher rather than a center point one. This is proving tricky based on angle and scaling issues.Because I know the center point coordinates and OSM provides the scaling values with the angle offset, it should be relatively straightforward to create the top left coordinate pair and then use the RasterGeoreferencer from there. However I am have trouble with the scaling and there's also some angle warping on the raster which makes the labels illegible. Is there a model that can be used within FME to fix scaling issues like these for OSM? I am guessing this is something to do with distance from the equator as Ireland is ~52degs where this sample is based and this would differ with examples closer to the equator, and so on.
Hi,How can I know what is the object type for an Ifc class instance.For example :Is this instance of IfcFlowTerminal is type of IfcOutletType or IfcAirTerminalType?Thank you
Hello,I am experiencing the problem during exporting to cesium 3d tiles format. On input I have OBJ meshed model, which I want to tile and then store as a Cesium 3D Tile b3dm.After my conversion I have tileset.json and only 1 data0.b3dm file. Cesium does not load it.
I think no but just to be sure: Is it possible to use a select statement on a geodatabase using the File Geodb Open API reader? I know the tooltip, just double checking with the community before going another route...
Hi,I have a CityGML file where a building has an outerBuildingInstallation. When I read the file with FME Reader the outerBuildingInstallation does not get the gml_parent_id but a generic one from the FME Reader. Without having the gml_parent_id (that is the ID of the building) I cannot merge the building with the outerBuilingInstallation.Why is that? Do I make mistake? I cant figure out why the gml_parent_id has got a value I cant use.Here is part of the CityGML file:<cityObjectMember> <bldg:Building gml:id="_22631"> <gen:stringAttribute name="citygrid_UnitID"> <gen:value>22631</gen:value> </gen:stringAttribute> <bldg:outerBuildingInstallation> <bldg:BuildingInstallation> <bldg:boundedBy> <bldg:RoofSurface> <gen:doubleAttribute name="Flaeche"> <gen:value>9.403696</gen:value> </gen:doubleAttribute> <gen:doubleAttribute name="Neigung"> <gen
Is it possible to write to MongoDB as an array datatype, either from an FME list or otherwise? Thanks!
I need to convert a large set of ESRI personal gdb (mdb) to file gdb. I can transform all feature dataset except some. I attach a sample with a feature dataset of this kind. My source personal gdb is data.mdb and DATA.XML is a workspace template exported from ArcMap 10.2 Desktop. I get and ArcObjects error number -2147216894.Is there a way to rewrite the grid size definition used in the template file? It seems that ArcMap always includes a gridsize tag in XML.Thanks in advance.
I know that there is currently a Bentley ProjectWise reader/writer in the works. Is there any plan to create a BlueCielo Meridian reader/writer as well? May be a difficult task, as many instances of Meridian are built on a proprietary object orient database called Hypertrieve. Maybe some clever user can come up with some custom transformers to access the Meridian api...