Using readers and writers to extract and load data.
Recently active
I am reading several views that contain non spatial data. When inserting the readers, I click the approx 10 views I want to read from and then am bringing them in as a single merged feature type. I am bringing them in as a single merged feature for simplicity and because I am using a trimmed version of fme_feature_type to name my shapefile writers. My workbench fails when I run it, because it is trying to read tables that I do not have access to. Has anyone else had this problem? I have other workbenches that are similar to this and it works for them. I know I can just bring in the tables as individual feature types, it just seemed odd to me. I'm using FME 17.0.1.1
Trying to read huge shapefile and write to SDE.Currently I'm running in local and taking longer time. Also tried to push it to server, it's longer in server too. Looking for any faster approach to load data.Any replies are appreciated.
We are trying to read SDE files in FME Server. But it displayed the follow error message: Could not open the Enterprise Geodatabase. The error number from ArcObjects is: '-2147216028'. The error message from ArcObjects is: {No SDEHOME environment variable set}. We installed FME Server 2017 64 bit, Oracle client 64 bit and ArcGIS 64bit Background. Who have same experience and help us find the solution? Thank you!
Hi,It seems to me that there's a bug in the ArcGIS Online (AGOL) Writer when one attempts to write features to an existing layer. I'm using FME 2017.1 (x64).The writer has Mode = INSERT and Truncate First = Yes.If I delete the layer in AGOL prior to running the workspace, it works. If I don't, I get this Python error:ArcGIS Online Feature Service Writer: Writing features for layer/table 'KommunerMedOverlap', with 57 inserts, 0 updates, and 0 deletesPython Exception <UnicodeDecodeError>: 'ascii' codec can't decode byte 0xf8 in position 2: ordinal not in range(128)Traceback (most recent call last):File "fmeutil.py", line 653, in wrapperFile "arcgisonlinefeaturesWriter.py", line 152, in closeFile "fmeutil.py", line 653, in wrapperFile "arcgisonlinefeaturesWriterPaths.py", line 785, in closeFile "fmeutil.py", line 653, in wrapperFile "arcgisonlinefeaturesWriterPaths.py", line 885, in _applyEditsOnLayerFile "C:\Program Files\FME 2017\fmepython27\lib\json\__init__.py", line 250, in dum
I have a .flt file that a client wants to use in Microsurvey. I actually have no background in iCAD files, but I thought I'd do what I can to help him if possible. The .flt came from USGS' TNM. I'm still very much a noob at FME, but it seems to me like it should either already be compatible, or something readily converted. If it needs conversion, I'd appreciate someone walking me through the steps. I can provide more info if I've left anything out.
Hi, I have a design file with all cells as graphic cell. I want to convert them in to point shape file as point. but after converting them to shape it comes in polygon. Is there any way to convert these graphic cell to point shape file directly with converting them to point cell. Does any one have microstation macros to convert graphic cell to point cell or any other procedure. thanks
I have a geometrycollection type KMZ. And I want to import it to ArcGIS. Could anyone show me how to convert it without losing attribute? Thank you
Hi, Im a newbie, have trouble importing my file from http://media.pcna.com/Integration/PCNA-Media.zip. I unzip it and reference the xml file. My goal is to export the xml to a csv file, but can't get it to read. Can someone help?
Hello everyone,I have a CityGML dataset from Montreal Open data that seems to be in EPSG:2950 I need to write this CityGML to a Cesium 3D-Tiles dataset (tileset.json + tiles) in EPSG:4978I tried Reprojector and CS-MAP Reprojector but in vain.I get an error when I try to do so :
I try to make a simple transformation but couldn’t reach my goal so I thought someone might be able to help me ? Here is the origin of my data : actually I read a dgn file which has only one layer called « Default » with more than 8.000 elements. These elements are cells. When I read this dgn file with FME , it splits the cells into lines but I don't want this. I want to get one point for each cell. So I use the transformers on cells in this order : 'labelpointreplacer' then 'coordinate extractor' then 'vertex creator' and finally 'sampler' (group by "igds_cell_sequence_number") to do this job and it works. In the end I have as many points as there are cells. But now I want to import these points into a PostGIS database.So I want to use : The internal parameter ‘igds_linkage{0}.string’ indicates wich layer each element should belong. The parameter ‘fme_feature_type’ is set do ‘Default’. I use the transformers ‘attrib
Hi,Could you provide me an example of Excel reader and writer library (https://docs.safe.com/fme/html/FME_Desktop_Documentation/FME_ReadersWriters/xlsx/xlsx.htm)?Is the library MS Excel version dependent (2011, 2013 etc.)? The process involves reading data from some of the geometrical shapes. Is this feature supported? Thank you !PS: C++ is preferred.
Hi ! I'm trying to create a database connection to a cosmosdb database using the mongodb api. This implies using a mongodb connection with ssl parameter. I can't find the ssl parameter in the mongo connection box in both FME 2016 and FME 2017, Is it not implemented yet ? Is there a way to connect providing the mongodb connection uri with any parameter ?
I have an Oracle SDE table created from MS Integration Service. The table has columns with mixed upper and lower cases. Look like FME data write into this table only looks for upper cased column names and I got this ORA-00904: invalid identifier error for all columns. We won't want to change the column names to be all upper case. Is there a workaround for this?
We are seeing an increasing number of clients wishing to manage log files using a central log collection and monitoring system. By default, FME Server stores all log files in the system share directory structure but this also contains repository information, etc.I would like to move just the log files (not the whole system share) to another location. Has anyone done this? It looks like it should be possible by editing a few settings in the fmeServerConfig.txt file but it's probably not recommended...Bonus question: how nicely would these changes play with a fault tolerant (Active-Passive) architecture?
Hi,I'm using FME to convert a CityGML file into B3DM for visualization in CesiumJS. However, when I visualize the B3DM file, it seems to have an endlessly recursive structure. Based on the B3DM documentation, my understanding is that each tile should have a feature table (with the list of features), a batch table (with tile metadata), and the binary glTF of the tile. However, when printing out the tile contents in the console, this is what I see. Has anyone experienced this before? Or have any idea how to make sense of it? My end goal is to be able to select features based on their geographical location. Any help would be appreciated.Thanks!Aditya
I asked this question on this tutorial page:https://knowledge.safe.com/content/kbentry/1533/index.html#comment-54869and was advised to post a question on the forum. Unforunately, I can't provide a datasample. I just wanted to know... Say, I have a DGN file, which was created with one of Bentley's software packages. And I wanted to transhorm that DGN file to a shapefile just like in the tutorial linked above, but also linked to that shapefile the same attribute table which was related to it in the Bentley software. I've noticed a "Linkage extraction " through MSLinks and Framme, in the reader parameter. What would happen if I joined an attribute table with a MSLink column to my shapefile geometry (a table origially linked to the DGN file in the software package), when I check MSLinks linkage extraction? Would FME automatically detect which MSlink in the attribute table relates to which feature in the shapefile?EDIT 1:If you check MSLINKS in the linkage extraction,
I have multiple jpegs in multiple folders. I first read in using the Directory and File pathnames reader. I didn't want all jpegs to be read so I created a tester to negate some of these. Next I used a string searcher to isolate only the jpgs and not world files or .aux. Next is where my issue arises. I want to now read in the jpgs that I have isolated using a FeatureReader. But within the feature reader parameters for dataset I don't know how to link back to the jpeg files. Any ideas would be greatly appreciated.Thanks
Hi all,i'd like to read a large Smallworld dataset (millions of features) in several tranches in FME Workbench. Maybe in blocks of 500.000 records at a time, since my workspace takes a lot of time to complete cause the complexity of geometries.If i want to use "Start feature" and "Max feature to read" parameters, can i assume the reading order is always respected without skipping any features?Any other technics to read large datasets?Thanks in advance!!!
Is there a way to force a table custom primary key in SpatiaLite db generated from scratch? It seems that FME always add an OGC_FID field and there aren't options to change this behavior. Only if I use a db template I can set my custom primary key.
Hi! I'm trying to duplicate an Excel writer, I get a new writer box (see image with arrow 1) , but not actually a new writer (arrow 2).I've created a somewhat complex writer and creating a new writer from scratch and getting all the parameters in is not a very sensible option because there is room for error. Trying copy-pate or duplicate does not seem to do the trick. The rationale behind what I want to do is to have 2 different XLS files instead of a file with 2 sheets or tabs. Any help is appreciated.
perhaps I am overthinking this, but here is my workflowuser provides a CSV file with two columnstable nameappleorangebananaqueryquery 1 = select count(*) from apple where colour = 'red'query 2 = select count(*) from orange where colour = 'orange'query 3 = select count(*) from banana where source_location <> (select top 1 source_location from orange)read the csvuse feature reader to dynamically read that table from a GDB (table name attribute as input)here is where I am overthinking.here is my desired output tabletable namequeryresult countappleselect count(*) from apple where colour = 'red'2orangeselect count(*) from orange where colour = 'orange'5bananaselect count(*) from banana where source_location <> (select top 1 source_location from orange)1I want to be able to run each row from the CSV table, and return the count result of the query.keep in mind, there are queries that rely on other tables as well. so my thinking is throw all of them
Inputing an excel file and a shp file.Exporting a kml file and a shp file.I am taking an excel spreadsheet and joining it with a shp file to merge the attributes from the excel file into the shp file. I then take only the joined records and filter out the extra records (4 in this case) the joiner created in the database. I then remove attributes from the database that are not needed that originated from the shp file. Then I apply the kmlpropertysetter and styler. The results go to a kml writer and a shp file writer.The kml is set to fanout the data based on a TYPE attribute. (Rock or Cobble). So if there is Rock and Cobble in the dataset it will export two kmls, one for Rock and one for Cobble. Right now it is naming the kmls by the TYPE attribute, this is what it does by default when you select fanout and tell it to base it on the TYPE attribute.I need the resulting kmls to be named by the COUNTY attribute and then the TYPE attribute: for ex. "Williamson Rock.kml" ('Will
I am new to FME, and I am trying to extract data of specific lines of the XML, but once I import the XML, I am unable to find these lines referenced anywhere. Is there a transformer that I could use that I could define the required line (by attribute contained in that line if possible) that would then extract the data so I could consume it