Skip to main content

- - Community Resources
Ideas
Product Updates
Events
Blog

EN

Home
Forums
FME Form
Data

Data

Using readers and writers to extract and load data.

6,701 Posts
24,706 Replies

When you subscribe we will email you when there is a new topic in this category

6701 Posts

Recently active

Most replies Most views Newest first

Directory and File Pathnames read gdb

Hi, I tried to use Directory and File Pathnames and FeatureReader to control workflow, I used Directory and File Pathnames linked to a gdb file, then put a Tester after it, connected passed port to FeatureReader. But it seems that gdb was treated as folder, every file in picture below was passed to FeatureReader, as result all feature were read muti-times instead of one, how can I set parameters right to get what I want? Or else, how can I control workflow if I don't want some gdbs to be read in fmt?

virtualcitymattEvangelist

3DTiles, glTF - Reflective surfaces

I've been experimenting with writing out 3DTiles and gLTF/GLB models with FME. One issue I've run into is that FME seems to be creating some kind of shininess to the surfaces which I can't seem to get rid of. The model just has a single color appearance which I want to be able to cast shadows if there is a light source, however, I don't want the appearance to be 'shiny', it should just be dull. I've set my appearance to be a single Diffuse (I've also tried Ambient) color. I've set the shininess to 0 and for I've set the Opacity to 1. This is in my resulting gLTF file, FME has given a 'metallicFactor' to the model which I'm guessing is why there is a shine to it: "materials" : [ { "name" : "[Color M02]", "pbrMetallicRoughness" : { "baseColorFactor" : [ 0.7764705882352941, 0.7764705882352941, 0.7764705882352941, 1 ], "roughnessFactor" : 1, "metallicFactor" : 0.5 } } ],This also comes through into my 3D Tile when creating them. I see the same properties in the b3dm file

FileNamePartExtractor for Multiple Files

Hi Experts,I am merging many csv files into one and I want to get the name of the input file in an attribute 'filename'. I am using FileNamePartExtractor for this purpose and its working correct with single input. But, when I use this with multiple files, it takes the name of the last input file and shows in all the rows. Can anyone help me with where I am going wrong!!

deanhowellContributor

Seeking XML Output Examples

Hello All,I am trying to understand the best way to take a series of features that are created when I import a DWG and export to an XML file based on the Esri Cadastral Fabric Data Model.Basically the validation creates a four tables that form a hierarchical structure - plan, parcels, lines and points. A plan contains multiple parcels and each parcel contains multiple lines and each line contains multiple points. The model is fairly complex but at present the data is output to PostGIS tables.I have an XSD that has been created from an existing XML - see attached zip file.What I am seeking is an understanding how to take the four tables as above and create the XML based on the XSD structure rather than sending them to PostGIS.Any examples or assistance would be very much appreciated.

Problem WFS query

Hello, I'm trying to get open LiDAR data from https://remotesensing.agiv.be/gis/geoserver/wms?REQUEST=GetCapabilitiesAs I only need an extent, a big one nonetheless (Xmin = 40050, Ymin = 202950, Xmax = 48385,3778, Ymax = 206209,4442, EPSG: 31370) I always get the same error on I guess the same area.---------------------------HTTP transfer error: 'Failure when receiving data from the peer'XML Parser error: 'Error in input dataset:'https://remotesensing.agiv.be:443/gis/geoserver/wfs?SERVICE=WFS&VERSION=1.1.0&REQUEST=GetFeature&TYPENAME=Agiv%3ALiDAR_DHMV_II_LAZtiles&NAMESPACE=xmlns%28Agiv%3Dhttp%3A%2F%2Fmobile-mapping.agiv.be%29&BBOX=40050%2C202950%2C48385.3778%2C206209.4442%2Curn%3Ax-ogc%3Adef%3Acrs%3AEPSG%3A31370&OUTPUTFORMAT=application%2Fgml%2Bxml%3B%20version%3D3.2&MAXFEATURES=30000' line:1 column:415988 message:input ended before all started tags were ended; last tag started is 'gml:posList''The XML Module halted on err

FME Cloud OAuth Web Connection Authorization times out

Hello,I am running into OAuth issues on FME Cloud 2019 with the Google API. After approximately 1 hour the HttpCaller no longer authenticates with the web connection. I can re-authorize the web connections and all is good. We want to run this on a schedule and do not want to re-authorize each time we start up. @GerhardAtSafe, ideas?

Filecopy Renaming Issue

I am having difficulty with the output of the filecopy writer that I am using. I have the filecopy_source_dataset setup correctly in the sense that it is pointing correctly at the full path of the original file. My filecopy_dest_dataset is set to write to the destination I want and create subfolders based on the value PLAN_SET_ID.I have set the filecopy_dest_filename to what I want the different record drawings labeled as. The issue I am having is on the creating of the .TIF files, some of the folders contain only the TIF's, which is what I want and only happens on about half of the .TIF being written out. Here is an example of what I am wanting. What I am getting for the other half of the .TIF's is this. Folders with subfolders that are empty, but include part of the name of a .tif that should be written out.Or a combination of subfolders that are empty, but include part of the name of a .tif that should be written out along with .tifs that wrote out correctly. I am a bit c

Unable to convert value taken from Z_ORIGIN keyword to a float data type.

Im trying to write a polygon from one polygon to another using FMEs GEODATABASE_SDE writer against a SDE database in SQL Server. Unfortunately Im totally stuck. Everytime i try writing something I get an error saying Unable to convert value taken from Z_ORIGIN keyword to a float data type. A fatal error has occurred. Check the logfile above for details This makes totally no sense as I very carefully have removed all posibility traces of anything 3D using both the 2DForcer and in the writers format parameters have anwsered no to if there is any Z values. Have anyone the slightest idea whats happening ???

Create new record with the attribute names as the attribute values

Hi,I would like to write to an Excel Spreadsheet where the header row is formatted differently from the rest of the spreadsheet. I tried using an Excel Template but that requires me to overwrite the output every time and I do not want to overwrite the existing output.I thought perhaps I could add a new record where the values are the same as the column names and then use a TestFilter and ExcelStyler to style that row of values differently from the rest, but I can't work out how to create a new record with the column names as values.I am using FME Desktop 2018.1Any ideas?Thanks,

jhawksContributor

Add WMS url to KML Navigation tree (as ground over?)

Trying to add WMS url in as a screen overlay/ ground overlay. I believe it has to be both, one to have it the navigation tree and one to display in GE. Here's the kml I'm trying to replicate: <ScreenOverlay id="kml_8"><Icon><href>https://seamlessrnc.nauticalcharts.noaa.gov/arcgis/services/RNC/NOAA_RNC/ImageServer/WMSServer?VERSION=1.1.1</href></Icon><overlayXY x="0" y="0" yunits="fraction" xunits="fraction"/><screenXY x="0" y="0" yunits="fraction" xunits="fraction"/></ScreenOverlay> <GroundOverlay> <name>Seamless NOAA RNCs</name> <visibility>0</visibility> <description>NOAA RNCs from:http://seamlessrnc.nauticalcharts.noaa.gov/arcgis/rest/services/RNC/NOAA_RNC/ImageServer&lt;br&gt; RNCs are updated by the 10th of each month.</description> <LookAt> <longitude>-117.

clemensbContributor

How to extract attributes from pdf

Hii am trying to extract attributes from a pdf (marked in the screenshot) In the inspector i see in the attribute pdf_all_layer_names 2 layer (Map Frame, Gemeinde) but not de Number and the Attributes. I also tried to expose additional attributes, but i didn't find the attributes, which are in the red box.i am using FME2019.1

Heirarchical ID matching with ChangeDetector (or UpdateDetector)

I'm using FME 2018 (via ArcGIS data interoperability extension) so am limited to the use of the UpdateDetector, but I assume the same question is applicable to the ChangeDetector.Is there a way to specify multiple match fields? My use case is that I have a WFS data set that I am using to inform updates of an SDE table. There is a matching ID column between the two (system_id), but when an item in the WFS base database is replaced, the ID will be written to a separate column (prior_id) and the record given a new ID. Therefore, the equivalent record in the table- which contains data not stored in the database- would be removed by the UpdateDetector and the new record written in if just matching on system_id. So I would envision a solution taking the form of specifying the UpdateDetector to check for a match between WFS system_id and SDE table system_id, and if none exists check for a match between WFS prior_id and SDE table system_id, and if none exists treat the record as an insert

KML HTML Description broken out to fields

I have been working on an issue for a few months to automate ‘Expose Attribute’ in such transformers as XMLXQueryExtractor. FME said it is on there development list, I/we have to crack open these HTML fields and type the Expose Attribute on average once a week. I have tried many gyrations, wanted to see if you have done this or have an idea how to automate. Many Thanks

fmelizardContributor

How do I use JAVA_OPTS in a JDBC connection

If your looking to make a connection with JDBC Reader and you need to specify JAVA_OPTS you can do the following: 1) ADD a new multi-parameter System Environment Variable called JAVA_TOOL_OPTIONS and specify your paramters.JAVA_TOOL_OPTIONS = -Djava.security.login.config=C:\folder\some.config -Djava=anothervalue 2) Restart FME

dollargisParticipant

Aggregating filenames from filecopy writer

In a workspace, I do a conditional copying of files into several places, and I want to deliver the results of the copy actions - especially the target filenames - in one final emailWhat I do have is a structure like this (the condition part omitted for clarity)where the email sending part already works.But I can't figure out how to send the target filename each filecopy writer receives basically as a list.Any help would be greatly appreciated.

I have recently just discovered this software and it seems like a really powerful utility. What I would like to do is take a kmz file of building geometry that has been verified in Google Earth and convert the file into a meshable file type such as an stl. Is this possible? I see from the data inspector that all of the points necessary to define each building exist in LLA format and data sets are defined as boundaries and polygons so I'm inclined to think it's possible, I just can't get it to work. I've also noticed that kml/kmz files can't be inspected in 3D which I believe is a manifestation of my issue. Any help is greatly appreciated! I've attached a screenshot from the data inspector in 2D and 3D as well as one from Google Earth

erik_janContributor

Writing to MilSoft STD format

HI All, Has anybody tried using FME to write to the MilSoft STD format?The format is not in the formats list on the Safe website.Maybe somebody created a work-around for the format.Any help is highly appreciated. Thanks,Erik

pratapContributor

Small world Readers

Hi,I'm working on Smallworld for the first time. I observed that understanding of Small world readers are not like other readers. It has a unique feature of adding readers based on status i.e. existing, abandoned, etc but reads all the data when we add existing or abandoned reader. I do not understand this logic. Why Smallworld reader is reading all the data every time when we placed only existing/abandoned reader...?In my case pole_exi table has 1lakh records and pole_aba have 9 records as well as there is another reader with name pole which has 25k feature.1. Why it has so many readers when it is reading all 125009 features every time2. Is there any settings we have to change or it is not in our control3. Even if we pass "10" in "max features to read", surprisingly, it reads all 125009 and pass 10 features out :) by which it is even more wasting the time while testing.Can we any one help me... Thank you in advance

bark0sContributor

Workspace runner many to one database writer locking

I've got a bunch of input files that are stored in sub-directories, I use a path and directory reader to grab these and send them into a workspace runner. A bunch of things take place and then I want to write them all out to 2 layers within a gdb. So dozens of files being brought together to 2 central layers (point, polyline).That's all fine...when I don't run the child workspaces concurrently. I of course get layer locks if I run multiple processes.Ideally I'd like to run multiple processes and have each feature inserted/appended to the table.Any suggestions on how to get around this gdb locking? I'm happy to go out to another spatial db format, that I can later update.

How to dynamically change the name of the reader

Hi I am reading a json file. Here is one I am reading now: https://outages.electra.co.nz/api/weboutageviewer/get_live_data?Start=&End=&Duration=0&CustomerResponsible=false&Historical=false&_=1564371572592What I want to know is how do I create a reader that can update dynamically if the number at the end changes ie.1564371572592 will not stay constant and this will change.

File Geodatabase Where Clause for geometry filter

Hi,I can't seem to find anywhere the sql syntax for performing a geometry filter in the where clause of the File Geodatabase - (Esri Geodatabase (File Geodb Open API) Reader). I would like to only read in geometry of a certain min x, min x, max y and max y bounding.Also, is using the 'where' clause for this the quickest way of filtering a File Geodatabase? I have millions of records so need to optimize the filtering.I am using FME2017.1Thank you in advance.

Combine multiple KML files into one

Hello - in FME Workbench 2018.1 I am adding 2 KML Readers (a point file and a polygon file) to a workspace, and just want to combine them into one KML Writer containing the points, polygons and user attributes. When I inspect the reader I can see the attribute table with all the user attributes, but when I inspect the writer there is no attribute table. The only transformer I have is KMLPropertySetter, and Include Attribute Table has "Yes" selected. What do I need to do to include the user attribute table in my resulting KML file?Thank you

GeometryPropertyExtractor and messy IFC Psets

Hello,I have a fairly messy IFC where I need to extract all geometry traits (Pset.Attribute) and I ran into an issue. Not sure what to do here...I'm using the GeometryPropertyExtractor to get all "Pset.Attribute" values and it gives me this:They are supposed to be the same "Pset.Attribute" (by name), but the GeometryPropertyExtractor decided to give one of them a "{0}".I think what's happening is, for some reason one of the Psets has a different ID, so technically it's correct, just not what I want? Maybe?I guess I can remove all "{0}" with the BulkAttributeRenamer and hope for the best? Or is there a way to force the GeometryPropertyExtractor to use the Pset name instead of the id?Or is there a different reason?

Conditionally join tables

Hi,i have two tables, one containing a code list and the meaning of each code:Code Meaning 500 Grass 501 Tree 502 Street The other table, coming from the land survey team, contains the coordinates and the code of the point.X Y Code 51XXXXX1 9XXXXXXX1 501 51XXXXX2 9XXXXXXX2 500 51XXXXX3 9XXXXXXX3 500 I want the meaning of each code in this table is added in a new column:X Y Code Meaning 51XXXXX1 9XXXXXXX1 501 Tree 51XXXXX2 9XXXXXXX2 500 Grass 51XXXXX3 9XXXXXXX3 500 Grass I m new to FME, usually i would do this with a script, e.g. in QGIS. My FeatureMerge options so far look like:Any hints are welcome!Thank you!

bubblebebContributor

KML Reader Error initialising

I have updated to FME 2016 and have had a few glitches. The recent one is using a KML reader which errors when starting with the following message. |FATAL |KML: Cannot open dataset `E:\\... data\\**\\*.kmz'|FATAL |KML: Error initializing readerAny ideas what the cause could be and how to fix it.

1
...
102
103
104
105
106
107
108
...
269

Page 105 / 269

Badge winners

mavohas earned the badge Welcome Back
narasimhahas earned the badge Welcome Back
zchervehas earned the badge Welcome Back
matt.landrumhas earned the badge Webinar Watcher (Grey)
lenusmarshas earned the badge Welcome Back

Show all badges

29,729: Posts

Powered by Gainsight

Terms & Conditions Cookie settings

Sign up

Already have an account? Login

An FME Account is required to contribute

Login

Login to the community

An FME Account is required to contribute

Login

Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.

Enter your e-mail address

Back to overview

Scanning file for viruses.

Sorry, we're still checking this file's contents to make sure it's safe to download. Please try again in a few minutes.

OK

This file cannot be downloaded

Sorry, our virus scanner detected that this file isn't safe to download.

OK