Using readers and writers to extract and load data.
Recently active
Hi all,I have a table with (XYZ) coordinates measured in time. Each time the Z-value (field J and further) of the specific point changes. The XY-coordinates (field E and F) stay the same. Sometimes the Z-value is missing.How can I project all of these point without manual adjustments to the table? So for the second row we have:1st measurement: X=94439,5, Y=435795,0 and Z=-0,803702th measurement: X=94439,5, Y=435795,0 and Z=-0,805003th measurement: X=94439,5, Y=435795,0 and Z=-0,806004th measurement: X=94439,5, Y=435795,0 and Z= missing here so no projectionand so further onIn the end I want to have a geodatabase.Many thanks for you reply!Alan
Afternoon, Just wondering if any one uses FME to extract data from Northgate M3 - as I am having trouble getting the UPRN out of the worksheets to then link to our spatial layer which is an oracle database and the joiner would be the UPRN.Below is the field that i am trying to extract on the address tab If anyone which db that sits in that would be appreciated ThanksDarren
I have an Excel Spreadsheet and in one of the sheets there are about 10 tables, they all have the same column names and all appear in the same column, e.g. Description will appear in Column A for all 10 tables etc.Rows are not predictable, these are based on how much data is displayed.Rather than creating a named range in Excel, is there an easier way within FME to do this, maybe combining all 10 tables into one large table? Also if in future there were to be 15 tables within one spreadsheet, could this be incorporated to make it future proof?
Hello, I want to do the same operation like this tutorial : https://knowledge.safe.com/articles/19918/how-to-expose-feature-attributes-from-kml-tag.html or this topic : https://knowledge.safe.com/questions/29326/how-can-i-extract-the-html-table-from-kml-descript.htmlIt consits in expose feature attributes from KML tag.I've done the step with HTMLtoWHTMLconverter and I obtaint this : <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head> <title></title></head><body> Survey Parameters<br /> <br /> <hr /> <table border="0" padding="10"> <tr> <td></td> <td>Planned</td> <td>Current</td> <td>Desired</td> </tr> <tr> <td>Scan Frequency [Hz]</td> <td>66</td> <td>66</t
We're having significant performance issues with Xref drawings created by FME. In AutoCAD there are options in the "Save As" dialog to allow the drawing to be saved with layer and spatial indexes, but I can't find a way to set this in the DWG or Map OD writers in FME. Is this available somewhere or simply not possible with FME 2018? If it turns out this is not possible in the currently available writers, then as a work -around, if I safe the template drawing with layer and spatial indexes, will this carry through to the drawings created by the writer in my workspace? Thanks,Gary
One of my workspaces recently broke because FME can't read in the features from AGOL. I'm running Win 7 x64 and FME Beta Build 18198. Has anyone else experienced this issue?When trying to inspect the AGOL feature layer, this is the error I receive:ArcGIS Online Feature Service Reader: Requesting portal informationUpdating access token for 'Daniel.Matranga Esri ArcGIS Online'JSON Message: {"error":{"code":498,"error":"invalid_request","error_description":"refresh_token expired","message":"refresh_token expired","details":[]}}JSON Error: JSONObject["error"] not a string.Unable to update access token for 'Daniel.Matranga Esri ArcGIS Online'Updating access token for 'Daniel.Matranga Esri ArcGIS Online'JSON Message: {"error":{"code":498,"error":"invalid_request","error_description":"refresh_token expired","message":"refresh_token expired","details":[]}}JSON Error: JSONObject["error"] not a string.Unable to update access token for 'Daniel.Matranga Esri ArcGIS Online'Python Exception <HTTP
I'm wondering if there is a glitch in the Geomedia SQL Server Reader. I have a number of feature classes that have boolean fields in them that I want to use in my FME process. When I read in the data with a Reader and connect it to an AttributeExposer, the boolean field comes across, but the data all shows as "<missing>". However, when I do the same thing but with a FeatureReader, it works fine. So, I'm wondering if there is a glitch with this type of Reader.
Hi, When i set the CAT reader to read a .txt file the first character it returns on the first line is a '¿ ' which messes up the content of the first row. All other rows are read fine. The contents of the first row become shifted because of the extra character at the start of the text line. When opening the text file in a text editor I see nothing wrong. Any help would be welcome.
Is there a way to check the page size of an AutoCAD DWG that is read into a FME workspace?
We are converting Microstation 7 design file data to Oracle 3D spatial (12C).When there is a complex line (line with an embedded arc or a line that starts as arc), then FME strokes the arcs. This is acceptable. However, the SDO metadata (SDO_ELEM_INFO) still has direct definition of the original microstation complex line (SDO_ELEM_INFO_ARRAY(1,4,2, 1,2,1) indicating arc and line segment). I would expect the resulting metadata after transformation would be a line(1,2,1). The problem with this definition is that we cannot use Oracle spatial primitives to use these geometries. SDO_GEOM.VALIDATE_GEOMETRY_WITH_CONTEXT flags these as invalid geometries.The only workaround we found was to transform them again (Oracle spatial to Oracle Spatial) with just a reader and writer in the workspace. This is not acceptable as we are dealing with millions of linear features.We have tried - Geometry validator - this does not repair this type of error.We tried to modify fme_type before converting it to 3D
Hi, I tried to use Directory and File Pathnames and FeatureReader to control workflow, I used Directory and File Pathnames linked to a gdb file, then put a Tester after it, connected passed port to FeatureReader. But it seems that gdb was treated as folder, every file in picture below was passed to FeatureReader, as result all feature were read muti-times instead of one, how can I set parameters right to get what I want? Or else, how can I control workflow if I don't want some gdbs to be read in fmt?
I've been experimenting with writing out 3DTiles and gLTF/GLB models with FME. One issue I've run into is that FME seems to be creating some kind of shininess to the surfaces which I can't seem to get rid of. The model just has a single color appearance which I want to be able to cast shadows if there is a light source, however, I don't want the appearance to be 'shiny', it should just be dull. I've set my appearance to be a single Diffuse (I've also tried Ambient) color. I've set the shininess to 0 and for I've set the Opacity to 1. This is in my resulting gLTF file, FME has given a 'metallicFactor' to the model which I'm guessing is why there is a shine to it: "materials" : [ { "name" : "[Color M02]", "pbrMetallicRoughness" : { "baseColorFactor" : [ 0.7764705882352941, 0.7764705882352941, 0.7764705882352941, 1 ], "roughnessFactor" : 1, "metallicFactor" : 0.5 } } ],This also comes through into my 3D Tile when creating them. I see the same properties in the b3dm file
Hi Experts,I am merging many csv files into one and I want to get the name of the input file in an attribute 'filename'. I am using FileNamePartExtractor for this purpose and its working correct with single input. But, when I use this with multiple files, it takes the name of the last input file and shows in all the rows. Can anyone help me with where I am going wrong!!
Hello All,I am trying to understand the best way to take a series of features that are created when I import a DWG and export to an XML file based on the Esri Cadastral Fabric Data Model.Basically the validation creates a four tables that form a hierarchical structure - plan, parcels, lines and points. A plan contains multiple parcels and each parcel contains multiple lines and each line contains multiple points. The model is fairly complex but at present the data is output to PostGIS tables.I have an XSD that has been created from an existing XML - see attached zip file.What I am seeking is an understanding how to take the four tables as above and create the XML based on the XSD structure rather than sending them to PostGIS.Any examples or assistance would be very much appreciated.
Hello, I'm trying to get open LiDAR data from https://remotesensing.agiv.be/gis/geoserver/wms?REQUEST=GetCapabilitiesAs I only need an extent, a big one nonetheless (Xmin = 40050, Ymin = 202950, Xmax = 48385,3778, Ymax = 206209,4442, EPSG: 31370) I always get the same error on I guess the same area.---------------------------HTTP transfer error: 'Failure when receiving data from the peer'XML Parser error: 'Error in input dataset:'https://remotesensing.agiv.be:443/gis/geoserver/wfs?SERVICE=WFS&VERSION=1.1.0&REQUEST=GetFeature&TYPENAME=Agiv%3ALiDAR_DHMV_II_LAZtiles&NAMESPACE=xmlns%28Agiv%3Dhttp%3A%2F%2Fmobile-mapping.agiv.be%29&BBOX=40050%2C202950%2C48385.3778%2C206209.4442%2Curn%3Ax-ogc%3Adef%3Acrs%3AEPSG%3A31370&OUTPUTFORMAT=application%2Fgml%2Bxml%3B%20version%3D3.2&MAXFEATURES=30000' line:1 column:415988 message:input ended before all started tags were ended; last tag started is 'gml:posList''The XML Module halted on err
Hello,I am running into OAuth issues on FME Cloud 2019 with the Google API. After approximately 1 hour the HttpCaller no longer authenticates with the web connection. I can re-authorize the web connections and all is good. We want to run this on a schedule and do not want to re-authorize each time we start up. @GerhardAtSafe, ideas?
I am having difficulty with the output of the filecopy writer that I am using. I have the filecopy_source_dataset setup correctly in the sense that it is pointing correctly at the full path of the original file. My filecopy_dest_dataset is set to write to the destination I want and create subfolders based on the value PLAN_SET_ID.I have set the filecopy_dest_filename to what I want the different record drawings labeled as. The issue I am having is on the creating of the .TIF files, some of the folders contain only the TIF's, which is what I want and only happens on about half of the .TIF being written out. Here is an example of what I am wanting. What I am getting for the other half of the .TIF's is this. Folders with subfolders that are empty, but include part of the name of a .tif that should be written out.Or a combination of subfolders that are empty, but include part of the name of a .tif that should be written out along with .tifs that wrote out correctly. I am a bit c
Im trying to write a polygon from one polygon to another using FMEs GEODATABASE_SDE writer against a SDE database in SQL Server. Unfortunately Im totally stuck. Everytime i try writing something I get an error saying Unable to convert value taken from Z_ORIGIN keyword to a float data type. A fatal error has occurred. Check the logfile above for details This makes totally no sense as I very carefully have removed all posibility traces of anything 3D using both the 2DForcer and in the writers format parameters have anwsered no to if there is any Z values. Have anyone the slightest idea whats happening ???
Hi,I would like to write to an Excel Spreadsheet where the header row is formatted differently from the rest of the spreadsheet. I tried using an Excel Template but that requires me to overwrite the output every time and I do not want to overwrite the existing output.I thought perhaps I could add a new record where the values are the same as the column names and then use a TestFilter and ExcelStyler to style that row of values differently from the rest, but I can't work out how to create a new record with the column names as values.I am using FME Desktop 2018.1Any ideas?Thanks,
Trying to add WMS url in as a screen overlay/ ground overlay. I believe it has to be both, one to have it the navigation tree and one to display in GE. Here's the kml I'm trying to replicate: <ScreenOverlay id="kml_8"><Icon><href>https://seamlessrnc.nauticalcharts.noaa.gov/arcgis/services/RNC/NOAA_RNC/ImageServer/WMSServer?VERSION=1.1.1</href></Icon><overlayXY x="0" y="0" yunits="fraction" xunits="fraction"/><screenXY x="0" y="0" yunits="fraction" xunits="fraction"/></ScreenOverlay> <GroundOverlay> <name>Seamless NOAA RNCs</name> <visibility>0</visibility> <description>NOAA RNCs from:http://seamlessrnc.nauticalcharts.noaa.gov/arcgis/rest/services/RNC/NOAA_RNC/ImageServer&lt;br&gt; RNCs are updated by the 10th of each month.</description> <LookAt> <longitude>-117.
Hii am trying to extract attributes from a pdf (marked in the screenshot) In the inspector i see in the attribute pdf_all_layer_names 2 layer (Map Frame, Gemeinde) but not de Number and the Attributes. I also tried to expose additional attributes, but i didn't find the attributes, which are in the red box.i am using FME2019.1
I'm using FME 2018 (via ArcGIS data interoperability extension) so am limited to the use of the UpdateDetector, but I assume the same question is applicable to the ChangeDetector.Is there a way to specify multiple match fields? My use case is that I have a WFS data set that I am using to inform updates of an SDE table. There is a matching ID column between the two (system_id), but when an item in the WFS base database is replaced, the ID will be written to a separate column (prior_id) and the record given a new ID. Therefore, the equivalent record in the table- which contains data not stored in the database- would be removed by the UpdateDetector and the new record written in if just matching on system_id. So I would envision a solution taking the form of specifying the UpdateDetector to check for a match between WFS system_id and SDE table system_id, and if none exists check for a match between WFS prior_id and SDE table system_id, and if none exists treat the record as an insert
I have been working on an issue for a few months to automate ‘Expose Attribute’ in such transformers as XMLXQueryExtractor. FME said it is on there development list, I/we have to crack open these HTML fields and type the Expose Attribute on average once a week. I have tried many gyrations, wanted to see if you have done this or have an idea how to automate. Many Thanks
If your looking to make a connection with JDBC Reader and you need to specify JAVA_OPTS you can do the following: 1) ADD a new multi-parameter System Environment Variable called JAVA_TOOL_OPTIONS and specify your paramters.JAVA_TOOL_OPTIONS = -Djava.security.login.config=C:\folder\some.config -Djava=anothervalue 2) Restart FME
In a workspace, I do a conditional copying of files into several places, and I want to deliver the results of the copy actions - especially the target filenames - in one final emailWhat I do have is a structure like this (the condition part omitted for clarity)where the email sending part already works.But I can't figure out how to send the target filename each filecopy writer receives basically as a list.Any help would be greatly appreciated.
