Using readers and writers to extract and load data.
Recently active
Hello All,I am trying to understand the best way to take a series of features that are created when I import a DWG and export to an XML file based on the Esri Cadastral Fabric Data Model.Basically the validation creates a four tables that form a hierarchical structure - plan, parcels, lines and points. A plan contains multiple parcels and each parcel contains multiple lines and each line contains multiple points. The model is fairly complex but at present the data is output to PostGIS tables.I have an XSD that has been created from an existing XML - see attached zip file.What I am seeking is an understanding how to take the four tables as above and create the XML based on the XSD structure rather than sending them to PostGIS.Any examples or assistance would be very much appreciated.
Hello, I'm trying to get open LiDAR data from https://remotesensing.agiv.be/gis/geoserver/wms?REQUEST=GetCapabilitiesAs I only need an extent, a big one nonetheless (Xmin = 40050, Ymin = 202950, Xmax = 48385,3778, Ymax = 206209,4442, EPSG: 31370) I always get the same error on I guess the same area.---------------------------HTTP transfer error: 'Failure when receiving data from the peer'XML Parser error: 'Error in input dataset:'https://remotesensing.agiv.be:443/gis/geoserver/wfs?SERVICE=WFS&VERSION=1.1.0&REQUEST=GetFeature&TYPENAME=Agiv%3ALiDAR_DHMV_II_LAZtiles&NAMESPACE=xmlns%28Agiv%3Dhttp%3A%2F%2Fmobile-mapping.agiv.be%29&BBOX=40050%2C202950%2C48385.3778%2C206209.4442%2Curn%3Ax-ogc%3Adef%3Acrs%3AEPSG%3A31370&OUTPUTFORMAT=application%2Fgml%2Bxml%3B%20version%3D3.2&MAXFEATURES=30000' line:1 column:415988 message:input ended before all started tags were ended; last tag started is 'gml:posList''The XML Module halted on err
Hello,I am running into OAuth issues on FME Cloud 2019 with the Google API. After approximately 1 hour the HttpCaller no longer authenticates with the web connection. I can re-authorize the web connections and all is good. We want to run this on a schedule and do not want to re-authorize each time we start up. @GerhardAtSafe, ideas?
I am having difficulty with the output of the filecopy writer that I am using. I have the filecopy_source_dataset setup correctly in the sense that it is pointing correctly at the full path of the original file. My filecopy_dest_dataset is set to write to the destination I want and create subfolders based on the value PLAN_SET_ID.I have set the filecopy_dest_filename to what I want the different record drawings labeled as. The issue I am having is on the creating of the .TIF files, some of the folders contain only the TIF's, which is what I want and only happens on about half of the .TIF being written out. Here is an example of what I am wanting. What I am getting for the other half of the .TIF's is this. Folders with subfolders that are empty, but include part of the name of a .tif that should be written out.Or a combination of subfolders that are empty, but include part of the name of a .tif that should be written out along with .tifs that wrote out correctly. I am a bit c
Im trying to write a polygon from one polygon to another using FMEs GEODATABASE_SDE writer against a SDE database in SQL Server. Unfortunately Im totally stuck. Everytime i try writing something I get an error saying Unable to convert value taken from Z_ORIGIN keyword to a float data type. A fatal error has occurred. Check the logfile above for details This makes totally no sense as I very carefully have removed all posibility traces of anything 3D using both the 2DForcer and in the writers format parameters have anwsered no to if there is any Z values. Have anyone the slightest idea whats happening ???
Hi,I would like to write to an Excel Spreadsheet where the header row is formatted differently from the rest of the spreadsheet. I tried using an Excel Template but that requires me to overwrite the output every time and I do not want to overwrite the existing output.I thought perhaps I could add a new record where the values are the same as the column names and then use a TestFilter and ExcelStyler to style that row of values differently from the rest, but I can't work out how to create a new record with the column names as values.I am using FME Desktop 2018.1Any ideas?Thanks,
Trying to add WMS url in as a screen overlay/ ground overlay. I believe it has to be both, one to have it the navigation tree and one to display in GE. Here's the kml I'm trying to replicate: <ScreenOverlay id="kml_8"><Icon><href>https://seamlessrnc.nauticalcharts.noaa.gov/arcgis/services/RNC/NOAA_RNC/ImageServer/WMSServer?VERSION=1.1.1</href></Icon><overlayXY x="0" y="0" yunits="fraction" xunits="fraction"/><screenXY x="0" y="0" yunits="fraction" xunits="fraction"/></ScreenOverlay> <GroundOverlay> <name>Seamless NOAA RNCs</name> <visibility>0</visibility> <description>NOAA RNCs from:http://seamlessrnc.nauticalcharts.noaa.gov/arcgis/rest/services/RNC/NOAA_RNC/ImageServer&lt;br&gt; RNCs are updated by the 10th of each month.</description> <LookAt> <longitude>-117.
Hii am trying to extract attributes from a pdf (marked in the screenshot) In the inspector i see in the attribute pdf_all_layer_names 2 layer (Map Frame, Gemeinde) but not de Number and the Attributes. I also tried to expose additional attributes, but i didn't find the attributes, which are in the red box.i am using FME2019.1
I'm using FME 2018 (via ArcGIS data interoperability extension) so am limited to the use of the UpdateDetector, but I assume the same question is applicable to the ChangeDetector.Is there a way to specify multiple match fields? My use case is that I have a WFS data set that I am using to inform updates of an SDE table. There is a matching ID column between the two (system_id), but when an item in the WFS base database is replaced, the ID will be written to a separate column (prior_id) and the record given a new ID. Therefore, the equivalent record in the table- which contains data not stored in the database- would be removed by the UpdateDetector and the new record written in if just matching on system_id. So I would envision a solution taking the form of specifying the UpdateDetector to check for a match between WFS system_id and SDE table system_id, and if none exists check for a match between WFS prior_id and SDE table system_id, and if none exists treat the record as an insert
I have been working on an issue for a few months to automate ‘Expose Attribute’ in such transformers as XMLXQueryExtractor. FME said it is on there development list, I/we have to crack open these HTML fields and type the Expose Attribute on average once a week. I have tried many gyrations, wanted to see if you have done this or have an idea how to automate. Many Thanks
If your looking to make a connection with JDBC Reader and you need to specify JAVA_OPTS you can do the following: 1) ADD a new multi-parameter System Environment Variable called JAVA_TOOL_OPTIONS and specify your paramters.JAVA_TOOL_OPTIONS = -Djava.security.login.config=C:\folder\some.config -Djava=anothervalue 2) Restart FME
In a workspace, I do a conditional copying of files into several places, and I want to deliver the results of the copy actions - especially the target filenames - in one final emailWhat I do have is a structure like this (the condition part omitted for clarity)where the email sending part already works.But I can't figure out how to send the target filename each filecopy writer receives basically as a list.Any help would be greatly appreciated.
I have recently just discovered this software and it seems like a really powerful utility. What I would like to do is take a kmz file of building geometry that has been verified in Google Earth and convert the file into a meshable file type such as an stl. Is this possible? I see from the data inspector that all of the points necessary to define each building exist in LLA format and data sets are defined as boundaries and polygons so I'm inclined to think it's possible, I just can't get it to work. I've also noticed that kml/kmz files can't be inspected in 3D which I believe is a manifestation of my issue. Any help is greatly appreciated! I've attached a screenshot from the data inspector in 2D and 3D as well as one from Google Earth
HI All, Has anybody tried using FME to write to the MilSoft STD format?The format is not in the formats list on the Safe website.Maybe somebody created a work-around for the format.Any help is highly appreciated. Thanks,Erik
Hi,I'm working on Smallworld for the first time. I observed that understanding of Small world readers are not like other readers. It has a unique feature of adding readers based on status i.e. existing, abandoned, etc but reads all the data when we add existing or abandoned reader. I do not understand this logic. Why Smallworld reader is reading all the data every time when we placed only existing/abandoned reader...?In my case pole_exi table has 1lakh records and pole_aba have 9 records as well as there is another reader with name pole which has 25k feature.1. Why it has so many readers when it is reading all 125009 features every time2. Is there any settings we have to change or it is not in our control3. Even if we pass "10" in "max features to read", surprisingly, it reads all 125009 and pass 10 features out :) by which it is even more wasting the time while testing.Can we any one help me... Thank you in advance
I've got a bunch of input files that are stored in sub-directories, I use a path and directory reader to grab these and send them into a workspace runner. A bunch of things take place and then I want to write them all out to 2 layers within a gdb. So dozens of files being brought together to 2 central layers (point, polyline).That's all fine...when I don't run the child workspaces concurrently. I of course get layer locks if I run multiple processes.Ideally I'd like to run multiple processes and have each feature inserted/appended to the table.Any suggestions on how to get around this gdb locking? I'm happy to go out to another spatial db format, that I can later update.
Hi I am reading a json file. Here is one I am reading now: https://outages.electra.co.nz/api/weboutageviewer/get_live_data?Start=&End=&Duration=0&CustomerResponsible=false&Historical=false&_=1564371572592What I want to know is how do I create a reader that can update dynamically if the number at the end changes ie.1564371572592 will not stay constant and this will change.
Hi,I can't seem to find anywhere the sql syntax for performing a geometry filter in the where clause of the File Geodatabase - (Esri Geodatabase (File Geodb Open API) Reader). I would like to only read in geometry of a certain min x, min x, max y and max y bounding.Also, is using the 'where' clause for this the quickest way of filtering a File Geodatabase? I have millions of records so need to optimize the filtering.I am using FME2017.1Thank you in advance.
Hello - in FME Workbench 2018.1 I am adding 2 KML Readers (a point file and a polygon file) to a workspace, and just want to combine them into one KML Writer containing the points, polygons and user attributes. When I inspect the reader I can see the attribute table with all the user attributes, but when I inspect the writer there is no attribute table. The only transformer I have is KMLPropertySetter, and Include Attribute Table has "Yes" selected. What do I need to do to include the user attribute table in my resulting KML file?Thank you
Hello,I have a fairly messy IFC where I need to extract all geometry traits (Pset.Attribute) and I ran into an issue. Not sure what to do here...I'm using the GeometryPropertyExtractor to get all "Pset.Attribute" values and it gives me this:They are supposed to be the same "Pset.Attribute" (by name), but the GeometryPropertyExtractor decided to give one of them a "{0}".I think what's happening is, for some reason one of the Psets has a different ID, so technically it's correct, just not what I want? Maybe?I guess I can remove all "{0}" with the BulkAttributeRenamer and hope for the best? Or is there a way to force the GeometryPropertyExtractor to use the Pset name instead of the id?Or is there a different reason?
Hi,i have two tables, one containing a code list and the meaning of each code:Code Meaning 500 Grass 501 Tree 502 Street The other table, coming from the land survey team, contains the coordinates and the code of the point.X Y Code 51XXXXX1 9XXXXXXX1 501 51XXXXX2 9XXXXXXX2 500 51XXXXX3 9XXXXXXX3 500 I want the meaning of each code in this table is added in a new column:X Y Code Meaning 51XXXXX1 9XXXXXXX1 501 Tree 51XXXXX2 9XXXXXXX2 500 Grass 51XXXXX3 9XXXXXXX3 500 Grass I m new to FME, usually i would do this with a script, e.g. in QGIS. My FeatureMerge options so far look like:Any hints are welcome!Thank you!
I have updated to FME 2016 and have had a few glitches. The recent one is using a KML reader which errors when starting with the following message. |FATAL |KML: Cannot open dataset `E:\\... data\\**\\*.kmz'|FATAL |KML: Error initializing readerAny ideas what the cause could be and how to fix it.
I have a PDF file like this: No.productleaderPartnerStatus1xxxcompany O5Parner1 (5%), Parner5 (80%) Parner67 (15%)ok2yyycompany 08Parner A (50%) Parner B (25%), Parner C (25%)ok3zzzcompany07Parner I (50%) Parner II (50%)ok4bbbcompany10Parner k (100%)ok And I need to have an excell file like : No.ProductLeaderPartner%Status1xxxCompany 05Parner15%ok1xxxCompany 05Parner580%ok1xxxCompany 05Parner6715%ok2yyyCompany 08Parner A50%ok2yyyCompany 08parner B25%ok2yyyCompany 08Parner C25%ok3zzzCompany 07Parner I50%ok3zzzCompany 07Parner II50%ok4bbbcompany10Parner k100%ok
Hello,my first problem: Writing a GeoJSON in the right order. Simple example:I want a output in the order A,B,C (like I set it in the writer). But the GeoJSON file is A,C,B, because the first row in B is empty. Second Problem: When the B column is empty. Only A and C is written. How can I get an empty column in the writer? Thanks for your feedback!
Hi @mark2atsafe @andreaatsafe, I am using FME 2018.1.1.1 Build 18578.I am trying to connect to a Microsoft SharePoint list via SAML since it is a SharePoint Online instance, but am getting the following error message. It is message type '200' which apparently indicates success, but this doesn't seem to be the case here...Creating reader for format: Microsoft SharePoint List Trying to find a PYTHON plugin for reader named `SHAREPOINT'Loaded Python module `sharepoint' from file `C:\\Program Files\\FME_Desktop_2018.1.1.1\\plugins/python27\\sharepoint\\__init__.pyc'Creating reader for format: Microsoft SharePoint ListTrying to find a PYTHON plugin for reader named `SHAREPOINT'Loaded Python module `sharepoint' from file `C:\\Program Files\\FME_Desktop_2018.1.1.1\\plugins/python27\\sharepoint\\__init__.pyc'Microsoft SharePoint List Reader: Items Per Request format parameter is set to '5000'Microsoft SharePoint List Reader: Logging into xxxxx as