Using readers and writers to extract and load data.
Recently active
I need to get geometry and attributes out of AIXM 5.1 into ESRI geodatabase.
Hi list.Does the FeatureReader reading WFS 2.0.0 data handle paging ?If a service is known to be limited to return a maximum number of features per call, paging, i.e. fetching the complete dataset in chunks, is a way to handle this.However, the only options I see are "Max Features" and "Count", but they don't incur paging, as far as I can tell.So how does FME handle WFS 2.0.0 paging ?CheersLars I
For some reason, I am unable to write to two consecutive specified columns in an excel spreadsheet. I have specified the Column B and 3 rows of that for one set of data. I have also specified Column D and 3 rows of that for the other set of data. It doesn't matter if I use the featurewriter or the xlsx writer. I am unable to get Columns B and D to populate together. My export comes from a template that has calculations in it. There is a ROW attribute that works correctly with fixed values of 37, 38, and 40. The column is specified as the field B and D for each data stream which is correct. They are both going to the same sheet and I have been unable to make the other column that is not writing write to another column or sheet somewhere. I do this often with other reports and have not yet had this issue. Driving me crazy. Any suggestions would be appreciated :) Thanks.
I have over 100 Excel files with standard format: each Excel file has multiple sheets corresponding to multiple days in one month on recorded hourly well flow data, and the format for each sheet is fixed. The top part of sheet has well header info while the lower part are recorded hourly well flow data. I need to combine the well header info and hourly flow data into standard table format and load into SQL Server database table. It is easy to read the hourly flow data into. My challenge is to also read well header info from the same sheet , combine with hourly flow data, and write all these data into a table. I am looking for examples and help on similar Excel data loading. Thanks in advance.
Hello, I would like to know how ca I read the Python data. I need to build a Workbench which as reader will be a Pyton script.py Is there any transformator? Thanks Tereza
I have a JPG image and a CSV file with data that can be used to geo-reference it (attached).I've built a Workspace to apply the geo-referencing data to the JPG. AttributeManager makes FactorY a negative number as I found I needed to do this when working out the process using ArcGIS. In the RasterGeoferenencer, I've selected the relevant fields from the CSV file as shown in the screenshot. OutX and OutY are the Ordnance Survey Eastings and Northings.When I run the Workspace in FME 2017.0.1.1 (64-bit), I get the following error:2017-06-08 15:39:10| 0.8| 0.0|ERROR |@GeoreferenceRaster: Failed to retrieve X Upper Left Coordinate parameterI've tried setting the coordinate system on the JPG Reader and/or the CSV Reader, and setting OutX and OutY to x_coordinate and y_coordinate in the CSV Reader and then as real32. None of these make a difference.Any ideas?
Hello I've created a workspace, writing an IFX 2x3 file with two Property Sets per feature, which works well.Changing the writer format to IFC 4 the two Property Sets are not written. I've no Idea, what the problem could be and hope, that somebody could tell me, what I'm doing wrong.You'll find the workspace and the Data attached. Regards, Hartmut
Hi everybody, I need to write to a sqlite database containing tables with foreign keys constraints, but FME seems not to enforce the foreign_keys pragma directive when connecting to the database. I discovered this by running a pragma foreign_keys; query against the database with an SQLCreator (it returned 0).This is obviously a problem, because in this way the SQLite Writer can insert records that violate referential integrity.Is there a way to make the SQLite Writer execute thepragma foreign_keys = 1;directive before start writing? Any suggestion is welcome! Thanks
Hi,I have a postgresql table pre_val_error with following fields GRID_ID|ERROR_LEVEL|ERROR121|1|ERROR1121|2|ERROR2121|3|ERROR3 The requirement is to write all the records to a csv file and insert a record in Oracle with count grouped by error_level. The Oracle table details are below GRID_ID|ERROR_LVL1_COUNT|ERROR_LVL2_COUNT|ERROR_LVL3_COUNT121|1|1|1 As you can see, the number or records has to be grouped by error_level and inserted in Oracle. I have tried aggregator, counter and nothing worked. Can anyone suggest a solution for this problem.
I have some text with HTML tags and properties. I need to strip out the HTML parts and leave plain text. I'm using the HTMLStripper which does the job pretty good, but replaces accented letters with non-accented ones plus adds question marks.How this behaviour can be fixed? Text example (before and after):<p>Iš liet. <em>šveñtas</em>, <em>-à </em>&lsquo;<span class="g7"><span class="g7">pagarbiai saugomas, laikomas; iš pagarbos neliečiamas</span></span>&rsquo;.</p>Iš liet. šven?tas, -a? ‘pagarbiai saugomas, laikomas; iš pagarbos neliečiamas’.
HelloI would like to put points on a single line to use the named transformer and get the length as a new attribute, but it seems that the feature merger used in this examplehttps://gis.stackexchange.com/questions/30518/fme-calculate-distance-of-point-along-a-linedoes not work on Esri Shapefile format, I don't know why.Any idea on how to fix points on a curve, while keeping each point as a line/different entity in the visual preview ?Thanks Edit: MeasureGenerator does not work as the curve has much more vertex than needed and they can't really be sorted
There are two lines and I need proper centerline between them. Approximate gives good results at the ends, but fails in some corners. Medial Axis fixes the corner issue, but messes up the ends (shortens them) and makes unnecessary additional lines.Is it possible to get the mix of both: medial axis with proper length ends and without additional line segments?For reference, see the attached pics. Workflow is:Make buffers from lines (end caps: square, corner: mitter);Dissolve buffers;Use CenterLineReplacer.
I need that tag because I want to move from a road to one of all buildings that are located near that road (hyperarch), if it possible and there aren't walls obv. I tried to search for hours, but I don't know in which file of my (generic) city model that tag is. Can anyone help me?
Hi, I got an error shown below when I try to add CesiumIonConnector into my workspace and authenticate to allow accesss to FME. Does anyone know how to solve it?
Apart from using a SystemCaller with a Template Zero row DBase File to File Copy/Paste, I can't see any immediate way of doing this easily in FME? The dBase Writer doesn't have a Truncate or Delete option, so the only messy way I can think of is to instead write a SHP file with illegal geometry (this Writer Hack will then create the SHP and dBase files but pass over the Features with geometry not of a type matching the SHP schema and hence output a "Zero Feature" SHP file), and then deleting all the output files except for the dBase. Any better solutions?
IFC is our default file format for working on BIM related projects. Unfortunately, Esri doesn't support a native IFC file export (yet). However, when importing an IFC file via FME into a ArcGIS Pro geodatabase, it naturally get converted into a multipatch feature where you can do all kind of attribute wizardry. However, people then request to get this modified multipatch feature back as a fully functional and valid IFC file. So my questions: 1.) what's our best option to return a multipatch feature back into an IFC file with correct attributes & geometry? Has anyone done this in FME so far and has some guidance?2.) there's this open OGC standard for indexed 3D scene layers (I3S) which result in an slpk file, when exporting the multipatch feature for web usage. Since it has open specifications, did anyone try to create a suitable converter that goes from slpk to IFC? Has anyone tried this?
Hi,I am copying a Portal hosted feature service to our internal Oracle SDE database.The relationship class is written ok and I can see that the correct attachment is associated with the parent table. However the image is not valid when I try to open it via ArcGIS Pro or ArcMap.I've read a lot of forum posts including this one but I still haven't got it working. Does anyone have an example with the last steps on how to get the downloaded attachment (binary data) into a SDE jpeg attachment that I could take a look at?Or outline the order of the transformers Binarydecoder, rasterreplacer (I see the raster in inspector, only if I use the decoder first), AttributeFileWriter (you write the file to a disk but not the database - do you use the AttributeFileReader to read it again and write to an attribute in the attachment table?) Any ideas would be appreciated. Thanks in advance,Annette
Hello everyone, I'm working with the citygml model and to implement the buildings module I used the xlink_href tool. It's all right for surfaces that are not shared between buildings. For those that are shared, i need to set the surface in one of the buildings as gml: OrientableSurface and reverse the orientation. This is necessary for the surface to be drawn in the citygml viewers in both buildings. I tried to use the Orientor but it didn't work and I also tried to create an orientation attribute and set it as a trait with the GeometryPropertySetter also without success. Would anyone know how to do this?At the moment my output is as in figure1 and I would like it to be as in figure2 (I edited it manually just to illustrate). Thank you in advance for your help. Mcdonnell
I am trying to open an IFC file from a client but I keep getting a syntax error:ISO10303-21: The input data file has a syntax error in instance '#105':syntax error, unexpected P21_TOK_STD_KEYWORD, expecting ')'ISO10303-21: Invalid file format, unable to continue readingA fatal error has occurred. Check the logfile above for detailsI looked up this element (#105) using Python ifcopenshell, and found that it is an IfcPostalAddress element. I've modified the content below so it is not the original address, but I have left in the word "États-Unis" (French for "United States), because I believe this is what is causing the syntax error."#105= IFCPOSTALADDRESS($,$,$,$,('123 Fake Street, Faketown, FS 01234 Ã\x89tats-Unis'),$,'','','','User Defined');\n"I thought I would be able to fix it by re-encoding it to utf-8 and saving it appropriately (Python snippet below):with open(out_file, "w", encoding="utf-8") as f_out: with open(in_file, encoding="latin1") as f_in: for line in f_in.read
Hey guys, I am converting an fGDB file to a Geopackage-File. The workspace in FME (2017, 32-BIT) runs successfully. If i want to view the Geopackage-File in the Data Inspector or QGIS, all is working well. But if i want to view it in ArcMap 10.3 or ArcGIS Pro through the ArcCatalog I always get the following error message: Does anybody know what could be causing this error? Thanks a lot in advance!
Hi All,I am extracting a postgres data using the SQL creator and trying to write it to the File geodatbase. but every time I am getting an error, but if i try to write it using the Excel format then the it successfully writes the data . I am getting an below mention error :GGeodatabase Error (-2147219884): The Field already exists.FileGDB Writer: Could not create feature class 'FeatureClass1'A fatal error has occurred. Check the logfile above for detailsGeodatabase Error (-2147418113): Unexpected operation.FileGDB Writer: Could not disable Load Only Mode for feature type 'FeatureClass1'Geodatabase Error (-2147418113): Unexpected operation.FileGDB Writer: Could not free Write Lock for feature type 'FeatureClass1'
Hello,I'm having an issue where FeatureWriters sometimes refuse to overwrite geodatabases previously populated with features loaded from the same translation. I receive the same (or very similar) error messages as found in this thread:https://knowledge.safe.com/questions/45200/writing-esri-file-geodatabase-issue.htmlI am currently using the following Table parameters for my FeatureWriters (in several translations):Feature Operation: InsertTable Handling: Drop and CreateA solution to this problem in my case is to delete the target feature class and reload the translations, but going forward this is not viable as these translations will be running on an automated basis and my client prefers very little (if any) maintenance to files or translations and doesn't want to delete feature classes and run one or two of these manually every week.I tried using "Overwrite Feature Class" in the writer parameters, but that does not work at all. I receive an error message that the GDB is locked, even
I am trying to export some data to dwg model space and then use that data in a layout via viewport, I am having success but I cannoot get the viewport to focus on specific coordinate in model space, here are the attributes I have created, everything works except the location.
Hi all,i am trying to connect with a Azure blob storage container, with the AzureBlobStorageConnector transformer with the fme 2020.0.2.1 version.but its not working i am getting this message: 'Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature. ErrorCode: AuthenticationFailed' and when i use the SAS url in the browser is all the containers that are available Could someone help about this topic, please? http://docs.safe.com/fme/2019.0/html/FME_Desktop_Documentation/FME_Transformers/Transformers/azureblobstorageconnector.htm
I have a couple of thousands of DWG files. Those have object data attributes. I need to extraxt that information. Then using AutoCAD Map 3D Object Data reader, it crashes on some files and stops reading other files (see the attached error message). Then using basic AutoCAD DWG/DXF reader – everything's fine, no crashes or skipped files, but no object data info are retrieved.Maybe it is possible to skip reading bad DWGs and continue to read the other ones with Map 3D reader? Any thoughts on that would be helpful.