Using readers and writers to extract and load data.
Recently active
Hi FME community,I’ve been tasked with trying to get the metadata we hold (in a SQL server database) into an INSPIRE-compliant format.I was hoping that we could use the metadata we currently hold (Title, Abstract, Temporal Extents etc) and run it through an INSPIRE XSD template. However, it’s pretty overwhelming when I look at the XSD templates (which I got from here - application-schemas/schemas at main · INSPIRE-MIF/application-schemas · GitHub) as there are so many nested parts (see screenshot). Is there a relatively straightforward method I could use for each topic of data (be they addresses, parcels of land etc - which have their own topic under the INSPIRE framework) which could be joined in FME and output an XML file which can be imported to the data in ArcPro? It seems a bit convoluted but I’m just trying to get an idea of where to go from here with a large shift in metadata storage and how we will be using it at my organisation going forward. Thanks
Hello, I am trying to update the date field records of a feature class using arcpy update cursor, but I am getting the following error (text in bold), please let me know If I am missing anything : Field Name : e_result_datetimeField Values : 20230803081554 Field datatype : Date Also, want to mention that I am able to update the string, double data type fields using updatecursor in FME with no issues - it’s only with date data type I am facing the problem. Python Exception <RuntimeError>: The value type is incompatible with the field type. [e_result_datetime]Traceback (most recent call last): File "<string>", line 46, in inputRuntimeError: The value type is incompatible with the field type. [e_result_datetime]Error encountered while calling method `input'PythonCaller (PythonFactory): PythonFactory failed to process feature
Hi AllI am trying to set up a workspace to react when a change is made to an ESRI Enterprise Geodatabase table but cant figure out how to watch for changes on the table.I could set up a scheduled task to read all of the data and then a change detector to update a further table but that doesn't seem a wise use of resources, particular if there are no changes.SO any ideas how to watch this database table for changes efficiently??Regards
Hi,I have a shapefile with lines, some of which show the degree of curvature...I would like to delete some lines based on the angle of curvature.How can I do that?
Hiich would like to write an gml Class "ExistingLandUseGrid" Object from the INSPIRE Specs http://inspire.ec.europa.eu/schemas/gelu/4.0Wenn adding a INSPIRE GML writer by pointing to the cooresponding GriddedExistingLandUse.xsd, the writer does not offer a featuretype "ExistingLandUseGrid" which is what i would expect. The only featuretype in the GML writer is "SpatialDataSet".Are there any options for the writer to enable writing a featuretype "ExistingLandUseGrid"? Or an example of a working script? RegardsPeter
Dear readers, A colleague of mine is trying to extract data from an FTP server.Using FME for this process would be beneficial to a large part of our organization, since it would enable us to create ETL processes for FTP-data. My colleague encounters the following problem:FTP/FTPS/SFTP transfer error: Timeout was reached The settings he uses are:URL: <url links correctly to a .csv on the FTP server starting with FTPS://FTP>Transfer type: Download to a File Target File: <.csv on OneDrive that does not exist yet>File Path Attribute: _downloaded_file_path Connection Mode: Passive / Active triedTransfer Mode: BinaryVerify SSL Certificates (ftps/ftpes only): Yes Authentication Type: Password < username and password correctly set> Proxy Type: Use FME Network Proxy Settings My colleague received the following error:FTP/FTPS/SFTP transfer error: FTP: command PORT failed We guess that this is a question for our FME functional manager, but any thoughts on how to solve this probl
Good morning, I'm currently facing a really weird issue. In my workbench I have a Reader that is loading a quite large Excel file, containing multiple tabs. I decided to remove all the unwanted data from the Excel sheet and keep only the tab I really need. When I try to update the Reader feature types the Excel tabs that I just removed keep showing! I have tried adding this Excel sheet to a new workbench; same result. What I've tried so far: - Adding the Reader to a new workbench in 2018.1 and 2019 beta. - Purged temporary files in FME. - Restarting FME / computer - Ran Windows disk cleanupNothing helped, I'm still seeing the tabs I deleted. Cheers, Ed
Hi All, I’m struggling to explode/conver the fme_aggregate to fme_polgon. I have already used 2DForcer and GeometryCoercer already, settings in the screenshot below:This comes from a DWG file and I need to create the outline boundary as AOI,Any ideas?
I am using ArcGIS Desktop 10.8.1, ArcGIS Pro 2.5 and it is working well in FME 2023+But in FME2024 version I have problem with reading and writing geodatabase, I get the following error message:“Unable to perform any licensing related tasks. Please check your Esri ArcGIS Compatibility workspace parameter, found in Navigator -> Workspace Parameters -> Translation. This 64-bit version of FME requires ArcGIS Desktop along with 64-bit Background Geoprocessing; ArcGIS Pro; or ArcGIS Enterprise. Refer to this this article http://fme.ly/qua for more details”“Failed to obtain any schemas from reader 'GEODATABASE_FILE' from 1 datasets. This may be due to invalid datasets or format accessibility issues due to licensing, dependencies, or module loading. See logfile for more information”“Workspace generation failed. Parameter file was C:\Users\Acer\AppData\Local\Temp\FME_1709721529788_18364”I guess the FME2024 version has problems with ESRI licensing or maybe a change in licensing methods.A
I am trying to do incremental updates from a secondary database (ArcSDE Geodb) because the entire original dataset is > 400k rows and times out or chokes FME. But I have tried about seven different ways and cannot get the WHERE clause to do anything meaningful (except 400 fatal error, which is maybe meaningful but not useful). If it does not fail fatally, it just brings the entire dataset in, or crashes at around 325k. The current attempt is DATEDIFF(day,CURRENT_TIMESTAMP,[Updated_On]) < 2 - I also tried various permutations of [Updated On] > DATEDIFF(day,-1,@GetDate) but that seems pretty clearly to not work for ESRI data (there are so many years of FME advice online, searching is a bit hazadous). [Updated_On] > DATEADD(day,-1,CURRENT_TIMESTAMP) worked using SSMS to hit the secondary database directly, but seems to do nothing in FME... I do not have any control over the source database format or view - but can confirm that [Updated On] is ‘datetime2(7)’ not string. In Da
MongoDB Reader creates Geometry Collections if the mongo collection contains more than one geometry field. Is there a way to choose which one of these fields should be treated as a geometry?
Hi everyone. I am reading data from an Esri Geodatabase. I have an issue with the Shape area reading as negative values. I have tried the geometry validator with Area Orientation set to “right” as with “Allow Left” and “Check doughnuts only” does not deliver any repair results. After the feometry validation, my repaired results still contain the negative area values. Negative values after geometry validatorIs there another way of solving this, I know the issue can be resolved in ArcMap wit geometry repair is there a similar way of doing this with FME?
I am attempting to convert a DXF/DWG file into the IMDF format. Following the instructions outlined in the first and second articles, I have successfully preprocessed and located the DXF file.However, I am encountering an issue at part 3 of the process. The Reader does not seem to be reading any features as expected.The tutorial I'm following can be found here. One thing, I updated the reader format from Autodesk AutoCAD RealDWG DWG/DXF to Autodesk AutoCAD DWG/DXF.Because I could not load module '/Library/FME/2023.2/plugins/realdwg/realdwg.so' .But i do not think that causes the problem.I would greatly appreciate any assistance in troubleshooting this problem. Thank you.
Hi. I need to know how can I get data between two characters.I have an attribute like this:10 12 Street (Main) CamnoreFirstly I need to get data between two characters ( ) and Secondly I need to convert it into two columns with this result:10 12 Street Camnore10 Main CamnoreIs StringSearcher is helpful? if yes how can I get this result?I really appreciate your assistance
I am using the FME REST API v4 to try and retrieve all Workspace Apps (and Automation Apps and Gallery apps) but I only get http status code 404, although I see in Fme Flow that we have over 20 Workspace Apps. I can retrieve Schedules, Automations and Workspaces just fine. Is it a bug? Result:HTTP/1.1 404 NO_RESULT {"message":"FME Server Workspace App automations does not exist."} 404 This works:$(FME_SERVER_URL)/fmerest/v3/schedules?limit=-1&offset=-1This doesn’t work:$(FME_SERVER_URL)/fmerest/v3/apps/automations?limit=1000&offset=0
Hi,I'm attempting to filter the records in the reader using the WHERE Clause, but I'm having trouble making it work. I'm trying to select all entries created after 20230718. I've tried the following equations, but it failed."created_date">20230718000000.000000"created_date" LIKE '20231718%' Error ArcGIS Portal Feature Service Reader: Encountered an unexpected error. The error code from the server was '500' and the message was: 'Invalid data type for expression ["created_date">20230718000000.000000].'. Details: '' ArcGIS Portal Feature Service Reader: Encountered an unexpected error. The error code from the server was '500' and the message was: 'Invalid data type for expression ["created_date" LIKE '20230718%'].'. Details: ''
Hi All,I’m new to process LiDAR data using FME and just received a bunch of LAS files that need to process as DEM (output). I was doing some reading and users sue the _classifciation to classify the data but have just realised that my data isn’t classify as my classification filed is 0? Screenshot below: I’ve also tried to use the PointCloudLASClassifier but the classification is 0 so doesn’t work. Any trick to classify the data and be able to export the DEM?
Message: ArcGIS Online Feature Service Writer: 'addResults' error for a feature in 'LAYER NAME'. The error code from the server was '1000' and the message was: 'Arithmetic overflow error converting expression to data type nvarchar.
hello ,i have detected self intersection with geometry validator does it possible to convert the line that detected to straight line ? I do not need the triangle at image (I have detected the issue location with geometry validator ,but how could I remove the part unwanted that occurs it ) .thanks in advance Fme 2021
I am working with a workbench that was authored close to 15 years ago. We had to change out the SDE Readers that are pulling in our GIS features. Since changing the reader the text be written to the DGN is coming out in different sizes. I have checked the output in the inspector and although the text is visually different the idgs_text_size attribute is consistent. I have tried to override this with the DGN styler but there is something else going on. Any ideas on how to either assess or fix this would be greatly appreciated.
Dear,I have a folder with 2208 xml files, approx. 30 GB. I read these files using a File Path Reader followed by an XML reader. From these files I select some features and save them locally. However, with my 16 GB RAM I run into memory issues. I already reduced the amount of data as much as possible within my workbench but still its not enough.Is there a method I read first 10 xml files – process them – save the outcome and go to the next 10 files, like a loop? My second question is whether it will safe memory in case I use a fanout expression or just write one final dataset with the results.My version is: FME 2023.2.1
Hello,I have encountered a problem, and I hope someone can provide guidance. I need to extract segments of a gable roof. I have attached a picture that illustrates my input data, which consists of polygons indicating the gable roof among other roofs in a digital orthophoto.I have utilized the CenterLineReplacer tool, which yielded satisfactory outcomes in most cases. However, I am experiencing difficulties in situations depicted in the image below (the line should be in a different direction): Does anyone have any ideas on how to solve this?Thank you in advance for your assistance.
Hi FME people,When creating a reader or writer, our typical procedure involves selecting an SDE database connection, such as SDE.EDIT.R. Following this selection, we establish a link to a deployment parameter, assigning it a local value. In this scenario, the local value corresponds to the database connection SDE.EDIT.R. This works well, except when we try ‘view source data’ or ‘view written data. The connection is set to the deployment parameter but the local value is not used instead.Anyone who is experiencing the same thing? We use:FME Form: 2023.2.2FME Flow: 2023.2.2Thanks a lot,Francis
Hi all, I seem to be getting a strange error when trying to attach an image from a feature layer in AGOL to Cityworks using the HTTP caller. My goal is to download and upload each unique image from the AGOL feature layer and attach it to the correct Work Order (i.e. picture of stop sign for sign work order, picture of a pothole for a pothole worker). My workflows is as follows, read in the AGOL feature layer, save each image using the AttributeFileWritter, create the work order, attach the image using the HTTP Caller. The error occurs when I try to upload the image will using a variable name in the output PATH + .jpeg extension. When I explicitly set the image in the file upload, it runs no problem. However when I use the variable name in the path I get a “ HTTPCaller_2 (HTTPFactory): [11]: HTTP transfer error: HTTP/FTP Transfer: Failed to open file 'C:\Path\To\Documents\WorkOrder_Pictures\.jpeg' for resource upload”. Here is my Cityworks API callI write the image to a folder using a
Hello everyoneI hope someone can help me. I try to convert an INTERLIS (Switzerland specific dataformat) into a File-Geodatabase and got always the same error message: Could not create the File Geodatabase 'C:\Users\Documents\IBI INTERLIS\Test\ibi_saxeten.gdb'. A possible reason is that you may not have permission to create the Geodatabase. The error number from ArcObjects is: '-2147220902'. The error message from ArcObjects is: {} Could anyone explain to me what this error message means, because I have literally no idea what causes this problem. Thanks a lot and have a nice week :)