Using readers and writers to extract and load data.
Recently active
Hi! I'm trying to import some CityGML to Infraworks via the quick translator. I'm getting errors that seem schema related, and I saw when I colleague used the full trial so that he could see the workspace details, that the schema seems not to be defined. Is that possible? In the log file I have:2018-03-21 10:16:07| 1.4| 0.0|INFORM|mapping file Keyword: `CITYGML_1_ONLY_FEATURE_TYPES_IN_DATASET' occurs 1 time(s) 2018-03-21 10:16:07| 1.4| 0.0|WARN |UniversalReader -- readSchema resulted in 0 schema features being returned 2018-03-21 10:16:07| 1.4| 0.0|ERROR |The XML Module halted on error, see the logfile for details 2018-03-21 10:16:07| 1.4| 0.0|INFORM|Closing the SCHEMA Reader 2018-03-21 10:16:07| 1.4| 0.0|ERROR |The XML Module halted on error, see the logfile for details 2018-03-21 10:16:07| 1.4| 0.0|INFORM|Translation FAILED with 4 error(s) and 54 warning(s) (0 feature(s) output) 2018-03-21 10:16:07| 1.4| 0.0|INFORM|FME Session Duration: 2.0 second
We used this article (https://docs.safe.com/fme/html/FME_Server_Documentation/Content/ReferenceManual/Directory-Account-Permissions.htm) to give permission to write files to the Data file in resources, which was successful to write to that parent folder. We then created subfolders in the Data folder through the web interface for different processes we have. The subfolders say that the owner is 'root' and therefore the fmeserver service account does not have permissions to write to that folder. Is there a way to make it so any subfolder will get the same permissions as the parent folder?
I need to use an FMW script to extract xyz files from a single netcdf file. When using the data inspector, I see that the netcdf has several variables and multiple bands of data for each variable. Ultimately I am desiring the u and v datasets. I feel like I am close with my FMW file, but somehow it is not extracting the u and v datasets correctly.My steps thus far are as follows:1. Open FTP directory: ftp://ftpprd.ncep.noaa.gov/pub/data/nccf/com/rtofs/prod/rtofs.YYYYMMDD/ (you need to adjust the date characters YYYYMMDD)2. scroll to bottom and open the rtofs_glo_uv_YYYYMMDD_hcasts.nc.gz file3. use gzip to uncompress the file: gzip -d rtofs_glo_uv_YYYYMMDD_hcasts.nc.gz4. Open FME Workbench 2015 (I only have 2015 desktop)5. Open FME script (attached here) 6. Provide input parameters:SourceDataset = rtofs_glo_uv_YYYYMMDD_hcasts.nc (netcdf file that was downloaded above) DestDataSet = C:\\The expected output is to have a u_velocity and a v_velocity xyz file. Each file will have the follo
Hi, I am writing to my database (SQL Server Spatial) and would like to in this case, overwrite the automatic sequential numbering system that SQL Server gives each attribute. Any idea how I would go about this?
I get this error while writing from SDE to PostGIS. The job runs forever and doesn't do anything.The specified coordinate exceeds the valid coordinate rangeIt has around 20K features. The features doesn't even read through inspector. But if I update the filter to have around 10K features, it does write.There is no issue with the features for sure. Just wondering if there is any limitation from SDE? I haven't seen this before and definitely it worked before for writing more than 100K features.Thanks
I can't figure out how to use a mongodb writer to update existing documents (specifically, I want to insert new elements in a list within the existing json structure) I don't even see any parameter to set the writer to update or upsert.
I have a workspace that is using a polygon drawn by the user to search multiple layers, write a html report and email the html report to the user. The html report is generated using the htmlreportgenerator, htmllayouter and writehtmlfile. The workspace is run in FME server.In the workspace, I'm saving the username, email and polygon to a SQL database to keep a record of what reports are run. However, I would like to be able to save the html report to the table as well so that it can be retrieved to see what results have been provided in the report. It would not be possible to rerun the report as the data changes and it is important to see the data results as it was generated for the user at that time. Is it possible to do this and if so what attribute type would you use to write the the html output to the SQL table?If this can't be done, does FME server save the output of the workspace somewhere? The HTML report would have the date and time appended to the report name so
Hi, I have about 50 feature classes in a geodatabase. In the feature classes, some of the field names are the same but a lot are different.For example: in one feature class field names are – Sample_ID, Easting_NAD83_UTMz19, Northing_NAD83_UTMz19, Au_INAA_ppb, Ag_INAA_ppm, Fe_ICPMS_ppm, Fe_INAA_ppm, Ca_ICPOES_ppm. In another feature class, field names are – Sample_ID, Easting_NAD83_UTMz19, Northing_NAD83_UTMz19, Ag_INAA_ppm, Au_ICPMS_ppb, Fe_INAA_ppm. I want to create a new feature class called “INAA_merged” where every field that contains “_INAA_” will be in it as well as the Sample_ID, Easting/Northing fields. I’d like all the other fields containing for ex: “_ICPMS_” and “_ICPOES_” to be deleted. So basically I want to merge all the fields together in one feature class that have the same name and delete all the others. Is this possible? Is there a way to set up conditions for merging – ie to keep certain fields if they contain certain characters and to delete others?
Hello, I am trying to read from a SDE feature class. when I try an add the reader I get the error "Error while retrieving layer info structure for layer 'shape' in table 'VEG.NONPRODUCTIVE_AREA_EVW'.:SDE_ERROR_CODE(-20) Layer not found" It the table is a a postgis table with versioning. I can see the table in my table list when connecting. Any one have any advise?
Hi all,I am doing my first steps with FME.I am trying to parametrize a CSV reader as follow:But parameter is replaced by its value:How to do this? Many thanks for your help!
Hello, I've geoereferenced a raster map using ArcGIS and exported it as GeoTIFF, however, I can't read it properly in FME data inspector, the image appears all black. Have anyone before faced the same problem ? Thank you.
Hi all,I am doing my first steps with FME.I am trying to read a MongoDB collection using the Mongo reader. To retreive the unique document identifier _id, I use the mongodb_id format attribute. Is it correct?Then when data are loaded, this attribute does not seem well encoded: Id (encoded: utf-8): ??{y?q?M????z?'How to fix this? Many thanks for your help!
Hi,i want to geocode addresses by my own ArcGIS Server geocoding service. The adresses are in a csv-format. In a second step I have transform the adresses from csv in to JSON. After that i want to get the geocoded adress via the HTTP-Caller because it is not possible to post adresses via the Geocoder-Transformer in the JSON-Format.It doesn`t helped me to post the address in a single LineString.Can anyone help me to geocode the adresses?
Dear All,I have an issue while using Dissolver and Aggregator I get multiarea of 5 polygons, which look merged in the Inspector. But when writing to PostGIS, aggregated geometry is split again into separate polygons. PostGIS is accepting only polygons as geometry. I know that if the PostGIS accepts only single polygon, but writer attempts to load a collection, then a collection will be split. The question is if there's a way to aggregate multiple polygons into one without collection. Single polygon needed only to set the extent of the layer, so envelope like feature will do as well.Thanks a lot for your answer.
Hi! I received an IFC file from a costumer and when I tried to inspect the file in Data Inspector, I received the attached error message. I don't know much about Revit so I'm at a lost. Any help to what this means, would be greatly appreciated! Thanks,David
Hi,When I try to open a workflow file that has a PostGIS reader or writer FME Workbench crashes immediately, without any dialogs or crash logs. This happens whether I open the file by double clicking, right click "Open with...", drag and drop the file or "File > Open".Similarly, if I try to open a PostGIS table in the data inspector it crashes too.I can open the database and its tables in PGAdmin so I know the database is up and running.I've just updated to the official release of FME Desktop 2018 on Windows 10 (I've previously been using the beta versions without issues). Thanks,Aiden
Hi I have a set of .asc files. I am looking to use FME to generate a point cloud then build a Terrain in an ESRI's dataset from the asc files. Is this something I can achieve using FME Desktop or do I have an ArcGIS 3D extension license.
Hello,I'm new in FME but have some basic knowledge. I am trying to fetch some data from excel file and use it for analysis and making graphical representations (charts) but I am stuck with the steps much prior to this analysis. Unfortunately, excel file is not very well organized nor structured so I'm trying to overcome this problems and I managed to read some amount of data in structured form.What I'm having problem with is this: excel is organized in many sheet: first (main) sheet has summary data from all the rest of sheets and one column is hyperlink to a specific sheet which holds the data in more detail. I would like that after the running of the workbench I have the ability to choose the specific sheet for analysis based on list of sheets I have already obtained from hyperlinks in the main sheet.I tried to figure this out for the whole day without success. I would appreciate any suggestions or pointers.Thank you in advance!
Feature writer transformer does not expose all writer parameters. I'm passing fme_db_operator to it with the value of update, but it tries to insert. In the past I have been better able to troubleshoot with bulk insert off (it's on by default), but I can't find it to turn it off. Is that because it's not exposed?
I am getting the following error,when trying to write all the NITF attributes to the PostGIS table.Bulk copy failed on table 'public.EGPL' using delimiter ':'. Error was 'ERROR: value too long for type character varying(4)CONTEXT: COPY EGPL, line 1, column nitf_fhdr: "NITF02.00"'A fatal error has occurred. Check the logfile above for details... Last line repeated 2 times ...FME Session Duration: 3.2 seconds. (CPU: 2.0s user, 0
Hi,I am getting the following error when I run a workbench in Desktop 2017Deatils of the error areProblem signature: Problem Event Name: BEX64 Application Name: fme.exe Application Version: 2017.7.26.17259 Application Timestamp: 58b68360 Fault Module Name: ucrtbase.DLL Fault Module Version: 10.0.10586.212 Fault Module Timestamp: 56fa10e8 Exception Offset: 00000000000698fe Exception Code: c0000409 Exception Data: 0000000000000007 OS Version: 6.3.9600.2.0.0.16.7 Locale ID: 3081 Additional Information 1: bef8 Additional Information 2: bef885c27b607af464c2616e642a10f8 Additional Information 3: fcaf Additional Information 4: fcaf8a62e8bcb47b1e98dbc1383ebc13 Read our privacy statement online: http://go.microsoft.com/fwlink/?linkid=280262 If the online privacy statement is not available, please read our privacy statement offline: C:\\Windows\\system32\\en-US\\erofflps.txtThe log file error message ischild killed: unknown signal The workbench is using workspa
We are performing an update to ArcGIS Online where the source data is from ESRI ArcGIS SDE database. This is the error that I encounter right now. I already tried to filter all fields using the NullAttributeMapper to replace Nulls with Empty String, but still I encounter this error. What is meant by this error? ArcGIS Online Feature Service Writer: 'addResults' error for a feature in 'Sewer Connection'. The error code from the server was '1000' and the message was: 'Data is Null. This method or property cannot be called on Null values.' ArcGIS Online Feature Service Writer: 3772 features successfully written to 'Sewer Connection', but the server rejected the 2 features in the last request due to errors. See warnings above. Aborting translation
When writing a table with oracle writer (type: spatial object) fme cannot create primairy key or index when the name of index is getting long. For example : CREATE INDEX "SPOT.ACCRES_KNOOP_OLVS_ID01" ON "SPOT.ACCRES_KNOOP_OLVS" ("ID") It gives a error: SQL Error: ORA-00942: table or view does not existHowever , the table does exist. Copying pasting the command from fme -logs to sqldeveloper gives the same error as in FME. However, if i shorten the indexame in command like e.g. ACCRES_KNOOP_OLVS_ID the command executes fine: index createdShould FME not fix the indexname by itself so it does not give error messages when making indexes ? Is this a bug ? Current FME version is Desktop 2017.1Best regards, Ed
Hello,I want to create a workflow that will monitor updates to tabular SQL Servers tables, which would send email notifications when my workbench has updated "error" tables. My workspace is effectively processing data and writing these error tables. But I am unsure of the best way to create database "listeners" that would note when new data has been added to an error table and then send out an email notification via FME Server. We have FME Server up and running and it does send email notification for when the workspace fails to run. But we have never sent email notifications for when a SQL Server table has been updated. Can anyone provide some suggestions on how to best do this with some links and possibly an example work bench that "Listens" for database updates? Thank you!
Details :I tried to add Dataset & Pathname reader followed by StringSearcher, FeatureReader and writer as format JPEG.I am getting desired output till feature reader , when i checked with logged but i am not getting files in output folder.