Skip to main content
  • EN

Data

Using readers and writers to extract and load data.

  • 7,092 Posts
  • 26,104 Replies
7092 Posts
asablomberg
Contributor
asablombergContributor
asked in Data
Alter dynamic attributes

I am reading dynamic data and sometimes there is text fields that contain newline characters and I would like to replace them with just a space. How would I go about this?   

594
asablomberg
Contributor
arnovananrooij
Contributor
arnovananrooijContributor
asked in Data
PostGIS polygon turns into Geopackage multisurface in dynamic mode

Hi all,I see some strange behaviour when converting a simple 2D box from PostGIS to Geopackage in a dynamic mode (using schema). I have attached a simple workspace which demonstrates the strange behaviour. A simple box is created in 2D and writen to PostGIS as a postgis_polygon. I ckecked the table in the database and is has the correct definition for  the geometry (geometry(Polygon, 28992)). After the creation the table is read wit a FeatureReader of type PostGIS with the same connection parameters. It is set to read the schema and the features. The result is directly written to a geopackage with a FeatureWriter. Geometry is from schema definition. When i open the result in the Data Inspector i see the geometry type has become geopackage_multisurface where i would expect it to be of type geopackage_polygon. At the moment we are upgrading our FME environment (Form and Flow) from FME 2021.2.6 to 2023.1.2 and ran into this problem. In FME 2021.2.6 i did not see this behaviour. This parti

1122
braggken
Supporter
marjan_nikoukar
Contributor
marjan_nikoukarContributor
asked in Data
Convert annotation class to polyline or point in FME

Hi,I have a annotation class in a file geodatabase. I want to convert it to a polyline or point in a file geodatabase. I could not fine any transformers in FME.  I really appreciate your assistance.

851
evieatsafe
Safer
adrian_farrell
Contributor
adrian_farrellContributor
asked in Data
Paged WFS Responses - OS Features

Hi I have a requirement to connect to the new Ordnance Survey (GB) WFS service in FME, their system responds with 100 records at one time, there doesn't appear to be an option in the reader to facilitate this paging implementation as in other products Any help to be able to access this API would be greatly appreciated

574
nielsgerrits
VIP
gisbradokla
Enthusiast
gisbradoklaEnthusiast
asked in Data
use listElementCounter to populate destination attributeValue

I have a list of Attributees{}.name That contains attributes for each feature. the element count is variable.I am also using the ListElementExtractor and like how it works but i need from  15-258 attributes to put the list into.is there a way to get the list values out as attributes? or at the very least mmake the list populate (i.e. attr1, attr2, - attr258) in the destination attributes? I am reading the schema from a (blank) gdb with coded domains and WAY too many attributes to use AttributeManager. the purpose is to get the attributes with the correct domain and values so a different schema can be fed into this format and output as gdb with domains intact.

291
geomancer
Evangelist
lcaruthers
Contributor
lcaruthersContributor
asked in Data
File GDB writer not outputting features

Hi Everyone, I am working on a project for school and am reading in CAD data and outputting it as GIS data. However, the File Geodatabase writer is not outputting any features. I am setting it up how I’ve always done it in the past, but it is not creating the GDB where specified as the translation log says no features are being written. I have no clue on why this is happening nor how to fix it, so any help would be appreciated. Thank you in advance.

511
j.botterill
Influencer
charisph
Contributor
charisphContributor
asked in Data
Modify / add additional file extensions on an existing reader

Hi all, I would like to use an existing FME reader to open a file with an unsupported extension. Is there a workaround to achieve this? Alternatively, how can I clone an existing reader and modify it to support the desired file extension? Thanks a lot! 🤓hp

726
charisph
Contributor
fme_hans
Contributor
fme_hansContributor
asked in Data
Divide planes by area sum

Hi,I have a layer with closed polygons. On top of these polygons lies a layer with smaller areas. These areas are divided into 3 types.Each type of area needs to be assigned a year, and the total area per table type should be approximately equal within the larger area.How can I create a distribution where each type of area is used, and the total area of each type is approximately equal?How could I achieve this with FME?

542
fme_hans
Contributor
kibailey
Contributor
kibaileyContributor
asked in Data
Excel Writer: Error Message is 'col must be in range 0- 16383'

I’m encountering this error and am at a loss. Everything in workbench (SQLCreator - Attribute Mgr - Excel Writer) seems to work fine, up until it gets to the writer. Excel Writer: Failed to write an attribute of type 'string' with value 'XMLDATA' to row '0' column '-1015942868'. Error message is 'col must be in range 0-16383'. Skipping featureWriter Mode: UpdateOutput Field Names: YesFormal Field Names: NoUse Attribute Names as Column Positions: YesStart Position: Start Column: 1 Start Row: 1  Any suggestions?  

361
liamfez
Influencer
P
prosufi95
asked in Data
how to read ssf data format in fme

1218
evieatsafe
Safer
lrs
Contributor
lrs
asked in Data
Writing ArcGIS Geodatabase attachments - origin row error

I am working with a script that transfers data from one SDE to another SDE. Currently, the script does not ensure that the relationship between a feature and its attachment remains intact. To address this, I extended the workspace based on the guidelines in this article: Writing ArcGIS Geodatabase Attachments.This solution worked fine when transferring data from one GDB to another GDB using the same data model. However, when we run the script from one SDE to another SDE with the same data model and data, it fails and I encounter the following error messages: Failed to retrieve origin row for a relationship feature with OBJECTID '3' from table/feature class 'BAGGERWERK_WATERLOPEN' GeodatabaseRelationshipFeaturesPipeline::GeodatabaseRelationshipFeatures (SortFactory): A fatal error has occurred. Could you provide guidance on how to resolve this issue?

603
allasimon
Participant
sharvikhawte
Contributor
sharvikhawteContributor
asked in Data
3D voxel for geological model

I have 4/5 layers of rasters representing the geological horizons. I want to visualize the layers in 3D voxel. what the result look something  like this. Is there a way to get this with fme? 

802
j.botterill
Influencer
anne_kathr1ne
Contributor
anne_kathr1neContributor
asked in Data
Writing features to ArcGIS Online

Hi, Im trying to write approx 17.000 rows into an existing feature service in AGOL. I get the error "entity too large for url", so I assume its too many rows. I have tested with a few rows, and that works fine.  I have tried now to use the ModuloCounter and a TestFilter to split into smaller groups, and then add multiple AGOL-writers, so they only write one group at the time. Im not sure this is the best way to go, as I would the need a lot of writers. Another is to use the FeatureWriter to do the same, but I would still need to do the setup for all the groups, so not sure if thats ideal.  Does anyone have any better suggestion? Best,Anne Kathrine

7557
jaywood
Participant
zachmsc
Contributor
zachmscContributor
asked in Data
Writing Data to SQL Spatial Database Results in Successful Translation With No Actual Data Written/Updated

I have been having a consistent problem in FME for quite a while now, across both 2023 and current 2024 versions. I have a workspace set up which has a SQL Spatial reader, a few transformers, then a writer back to the database of origin.Any time I run this database, it goes through and successfully runs it within FME. However, as soon as I go to another source and read the database table(s) involved, such as SQL Server Management Studio or QGIS, I find that the data is still the exact same as it was originally, like FME had done absolutely nothing. Likewise, if I make a new table, it will simply make an empty table with only the columns/fields intact, with no attributes at all. I have no problems editing data from this database in QGIS or SQL Server Management Studio and have even had colleagues run the exact same FME Workspace with no such problems. I have also had two different FME installations on two different devices behave exactly like this.I'm pretty much out of ideas here. Has

1544
zachmsc
Contributor
markrmunoz
Contributor
markrmunozContributor
asked in Data
Invalid Pointer error Writing to MS Access

Hi I'm trying to update attributes in a copy of an MS Access database. The workbench fails at the writer due to a provider error '(-2147467261) invalid pointer. Any ideas as to what the issue may be?

693
crystalatsafe
Safer
cnorth
Contributor
cnorthContributor
asked in Data
Insert Salesforce issue with related record (Sharing record)

Hi, Currently have a process that recieves a list of record id’s from Salesforce using HTTP callouts, the record ID include the ID of records from a custom object and the id of a number of users, we are trying to create an insert in the customobject_share to create records to give the list of userId access to the records from out custom object.  We currently are getting an error,Salesforce Writer: Record 1 was not processed successfully: 'FIELD_INTEGRITY_EXCEPTION:invalid parent id: Parent ID:ParentId --' The parent id is the record Id of the salesforce record we are creating a related share record against. the record exist in salesforce and in testing we have event tried hardcoding te ID of both records, and user Id with RowCause = “Manual” and Access Level = “Edit”. The form is running as using the Salesforce reader/writer with a System Admins login details? Thanks  

191
hkingsbury
Celebrity
sharvikhawte
Contributor
sharvikhawteContributor
asked in Data
Uploading raster (tif) in Arcgis online using FME

I am trying to upload a tif stored in my PC locally to the contents in arcgis online. I can upload the shapefiles efficiently but not tifs. Can someone advise? Thanks!

1114
AliAtSafe
Safer
nanchuangyeyu
Contributor
nanchuangyeyuContributor
asked in Data
How to create a Esri Personal Geodatabase whose structure is specified in an Excel table? ?

Hi everyone, I want to create a Esri Personal Geodatabase whose structure is specified in an Excel table. Is that possible? How can I achieve that?

744
nanchuangyeyu
Contributor
carmijo
Supporter
carmijoSupporter
asked in Data
Reading ArcGIS Online Feature Service shared with the organization - not owned by the user or shared with a group.

Using an Esri ArcGIS Online connection and the Esri ArcGIS Online (AGOL) Feature Service reader, I'm unable to read a feature service that has been shared to our organization.When looking at the select form, i’m only provided my user folder and a groups folder. The feature service i’d like to read isn’t owned by me or shared with a group. Is this as-designed or is there another way to read it? Thank you  

1542
todd_davis
Influencer
marjan_nikoukar
Contributor
marjan_nikoukarContributor
asked in Data
How to update a dataset using a change table in FME

Hi everyone,I have a feature class in a File Geodatabase that I need to update. I have two sources for these updates: a CSV file with codes of the changed features, which I should use to remove those features from the main dataset, and separate shapefiles for each changed feature that needs to be added back to the main dataset. (I shouldn’t delete and update the whole dataset, I should update the changed features only.)What is the best way to approach this in FME?Thank u in advance for your time. 

1195
jobvdnoort
Supporter
tono
Supporter
tonoSupporter
asked in Data
Destination Feature Type Routing Correlator

Hi,When running a workspace with cache mode activated i can see in the Translation log that fme form writes ffs, and writes and index and so starts it with a process i am not familiar with: “Destination Feature Type Routing Correlator” and it takes so long to go through.What is this process about? is it possible to avoid?Thanks in advance!

521
redgeographics
Celebrity
gboquin
Contributor
gboquinContributor
asked in Data
SQL Server failing in SQL Executor in FME Form

Hi, I have a SQL Executor calling a stored procedure.  I am running FME Form FME(R) 2023.2.1.0 (20240104 - Build 23774 - WIN64) and am getting this issue:Microsoft SQL Server Non-Spatial Reader: Failed to connect using Microsoft OLE DB Driver 19 for SQL Server. Provider error '(-2146824582) Provider cannot be found. It may not be properly installed.'The transformer is not failing, but its not producing a result either. Not sure how to move forward with this one. Any thoughts?Thanks 

992
gboquin
Contributor
marjan_nikoukar
Contributor
marjan_nikoukarContributor
asked in Data
Split attributes and write them as feature classes in a new file geodatabase

Hi, I used an AttributeFilter transformer to split attributes. I want to write each of them as a feature class in a new file geodatabase. What is the best way to do that?Thank you for your assistance.

551
jobvdnoort
Supporter
A
ashhart
asked in Data
Writing to existing SQL Server table

Hello. I have a workbench that reads an XML and splits it into attributes with their values (with one spatial attribute). I am trying to write this data to an existing, empty SQL Server table. If I set the writer to create a new table, it runs fine, but I can't seem to get it to populate my existing table. The workspace runs, and passes with warnings, but when I go to my SQL Server table it is still empty. Any ideas?  My writer is configured as follows: Feature operation: Insert Table Handling: Use existing  The translation log claims that all of my features have been written, and the data inspector shows them fine, but it just won't populate my table.  Thanks! 

4846
bo
Contributor
kanhan1
Contributor
kanhan1Contributor
asked in Data
Dynamic update user param

Hi,I want to fill the User Param Choice with data from a table in MS SQL. This works fine with the 'Import->From dataset...' option. but this function only populates data statically. Is there any way to 'hack' this to fetch data dynamically? The idea is that this Workspace should work in a App in FME Flow.  

2762
kanhan1
Contributor

Badge Winners

Show all badges

Community Stats

31,810
Posts
120,825
Replies
39,437
Members

Latest FME

Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.