- - Community Resources
Ideas
Product Updates
Events
Blog

Home
Forums
FME Form
Data

Data

Using readers and writers to extract and load data.

6,449 Posts
23,854 Replies

When you subscribe we will email you when there is a new topic in this category

6449 Posts

Recently active

Most replies Most views Newest first

abnan0001Contributor

asked in Data

Excel to GDB with Domains, Fieldnames and Field Alias

Hi Everyone..I was able to find a workspace from community this template attached and did few changes to it and it is working as expected with sample template provided in workspace to create a GDB with domains , fieldnames and field alias. 1) This workspace consider single default words in domainlist provided in excel only like Domain1 - Attached Sample_1_Workspace template - works with 2023 version.This workspace consider single altered words domainlist like alphabets(A,B,C….) in Attached Sample_2_Workspace_template - works with 2023 version This workspace consider 2 words in domain list like FME form in attached sample_3workspace_template -doesnt work and workspace fails.. If there are any special characters or space it doesnt work like : FME Form or FME_Form. I have attached error log and all sample data files with with each workspace.. Any Experts Kindly support : How can i resolve this with text as FME Form or FME_Form (with a space or special characters. Thanks

5 hours ago

pearly1Contributor

asked in Data

Correctly reading table in CAD

I have an asbuilt with a table that has all the locations/names of the structures. The table layer name is “text” and when i bring in the drawing, the text layer is empty. Im sure its how i have my parameters set up so a little guidance on how to read this would be great.

6 hours ago

kersyContributor

asked in Data

Value of attribute could not be converted to type 'timestamptz'.

I am trying to upload ifc data through PostGIS Writer and while doing so, I want to generate and store the time of upload as well. However, it gives the following error: “Value of attribute 'begin_lifespan_version' could not be converted to type 'timestamptz'. Feature will be logged and skipped.” can someone help? I am a beginner in FME as well.

6 hours ago

sar_coppContributor

asked in Data

Transposing Using the postcode and question number as a pair, possibly a list?

Morning, I am trying to transpose some data from a survey, I want to sort of Pair up the Postcode and the Question Number, then have columns of Responses with counts. Ive attached images and a fme workbench of the original data and the output I want. I have 36 questions, so want to know if there is a way to do this using a list? So I dont have to repeat this 36 times, I am not understanding how to do lists!I want to output the postcode as a column, then have a question number column. So each postcode will be repeated depending on the number of questions e.g.Question No PostcodeQuestion 1 M19 4NYQuestion 2 M19 4NYQuestion 1 M4 7ATQuestion 2 M4 7AT Then I want the next columns to be counts of the answersQuestion No Postcode Strongly Agree Agree Neither Disagree Strongly DisagreeQuestion 1 M19 4NY 2 3 1 0 2Question 2 M19 4NY 5

12 hours ago

txalapartaParticipant

asked in Data

MongoDB writer with nested documents

I am working with MongoDB trying to upsert nested MongoDB documents. The JSON format would be something like:{“name”: “XXX”,“tables” : [ {“name”: “XXXX”,“num_rows” : 3423,“fields”: [ {“name”: “XXX”, “type”: “yyyy”} , …. ]}, ….] I have no problems when I use a JSON string attribute inserting a new document with Document Source”: Json Attribute” and Insert operation.But, I do need to UPDATE existing features.In this case, the nested dcouments (tables in the json above) are created as plain strings. Which is a real inconvenience in my use-case.{“name”: “XXX”,“tables” : [ “{“name”: “XXXX”,“num_rows” : 3423,“fields”: [ {“name”: “XXX”, “type”: “yyyy”} , …. ]} “ , ….]Valid datatypes do not currently include arrays (of objects) Is there any alternative or workarounds?

14 hours ago

afodContributor

asked in Data

Cant access SDE features that are in a 'FeatureDataset' after upgrade to 2023 FME

Hello, we have a bunch of workspaces reading in from an SDE, all is fine in 2022 FME but after the upgrade to 2023 we can no longer read features that sit inside a feature dataset using the SDE reader. Features outside a feature dataset are fine. We have just discovered that a FeatureReader transformer will read them so maybe this is a bug in 2023?

1 day ago

aaronContributor

asked in Data

How to write attachments to ArcGIS Online?

I need to copy records from one hosted feature layer in ArcGIS Online to another hosted feature layer. Schemas are identical. Some of the records in the source data have an attachment (only 1 per feature/record so only in the 0 position in the list). No problem getting all of the records to write from the source to the destination but the attachments are not included. Anyone know how to do bring over the attachments with the rest of the data?

1 day ago

scottshenbergerContributor

asked in Data

Format error when using ArcGisOnline

I have an error message: ArcGIS Online Feature Service Writer: Encountered an unexpected error. The error code from the server was '400' and the message was: 'Cannot perform operation. Invalid operation parameters.'. Details: ''deletes' parameter is invalid, Input string was not in a correct format.' ARCGISONLINEFEATURES writer: An error has occurred. Check the logfile above for detailsI have checked the log file but honestly I am not sure what I am looking for. Attached is the logfile and my AGOL service format.Thanks

1 day ago

mariasContributor

asked in Data

What can be usedsimilar to FeatureReader transformer to read schema and data?

I need to extract TAB or Shape file from S3 and read data from it including schema.I have discovered in this articlehttps://community.safe.com/data-7/how-to-extract-the-actual-shapefile-from-featurereader-geometry-and-attributes-14548?tid=14548&fid=7that FeatureReader apparently can’t read all attributes that are in the file, until I expose them. But what is I don’t know the list of all attributes? I don’t know what exactly will be uploaded by other people to S3 bucket, I just need to read the file. Could you help, please, to identify the correct settings of FeatureReader or another way to extract data?

2 days ago

asablombergContributor

asked in Data

GeoPackage, Failed to add data - no such table main.GDB_Items

I am writing a geopackage file in FME but when I try to use it in ArcGIS Pro I get this error message. Any idea what could be wrong?“Failed to add data ...main.GNG.anlaggningsomrade. Invalid parameter value passed to function[no such table: main.GDB_Items}”FME Workbench 2022.2.2ArcGIS Pro 3.2

2 days ago

liseParticipant

asked in Data

benichollsContributor

asked in Data

Parse JSON from Bentley OpenGround

I would like to be able to parse a response from the Bentley OpenGround REST API, and am getting somewhere but not far enough!I can use a mixture of attribute exposer and JSONFragmenter to get some information out, but it never quite looks like the table i want. i will be doing multiple queries over the coming months into this api, and will have a different number of columns each time, so cant parse through each data record based on the number of columns i want in that particular query! I can get to the point where i have a feature for every feature in the database, and then i have a _response_body which i need to parse further. i can keep parsing it through jsonfragmenter, and i end up with a jumbled mess where columns are names wrong and the colunn information i want is in the wrong part of the table. it is a simple structure, such as “DataFields” “heade” for column name and “value” for data itself. i will add some screenshots to aid in query. any help would be greatly appreciated!Th

3 days ago

lanceContributor

asked in Data

XML Files - Bang head on desk here

Starting soon I will receive about 150 XML files each with a different schema but the element name is include in the file name. I have many ways to use a single FME workspace to process these files dynamically but have been unsuccessful. I figured using a Directory File reader and passing the file names to FeatureReader setup as XML then saving to a Dynamic Writer to a database. No joy. What am I missing for something so simple.

3 days ago

mamaieron

asked in Data

AppearanceSetter changing GeometryTraits

Hello everyone. I am working with citygml modeling and I am facing some problems with appearanceSetter. I have a workspace similar to the con terra that I used as a base.Originally everything is fine but when I put an appearance setter before the roofSurface writer (ground or wall too). The geometry traits previously set are automatically modified by the FME. In the image below is the original output.And in this the output with the appearanceSetter.The geometry trait created for polygons was moved to linearRing and a new one was created for polygons.The file is normally opened by the Data Inspector, however, when running Val3dity to perform the validation, the xlink is broken, as it must point to a polygon. EDITED: I forgot to mention that to reproduce the situation above i placed a appearanceRemover just after the reader to remove the original appearance of input data.Does anyone know how to fix it? Thank you in advance for your help. Mcdonnell

4 days ago

helmoetzContributor

asked in Data

Postgis varchar() interpreted by FME as varchar(80)

Hi, I have this dynamic setup where a postgres FeatureReader reads the schema and the results are written to another database using this schema information to create the new tables. However, a table having colums of type unqualified varchar will be regenerated with varchar(80) and if records are loaded having length >80 the workspace will crash, saying: Error was 'ERROR: value too long for type character varying(80)I attached the workspace. Is this a bug in FME or am I doing something unbelieveable silly?😕 I'm using FME(R) 2020.2.3.0 (20210129 - Build 20820 - WIN64) on a postgres 12.9 (Ubuntu 12.9-1.pgdg18.04+1).

5 days ago

mradamlenz

asked in Data

Projecting Geotiff Raster to Maps - 101

Hi All,I'm new to FME and have been pouring over videos to try and do my first projection. Any help is much appreciated. I have a raster geotiff I'm trying to project onto a City map. I'm hoping someone can point me in the right direction with a start to finish video or thread to help guide me through the process.Thanks,Adam

6 days ago

palhermansContributor

asked in Data

FME2024.0 "Esri ArcGIS Layer" Reader failes when LYRX from ArcGIS Pro includes Group Layers

Hi. reading LYRX (“layer files”) from ArcGIS Pro failes when the layer file include Grouped Layers. It works just fine if the LYRX file does not include Group Layers.Thea Layer File/LYRX is made with ArcGIS Pro 3.2.2 where the LYRX file is in JSON format.

6 days ago

pauldContributor

asked in Data

Looping through string to remove known text

Hi.I have another string replacer question.I have a file of text strings (appellation) similar to LOT 1/2 DP 17528 LOT 1/2 DP 13552 RS 16932 18812 19986/7 PT 18813 20862 BLK IV KAPUNATIKI SDI have split the LOT 1/2 DP 17528 & LOT 1/2 DP 13552 into 4 separate rowsTemp, _result, appellationLOT 1/2 DP 17528, LOT 1 DP 17528, appellationLOT 1/2 DP 17528, LOT 2 DP 17528, appellationLOT 1/2 DP 13552, LOT 1 DP 13552, appellationLOT 1/2 DP 13552, LOT 2 DP 13552, appellation What I want to be able to do is remove the temp part from appellation The resultant appellation in this example would be: RS 16932 18812 19986/7 PT 18813 20862 BLK IV KAPUNATIKI SDI have tried doing this but only ended up removing one part from each instead of both Temp parts. I would be very grateful for any help with this.Regards, Paul

7 days ago

gisbradoklaContributor

asked in Data

dynamic crs to separate data stream

I am reading datasets and can extract the crs (stateplane) from them. but I need to use a spatialrelator for data that is in WGS. I need to be able to determine incoming crs and set a reprojector on the wgs data right before the spatialrelator. In the past i have only been able to accomplish this by stringing together more than 1 workbench.Any ideas on how to get this done in 1?

7 days ago

bayramContributor

asked in Data

CITYGML EXPORT

Hello,I created a model and I want to export it as citygml, but the model does not appear in the FZK gml viewer. How can I solve this problem? Am I making a mistake in the writer? I added my data and FME WB. Could you please check it and share what I am missing?

7 days ago

alfonsContributor

asked in Data

change coordinate format DM -> DD

Hello, I'm sure it's a pretty simple question, but somehow I can't see the wood for the trees to find the answer.I have a list (.xls) of coordinates in the form <name>, <x_coord>, <y_coord>. The coordinates are, for example, in the form N 51° 06.796 E 3° 14.545.Now I want to save the information in an FGDB and would have expected the VertexCreator to help me with this. Unfortunately, all my values are completely rejected. My further attempts - removing all spaces; using the HUB Transformer "DegreeFormatter" or Decimal Degrees Calculator all failed. For this reason, how do I manage to get the coordinate format into a form so that the VertexCreator accepts it? Thank you very much.

7 days ago

fdh12316

asked in Data

How can i change the unit of an IFC file to meter

Hi there,I have a question about the unit of coordinate in the iFC file, which i wanna clippe with another solid. The coordinates are 1000 times bigger, and the clipper could thus not find overlaps. How can I adjust the coordinate value: using the expression evaluator, and divide all the coordinates with the value 1000? if tho,How can I access all my coordinates because I could not find an attribute that enables that? I also tried the coordinate extractor but it is only another copy of the coordinate values, which doesn't update the geometry coordinate value in the IFC . If anything i understand it wrong pls point it out! thanks in advanceFang

7 days ago

xingfu07Contributor

asked in Data

Adding in a custom coordinate system

Hi All, I’ve tried to add in a custom coordinate system and it just doesn’t work. I’ve restarted my computer twice and the co ordinate system won’t come up in the csmap reprojector. I would love to use the online webform that has been created, but I am unsure at which username and password is required to log-in. My usual FME doesn’t work. I’ve created a file in the FME folder in documents and edited both coordinate files in the reprojector folder. This is what I have written below:# COORDINATE_SYSTEM_DEF PerthCoastalGrid1994 \\# PROJ TM \\ # UNIT METER \\# SOURCE "spatialreference.org" \\ # DT_NAME GDA94 \\# EL_NAME GRS1980 \\ # DESC_NM "Perth Coastal Grid 1994" \\ # GROUP AUSNZ \\ # PARM1 115.8166666666667 \\ # ORG_LAT 0 \\

8 days ago

Page 1 / 258

Badge winners

salgardihas earned the badge Teacher (Grey)
chlaw2001has earned the badge Student (Grey)
delunamahas earned the badge Teacher (Grey)
crystalatsafehas earned the badge Student (Grey)
tbsm223has earned the badge Teacher (Grey)

Show all badges

28,770: Posts

Terms & Conditions Cookie settings

Sign up

Already have an account? Login

An FME Account is required to contribute

Login to the community

An FME Account is required to contribute

Enter your username or e-mail address. We'll send you an e-mail with instructions to reset your password.

Username or e-mail

Back to overview

Scanning file for viruses.

Sorry, we're still checking this file's contents to make sure it's safe to download. Please try again in a few minutes.

This file cannot be downloaded

Sorry, our virus scanner detected that this file isn't safe to download.

A product of

Company Products Giving Back Careers Contact Feedback

Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.