- - Community Resources
Ideas
Product Updates
Events
Blog

Home
Forums
FME Form
Data

Data

Using readers and writers to extract and load data.

6,780 Posts
24,934 Replies

When you subscribe we will email you when there is a new topic in this category

6780 Posts

Recently active

Most replies Most views Newest first

abdullah83Contributor

asked in Data

IfcGeographicElement features can’t see object in FME

Need Help I have been using FME Inspector to read an IFC file. Inside the IFC file, there are IfcGeographicElement features (DTM surface). My question is, why doesn't FME display the shape of the object?I noticed in the FME the number of vertices is zero, but when I opened it in Open IFC Viewer, I can see the IfcGeographicElement object, and there are vertex values 15515.Attached herewith are screenshots of both applications views.

163

1 year ago

abdullah83Contributor

asked in Data

Why geometry of geopackage change from 3d to 2d?

I am trying to map a geopackage file to another geopackage file (same format). Why is the geometry in the reader I input in 3D format, but the geometry produced by the writer is only in 2D format? Please help.

443

1 year ago

arthyContributor

asked in Data

ArcgridSnaper and "the number of points is less than required for feature" issue

Hello everyone,I have produced a dataset in oracle. I can't display those data in ArcMap because of the issue "the number of points is less than required for feature".After reading several posts, I now understand that the issue come from the esri grid (toleand and precision) which snap point and this snapping process when displaying data in esri create some geometric issues like self-intersections that were not in oracle.I thought of writing to a temporary geodatase. However, after importing the feature from the geodabase to oracle, I still face the same issue. Because, it is an esri issue, the geometry validator is not able to detect them. However, there used to be a transformer called ArcSDEGridSnapper which is now deprecated. How could I replicated the ARcGRidSnapper function in my workbench in order to identify and exclude (or repair) the features that will cause geometric issues in ESRI?

I have a dbf File I want to open into FME Desktop. I can read only the half of the records of the file which is bellow 2G.

I put the link of the data I try here which is open data from the Quebec Governement Rôles d'évaluation foncière du Québec - Rôles d'évaluation foncière du Québec géoréférencés 2013 - Données Québec (donneesquebec.ca) The problem table is B05V_UNITE_EVALN.dbf . When I open it in ArcGIS Pro, I can see 3000000 + records but in FME I see only 1500000.

153

1 year ago

kircContributor

asked in Data

LandXML Reader/Writer - version support

Hi all, here it says that the current supported version for the LandXML Reader/Writer is version 1.1 and that further support is planned for the future. The current version of LandXML is 1.2. Does anyone know anything about the support for version 1.2 or the future version 2.0? Thanks in advance in case you have any info, cheers!

165

1 year ago

abdullah83Contributor

asked in Data

IFC Project Base Point

I am using Civil3D to model a wastewater pipe system using the CRS EPSG:3380. Then, I converted it to IFC using the IFCInfraexport command. During this export process, I noticed that an IFC Project Base Point was created.I then imported it into FME, using the Reader (IFC) and Writer (Shapefile), also with CRS EPSG:3380. After opening the resulting file in QGIS (also using CRS EPSG:3380), the objects are not in their correct positions.I suspect this may be related to the IFC Project Base Point. Can anyone help me understand this situation and how to correct it? Is there a transformer that can modify the IFC Project Base Point?

1175

1 year ago

abdullah83Contributor

asked in Data

Cant find IfcRelConnectsPorts feature in FME reader

Seeking help,I have an IFC file containing information about sewage pipes. In theory, it should contain two types of object classes (with geometry), namely IfcDistributionChamberElement (Manhole) and IfcPipeSegment (pipe). These classes use their subclasses as connectors, namely IfcDistributionPort (at each object node). Finally, these IfcDistributionPorts should be connected to each other using a relationship class called IfcRelConnectsPorts. My problem is that when I import the IFC file as a reader, the class IfcRelConnectsPorts cannot be found, which prevents me from establishing connections between manholes and pipes. Here is a screenshot. in IFC file (open by notepad) Here is a screenshot. fme dont hv any feature or attribute name IfcRelConnectsPort.

136

1 year ago

jakethepainterContributor

asked in Data

Why is reading ArcGIS Online attachment failing in 2023 when it works great in 2019?

We are upgrading from 2019.2.1.0 to 2023.1 I have many AGOL feature services with attachments that we read and then write to a local database for archiving.For example, I have a feature service that has about 50,000 point features. Only 4 of them have attachments (500K jpegs). I have it set to read in batches of 1,000.In 2019, if I read this feature service with Attachments = Yes, it takes about 1 minute to read everything. If I open the same workspace in 2023 and run it without updating the reader, it times out every time. Sometimes it doesn't even log that it reads the first 1000 before timing out in about 7-10 minutes. Other times it has read for over an hour but still times out. If I disable attachment reading, it reads quickly in 2023. If I upgrade the reader to 2023, the reading still times out when attachment reading is enabled. The same is true for FME server. On our old 2019 server this workspace runs in about 2 minutes, but it times out every time on 2023. Help, pleas

552

1 year ago

aaronContributor

asked in Data

Need compatible version of MS Access Runtime to run on FME Desktop/Form and FME Server/Flow

I have an FME 2020 workspace with an MS Access reader in it. MS Access 2013 Runtime is installed on my laptop. The workspace runs correctly. I have two FME Server/Form environments (2020 and 2023) with MS Access 2019 Runtime installed on them. The workspace will not run on either of them. 2020 FME Server wants the MS Access 2013 Runtime installed ("ADO Provider not installed or not compatible with FME architecture. Try installing Microsoft Access 2013 Runtime (x64)"). 2023 FME Flow wants the MS Access 2016 Runtime installed ("ADO Provider not installed or not compatible with FME architecture. Try installing Microsoft Access 2016 Runtime (x64)"). Is that because those versions of FME Server/Flow only work with those versions of MS Access Runtime? Or, is the MS Access Runtime info embedded in the FME workspace and I should install MS Access Runtime 2019 on the my laptop to work with either version of FME Server/Form? Bottom line is I would like to run the workspace o

1720

1 year ago

arthyContributor

asked in Data

Web connection issue related to SalesForce

Hello,I have a SalesForce account that is working but for somehow I'm not able to read data with FME.When setting my account, it was first asking me my username and my password After fixing this, now I'm not being able to create the web coinnection for Saleforces. I'm only receiving the following error message . I went to manage web connection and tried to test the web service for Saleforces but still the same error.Any ideas?

Alternative methods for feature cache of large datasets

Feature cache works great as a method to inspect all transformers after a workbench has ran, but causes a few issues for me.it hurts performance to cache each and every transformerI lose all the cached data if FME crashes or I close the workbench.I am looking for advice on ways to cache or otherwise save output from only a few transformers. My only idea is to write features out to a file, and later bring back in with a writer. are there other transformers I may not be aware of that can save/recall data after restarting the workbench?

1 year ago

hayden

asked in Data

Restore Cached Data after crash

Is it possible to restore the cached data from a ran workbench that has been closed or crashed? I am running a heavy dataset (10s of millions of features) and it takes several hours to run. As I troubleshoot the workbench after, I am bouncing around between the cached inspectors for each transformer. My workbench has been crashing often in this process, and I wish I could re-open it and get back to the full cached state without having to re-run it.

Best file type to read/write locally for performance

I would like to write features to a local file to read back in later and looking for the best file type for performance.My local machine does not have any DBs to use for this purpose (Oracle, SQL, etc)so I am limited to files I can save to a PC folder.Does FME have it's own DB file type that I could be utilizing? I tried FME Feature Store (FFS) but it has limited functionality, such as not being able to set truncate or table drop on runs. And it's main purpose is to add encryption which I don't need and seems like it hurts performance.I have read on here that CSV is very high performance but I don't want to lose metadata such as data type, len, etc.Excel has a feature limit that does not work for me.

101

1 year ago

billybobContributor

asked in Data

read only the newest file from a One Drive directory

Does any one know how to read the latest file from a One Drive folder. I'm able to select a file from the One Drive folder using a Microsoft OneDrive (Graph) web connection, but I can't find a way to read all the files in the folder and then find the latest. I have read the question regarding reading the newest file for a folder https://knowledge.safe.com/questions/65203/read-only-the-newest-file-from-a-directory.html. It's very helpful condeptually. I would appreciate an thoughts or insight. Thanks,Frank

687

1 year ago

so_much_moreContributor

asked in Data

Uppercase characters for spatial column in feature writer not accepted

I am using the feature writer to write new column into a geopackage database. Instead of using the default geom I need to change the name to Geometry, but the writer does not seem to accept the spatial column to have uppercase characters?When I specify the spatial column to be called Geometry it is being writter as geometry in the table. Any ideas how it can be solved?

1 year ago

oscardInfluencer

asked in Data

DB2 - Use SQLCreator to call stored procedure

Good morning!I need to call a stored procedure of DB2 from FME. If I use the IBM Data Studio this script works fine:CALL TESTING.TEST_PROCEDURE('input1','input2',?,?,?,?,?);It has 2 input parameters and 5 output parameters.But when I use the same script in the SQLCreator of FME i always get this error: Literal replacement parsing failed for procedure call to DB2 for z/OS. Failing SQL text {my_script}. ERRORCODE=-4463, SQLSTATE=42601 Does anyone have any experience calling DB2 stored procedures in FME? Any help would be appreciated.I'm using FME 2021.1.

2174

1 year ago

lazarlubomirContributor

asked in Data

Sharepoint on premise - how to upload files?

Helllo everyone, I´m quite struggling about loading files to MS Sharepoint on-premise. I´ve found some tutorials, but just for Sharepoint Online. Any ideas please?Lubo

1562

1 year ago

topotomaContributor

asked in Data

KML - How to have symbols without HTML table in TOC

Hello, I am exporting data to kml and in the placemarks list, an html table is included.I would like to have only the placemarks listed without the html table preview. Here is an example: What I would like to get is this:Would anybody have experienced this and know how to sort it out and make it look like the second one ? Thanks,Best regards,Thomas

1 year ago

robinrutten

asked in Data

AGOL attachments: Reading features with mulitple attachments

I created a workbench that reads attachments of AGOL feature layers and write them into a GDB. The scripts works fine for features with one attachment. However, when a feature contains multiple feature, it's only reading one in my script. Does anyone know which setting I have to set, to read all attachments for every feature?

193

1 year ago

felipeverduEnthusiast

asked in Data

Read a .dbs file with FME

I am interested in read several .dbs files with FME. I have read that this .dbs is a Gupta Technologies SQLBase database file.Is there any chance that FME can read this format?

351

1 year ago

scarecrowContributor

asked in Data

Deleting folders and contents with python caller

Hello all, I've been trying to delete a folder containing some data after processing. I cannot use a temp folder so have turned to python to complete the task.I've tried following the comments in the following thread but it doesn't seem to work for me File Copy - DELETE (safe.com)I can get the process to work when 'hard coding' as shown below. But when I try to make it dynamic by replacing 'location' with feature.getAttribute('filepath') and dir with feature.getAttribute('directory') it fails.The filepath and directory attributes are shown below:'filepath' = C:\\Project\\FME\\processing\\data'directory' = 20230925

17211

1 year ago

dor1411993Contributor

asked in Data

Reading new File format: SKN

Hello everyone!We have a file format that we work with, which is used by our engineering world to create bills of quantities for engineering projects.The name of the format is SKN, and in fact it is very similar to csv/excle,Is there a way I can just read it and put it in my workflow?I am attaching an example of the file here (zipped)Thank you!

312

1 year ago

suchita_hessParticipant

asked in Data

Read AGOL related table

How do we read AGOL relationship classes/related tables in FME? The published table can be imported separately but the relationship linkage is not maintained in the parent feature service. Is there a way to import feature service in a way from where the related table is linked?

210

1 year ago

rob_m_esriukContributor

asked in Data

Seeing 504 errors using the ArcGIS Online Feature Service Writer on a small dataset, only on rewrite not create

I have 4 classes that I read from ArcGIS Online, reformat and write out to new classes, all are small and to create each output I've set "create if needed" to yes on the writer and each class is set to "use existing". All 4 classes are created. I then set the model to "create if needed" to no and each class to truncate existing. 3 of the classes update. the 4th with 55 rows truncates existing but then the writing times out with an HTTP 504 error. I've tried setting the features per request to 1 and that times out also. If I delete the class in AGOL I can still create it from the mode but i can't update it.

DWG and DGN reader not exposing fme_basename and published parameter

I am reading into my workbench both dwg and dgn files using two feature readers. I am able to expose the fme_dataset for the dgn file by using an attributeExposer transformer, but the fme_dataset will not be exposed for the dwg files. I am able to get the fme_feature_type and fme_text_string for each FeatureReader. But I want to add the file name (fme_basename) as an attribute to each feature so I know what file it came from. Things I have tried:separate AttributeExposer transformers for each FeatureReaderThe same AttributeExposer transformers for both FeatureReadersAdding fme_basename to the <Generic> Port, along with $(SourceDataset_DWG)checking all of the needed options in Reader > Parameters > Schema AttributesWhen I open the inspector after the Exposer, all of the attribute fields are there but the values are empty. All of this information (file name, project name) comes from published user parameters. How can I add the value for these user parameters into my features

674

1 year ago

Page 25 / 272

Badge winners

bwnhas earned the badge Thumbs Up for OGC Formats
verohhas earned the badge Teacher (Grey)
verohhas earned the badge Welcome Back
aaronhas earned the badge English Speaker
aaronhas earned the badge Welcome Back

Show all badges

30,126: Posts

Terms & Conditions Cookie settings

Sign up

Already have an account? Login

An FME Account is required to contribute

Login to the community

An FME Account is required to contribute

Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.

Enter your e-mail address

Back to overview

Scanning file for viruses.

Sorry, we're still checking this file's contents to make sure it's safe to download. Please try again in a few minutes.

This file cannot be downloaded

Sorry, our virus scanner detected that this file isn't safe to download.

A product of

Company Products Giving Back Careers Contact Privacy Feedback

Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.