- - Community Resources
Ideas
Product Updates
Events
Blog

Home
Forums
FME Form
Data

Data

Using readers and writers to extract and load data.

6,814 Posts
25,102 Replies

When you subscribe we will email you when there is a new topic in this category

6814 Posts

Database block size in Database Reader vs SQLExecutor

How to compare a series of tables across two databases? We use a setup that reads a .csv file that has two columns:- Column 1 is a query in database 1, and- Column 2 is a query in database 2Both queries should return the same and ordered list of values (basically, a list of identifiers) and are fed into an SQLExecutor. Both resulting streams are compared using a FeatureMerger, and the UnmergedRequestor and UnUsedSupplier ports report the features that are not present in both databases. This works nice for relatively small tables. If a table is queried that has more than a million records, performance drops. I solved this performance issue by using database readers. I specified the Number of Records To Fetch At A Time (ORACLE) and the Bulk Read Size (MSSQL) and set them so that 100.000 records are fetched each time. I could work this out using a WorkspaceRunner, however my question is: Where are these parameters on the SQLCreator/SQLExecutor transformers?Parameters available on the SQLE

4310

4 years ago

antoineContributor

asked in Data

Is there a way to get a list of a pointcloud components (not in inspector only)?

There is no PointCloud element in the Python API and I can't find any function or factory to do it. Has anyone found a way (apart from parsing the log)?

4 years ago

wmwatson91

asked in Data

Do I need to wait until FME 2020 is released before I can use FME with ArcGIS 10.8? Thanks.

Will there be an update for FME 2019 to make it compatible with ArcGIS 10.8, or do I need to wait until the release of 2020?

Hi FME Fans,Is there a way to run a .dll file from within FME and also pass data to it?Cheers

470

4 years ago

solyContributor

asked in Data

How could I look for name in excel styler for csv file

I have around 100 cells that located in third attribute (machine ) and I want to change the back ground for cells that has specific name old to blue color,i converted the excel file to csv then write again to excel file in my workspace so If i want to look for names in csv ,so my question does excelstyler has another methid not as excel to look for names in csv file.I tried a lot by excel styler but it does not give me result .and also I want to select input for excel styler as row bcs I found it has only option for attributes ,for column but I want to work on rows to alignment line for each specific row ,FME version 2018.Thanks

4 years ago

benvkContributor

asked in Data

How to filter DXF elements into shape files of various geometry

I have a DXF file containing many feature types, including points, polylines, polygons, text etc. I want to extract the polylines into a shapefile of polyline geometry, and the points into a points shapefile. I am having a hard time finding the transformer(s) needed to carry out this simple task. Can someone please point me in the right direction? Regards, Ben

Doing a JOIN in ArcSDEQuerier or ArcSDE reader

Hi all, I found this wonderful piece of informatjon in the description of ArcSDE reader (http://docs.safe.com/fme/html/FME_ReadersWriters/Default.htm#sde30/sde30.htm%3FTocPath%3DFME%20Readers%20and%20Writers%20(formats%20supported%20by%20FME%202014)%7CEsri%20ArcSDE%20Reader%2FWriter%7C_____0): The SDE reader can also perform multi-table join queries, thereby exploiting the full power of the underlying RDBMS. Cool... I wanna do that, and most likely either the ArcSDEQuerier or ArcSDE reader, whichever is most convenient. But I can't seem to wrap my head around the syntax. Composing a valid JOIN statement in SQL should be fairly straightforward, but that doesn't really seem to fit into the concept of WHERE Clause. Or I cant figure it out... Problem description: I have a huge ArcSDE database. I and I want to extract all data from table A, and a subset of information from table B, only extracting those whose ROUTE_ID values match ROUTE_ID in table A. In my head, this should

170

4 years ago

pchengContributor

asked in Data

Sharepoint & IQY file in FME

Hey FMErs! Wondering if anyone here has worked with an Internet Query (.IQY) file downloaded from a sharepoint site before! When I open the link in a web broswer (IE11 in this case) it'll prompt a download for this IQY and it'll resolve into a excel spreadsheet! I want to run a automated process on this file from FME but when I use use a XLS reader or HTTPCaller transformer all I've been getting are error messages or the 'GET' response:(left out some details as its internal)WEB1https://xxxxxSelection={7EB5B6E8-8458-4F92-925E-A3EB593229E7}-{9814593B-164C-4604-A5DD-4CAF07C9FDF7}EditWebPage=Formatting=NonePreFormattedTextToColumns=TrueConsecutiveDelimitersAsOne=TrueSingleBlockTextImport=FalseDisableDateRecognition=FalseDisableRedirections=FalseSharePointApplication=https://xxxxx/_vti_binSharePointListView={9814593B-164C-4604-A5DD-4CAF07C9FDF7}SharePointListName={7EB5B6E8-8458-4F92-925E-A3EB593229E7}RootFolder=Thanks in advanced, Patrick

300

4 years ago

erik_janContributor

asked in Data

Issue with writing to SDE feature class and archiving enabled

Hi FME and ESRI users, In an SDE feature class archiving is turned on.When writing a record in update mode, an attribute is blanked out, all other attributes get values.But redirecting to the Data Inspector shows a value for the attribute. We checked the data type in SDE and FME and they match. Any idea what could be causing this issue? Your suggestions are highly appreciated.

120

4 years ago

gogopotter90Contributor

asked in Data

how to inspect on workbench and get attribute lists from oracle non spatial file

could i ask how to get the list of attributes from oracle non spatial.iam a little bit confused from oracle files .i added a reader for oracle non spatial but i could not check the oracle on inspector when i conenct with inspector i see nothing,is there a way to inspect it while iam working on workbench . and when if i want to extract only the attributes ,how could i do that i want to extract the attributes in table Thanks in advance

4 years ago

gauripearl

asked in Data

How to insert geometry column into Oracle from SQL Executor?

I am using SQL executor to perform insert command to insert table data into Oracle. I have a column with geometry and data type as SDO_Geometry in already existing oracle table. The current geometry I want to insert into oracle is in UTM format. I have converted it using geometry replacer and reprojector in my workspace.SQL Executor is throwing an error with too long string literals and can not be inserted.What is the way that I can insert geometry column with UTM coordinates into Oracle SDO_Geometry column?

Is it possible to increment a sequence with another value in Oracle writer in FME workspace?

Here I am trying to get the sequence before writing data into oracle. All I need to do is to start this sequence with the value of 5000000000.As we can use this in insert function - "insert into nb (XFM_ID) values(XFM_ID_SEQ.NEXTVAL + 5000000000)"But this is not working in FME. I tried all possible ways to insert sequence starting from 5000000000 like using sql creator,executor etc. Sequenced Table columns does not accept " XFM_ID:XFM_ID_SEQ.NEXTVAL+5000000000"

I am looking to insert a new feature into the output geodatabase here. The translations uses a feature reader to read a source geodatabase based on a user defined parameter which defines which feature classes the user wants to bring over. However I also need to add second feature class but instead of reading from a source I need to add a new record which i intialize from the creator. In this case though because the output writer uses a dynamic schema, I don't know how to insert this new feature or to indicate which feature class it should be written to. Any ideas?

The error number from ArcObjects is: '-2147215875'

I am trying to write data from GDB file into oracle database using SDE writer. Its giving below error Could not unregister GlobalID column 'GLOBALID' in table TABLE_NAME. The error number from ArcObjects is: '-2147215875'. The error message from ArcObjects is: {Table is already locked [TABLE_NAME]}

Data QA: GML geometry - missing vertices

Hello,I need to create and implement control rules which can show that my object (in GML) has dupplicate vertices.I have reader "GML file" with the element which geometry is:<bdz:geometria> <gml:Polygon gml:id="geom.9acd0ccc-6a2a-4dfa-8e70-487b08d83e7b" srsName="urn:ogc:def:crs:EPSG::2176"> <gml:exterior> <gml:LinearRing> <gml:posList>5947602.08 5474440.64 5947598.85 5474445.14 5947598.85 5474445.14 5947602.08 5474440.64 5947603.36 5474448.24 5947606.44 5474443.96 5947602.08 5474440.64</gml:posList> </gml:LinearRing> </gml:exterior> </gml:Polygon> </bdz:geometria> It contains 7 vertices (1 dupplicate vertex), but when I add inspector, FME show me that there is 6 vertices: Neither the geometry validator nor the coordinat

4 years ago

laurammg

asked in Data

Publishing Rasters on ArcGIS online

Hi, the result of my workflow is a raster and I would like to publish it on ArcGIS online automatically with FME. Is it possible? Thanks

Make non spatial data spatial

HelloI am attempting to make non spatial Oracle table data spatial, I have created the FME workbench no problem but when it comes to writing the results to an Oracle table using Oracle Spatial table no points are displayed just the data.Basically I am missing the accompanying spatial data to the points. I am sorry if my post makes no sense and please ask any further questions. Many thanksSy

Suddenly, FME Server unable to access ArcGIS License

We are running FME Workbench 2019.1 and FME Server 2019.1.2 on Windows Server 2016 Standard. On the same server, we have ArcGIS Desktop 10.6.1 and ArcGIS Server 10.6.1. The Desktop license is running off a concurrent License Manager on another server. We have the ArcGIS Desktop license set to Advanced on ArcGIS Administrator.This setup has worked fine until about 4 days ago. Now many of our FME services are failing. The workbenches run fine. An excerpt from the log on one FME service is shown below. I have already exhausted my search for solutions. I am considering trying to install ArcGIS Pro on the server, though I would prefer not to. Any suggestions would be appreciated. Thanks!145The Esri Product license used by this machine does not include licenses for the 'ArcServer' licensing level146The ArcObjects license 'Advanced' is being selected from the Windows registry147The Esri Product license used by this machine does not include licenses for the 'Advanced' l

Hallo guys!I want to transform/convert an OSM file to an OpenDRIVE file. So my question is if anyone have some informations for me how that's possible and which steps are needed for. I also tried it with "GIP" data. My main problem is i have no idea how to add attributes or to get an OSM data into a structured OpenDRIVE file. Thats my first time with FME so i have no Idea how that should work.

Microstation DGN Links not supported in FME?

Hi, I have a DGN file that have links attached to elements.However, I cannot see the links exposed/supported in FME.Please see a screen grab showing the link attached to element in Bentley Microstation.I cannot get to this object in FME.Please can you help?Many thanks, Magda

Oracle to shapefile - Unable to output data to file

I am attempting to convert from Oracle to a shapefile and getting the following error consistently. look like it is failing on the .dbf file. Any suggestions?

200

4 years ago

asellarsParticipant

asked in Data

Extracting Nested JSON

Hi All, I am having quite the time getting some nested JSON out of an attribute. My current workflow flattens out the original JSON, then fragments out the nested meetings. I then flatten it out again to expose each attribute for each meeting. I get to a point where I now have an attribute "agendaitems" that can have different amounts of items for each meeting. This is where I am having trouble flattening it out from here. The attribute is filled as <missing> but when inspected has the items listed in the attributes as: agendaitems{0}.billagendaitems{0}.descriptionagendaitems{0}.item and so on for however many items are associated to it. Is there anyway to get these to all flatten out?

What does error "Failed reading coordinate data at offset 0" mean?

Using reader MITAB on a file written in 2006 and getting the error "Failed reading coordinate data at offset 0".Sometimes, the error doesn't stop further actions, sometimes it does.Can anyone point out the likely issue?Cheers!

problem when translate DXF

Hi folkswhen i try a translation from DWG to a GIS format, i encounter that message : FATAL |Incompatible visitor version 1 in FMEOrientedArc::acceptGeometryVisitorConst. This release of FME is compatible with versions 6 to 6 I tried many ways to solve it, GeometryValidator, Orientor, but the message still remainsthe DXF file is hudge, so i can't attach to the message ...regards

Error when using FME to create a table into Oracle when the user is not the owner.

Hi all I have a workspace that is managing a dynamic migration from one Oracle 11g database to Oracle 12c. The process is quite mature and has mainly be used to truncate and insert new data. We now have to extend this process to create tables if required. The writer connection is a different user (USER1) to then one where the table is being created into (USER2). USER1 has heightened privileges and is used for all maintenance tasks. Grants include 'create any table', among others. We have started to encountered exceptions when attempting to drop/create a table into the destination instance. Using Feature Writer or a writer/Feature Type, when we try to 'drop/create' or 'Create if needed', we get the following exception raised: "Cannot create table 'TABLE_FROM_FW' within schema 'USER2', when logged in user is 'USER1' ".Full log with debug here: I have tested with Oracle Spatial Object and Oracle Non-Spatial writers both withe sam

500

4 years ago

Page 85 / 273

Badge winners

mariakahas earned the badge Integrate Spatial Data
pkakalikahas earned the badge FME Form Advanced Training
emerson_styleshas earned the badge Integrate Spatial Data
spatialcasehas earned the badge Welcome Back
louise.gavhas earned the badge FME Form Basic Training

Show all badges

30,386: Posts

Terms & Conditions Cookie settings

Sign up

Already have an account? Login

An FME Account is required to contribute

Login to the community

An FME Account is required to contribute

Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.

Enter your e-mail address

Back to overview

Scanning file for viruses.

Sorry, we're still checking this file's contents to make sure it's safe to download. Please try again in a few minutes.

This file cannot be downloaded

Sorry, our virus scanner detected that this file isn't safe to download.

A product of

Company Products Giving Back Careers Contact Privacy Feedback

Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.