Skip to main content
  • EN

Data

Using readers and writers to extract and load data.

  • 6,987 Posts
  • 25,736 Replies
6987 Posts
dannymatranga
Contributor
dannymatrangaContributor
asked in Data
PowerBI Semantic Model Reader?

Is there a way to read data from a PowerBI semantic model in FME Form?

142
dannymatranga
Contributor
hkingsbury
Celebrity
hkingsburyCelebrity
asked in Data
Mapping data types stored as attributes

I feel like I’ve seen this done before, but can’t put my finger on how to do it…..A warning, I’m working with 2020.2 so cast your mind back a few years….. Essentially I have a workspace that is taking data from MSSQL and writing it out to SQLite. In theory this is just a straight Reader to Writer.However, the SQLite file is being created fresh each time the process runs and it’s creating any number of tables based on relationships within the MSSQL database - these relationships need to be maintained. Therefore, the schema in the SQLite table is being created with an SQLExecutor and CREATE TABLE statements. Currently, I’m reading the schema from MSSQL and manually mapping each attributes data type from the MSSQL definition to SQLite using an AttributeValueMapper. Now it wouldn’t take too much effort to make this list exhaustive, but there is surely a better way. I initially thought of using a schema attribute and let FME do the MSSQL→ FME→ SQLite mapping with a dynamic writer, and that

392
hkingsbury
Celebrity
marava
Contributor
maravaContributor
asked in Data
Filter Reader Data based on other reader

Hi, I need to filter some ArcGIS Online Portal Readers based on a unique ID that exists also in another reader (file geodatabase). ExampleReader 1: ArcGIS Portal features w/ UID Reader 2: File Geodatabase features w/ UID Reader 1 should only load features that have a UID that match Reader 2. The UID fields are the same format. Is this possible? Please let me know if can provide further details. I am a beginner/intermediate FME user focused on GIS. Thank you!

80
marava
Contributor
bec
Contributor
becContributor
asked in Data
Overwrite existing data without overwriting the data and formatting in the rows above the existing data via FME Excel Writer

I’m using an Excel Writer in FME Workbench 2024.2.  This is a workspace that will be run yearly and I have created an excel file to write to in the format that I need so that the end user gets the same output year after year.  I would like to use FME to clear any existing data in the excel file from row 5 on down, but still keep the data and formatting that exists in the rows above.  I tried Truncate Existing Sheet/Named Range: with my Start Row set to row 5, but cleared all of the existing data from the sheet and then wrote the new data starting at row 5.  In the below images, you can see what the sheet looks like before the writer is run, the writer parameters, and what the sheet looks like after the writer is run.  Is there a setting in the Writer Parameters that I need to change to ensure that the data and formatting in rows 1- 4 doesn’t get deleted?  Or is there another way that I need to approach this?  Any help would be greatly appreciated.  Thanks! Sheet view before writer is r

131
nielsgerrits
VIP
mr_fme
Enthusiast
mr_fmeEnthusiast
asked in Data
Problems with python interpreter

Hi,I had a script that was running perfectly in FME 2024.1. I upgraded to 2025.0, encountered some issues, and my script stopped working. I then reverted to the previous version, 2024.1. However, my script still doesn't work. When I run it in PyCharm using Python 3.10, it runs perfectly, but in FME I get the following error:Message Type: fme::internal::_v0::py::ExceptionPython Exception <SyntaxError>: invalid syntax (functions.py, line 68)In the FME options, I have the same paths set as before: PYTHONHOME: C:\Users\XXXX\AppData\Local\Programs\Python\Python310 Interpreter: "C:\Users\xxxx\AppData\Local\Programs\Python\Python310\python310.dll" What could be the issue? Could you please help me? Thank´s

152
mr_fme
Enthusiast
rmpayaby
Contributor
rmpayabyContributor
asked in Data
Converting DFX to IFC

I have a DXF-file consisting of lines and surfaces. Is there any standard method on how I can convert such files in IFC?

241
desiree_at_safe
Safer
tb09114
Supporter
tb09114Supporter
asked in Data
How to publish a choice parameter from script?

I need to have a choice parameter offering all the database tables of a specific database for the user to choose from before running the workspace.I do have the Python code ready, and created a scripted parameter, returning a list of all table names.I referenced the scripted parameter as input for the choice parameter, but that only gives me a dropdownbox with the name of the parameter linked to it. Is it possible to populate a choice parameter from a Python script (scripted parameter), and if so how?

704
mackinneyjg
Participant
couch
Contributor
couchContributor
asked in Data
N-Triples (.nt) data

Anyone come across .nt files (N-Triples) files? Their doesnt seem to be a native FME reader for it.

171
hkingsbury
Celebrity
gg33130
Participant
gg33130Participant
asked in Data
Parameters for StringSearcher

Hello !I want to extract files wich have everytime a different expression but  there is some text who would be everytime in the file name :Example : [200072692_ZONE_URBA_20220412] “ZONE_URBA” will be the text present everytime in my extract file.I want the String Searcher to download the files which only contains “ZONE_URBA”The numbers in the example before and after “ZONE_URBA” will change everytime because they are the number of the EPCI (french administrative numbers)After calling ZIP file from HTTP with HTTPCaller, I use the ZipExtractor to extract the zip file.Then I use the StringSearcher but I don’t know how to configure the parameters to recover only the ZONE_URBA files … Anyone got an idea ? I think it’s not something impossible but i’m beginner.Sorry for my english, I promised I’ll improve !!Thanks a lot Here my work chain : 

141
raghavendrans
Supporter
nick3_s75
Contributor
nick3_s75Contributor
asked in Data
How can i extract data from this line?

How can i extract data from this line?  {"sl:stand": {"sl-bag-extract:bagObject": {"Objecten:Pand": {"Objecten:status": "Pand in gebruik", "Objecten:geometrie": {"gml:Polygon": {"@srsName": "urn:ogc:def:crs:EPSG::28992", "gml:exterior": {"gml:LinearRing": {"gml:posList": {"#text": "114062.419 542728.035 0.0 114067.343 542726.792 0.0 114067.637 542728.082 0.0 114068.445 542728.468 0.0 114069.011 542730.438 0.0 114068.36 542731.246 0.0 114068.639 542732.471 0.0 114063.762 542733.793 0.0 114064.204 542735.512 0.0 114057.619 542737.161 0.0 114055.721 542729.372 0.0 114055.572 542729.428 0.0 114053.081 542719.589 0.0 114059.937 542717.727 0.0 114062.353 542727.763 0.0 114062.419 542728.035 0.0", "@count": "16"}}}, "@srsDimension": "3"}}, "Objecten:voorkomen": {"Historie:Voorkomen": {"Historie:BeschikbaarLV": {"Historie:tijdstipRegistratieLV": "2010-11-26T12:31:16.436", "Historie:tijdstipEindRegistratieLV": "2014-10-27T12:32:20.904"}, "Historie:eindGeldigheid": "2014-09-25", "Historie:beginG

537
nick3_s75
Contributor
pkno
Contributor
pknoContributor
asked in Data
Writing 3D Geometries to Oracle not working

Hello,I am trying to write 3D Geometries (Points and Linestrings) to an Oracle-Database (21c Express). In the process all z-Coordinates get lost and SDO_GTYPE is wrongly written as 2001 or 2002 (2-Dimensional Geometries) instead of 3001 and 3002.Multiple Geometries are entering the writer as an aggregate (example below).  Apart from the z-Coordinate missing these are written and interpreted correctly when writing to Oracle. I have checked the “Contains Z Ordinate” Option in the Writer-Parameters.Does anyone have anymore ideas what I could try?

211
pkno
Contributor
wrijs
Contributor
wrijsContributor
asked in Data
Remove connecting thing areas from complex polygon.

Hi there, I am working on a data clean up model. Many thin areas and remnants have been cleaned up but I can't figure out how to this last thing.There are some thin areas that connect on both sides to a large and very complex polygon. (See image)Does anyone have a suggestion for how to remove these type of areas? So only the thin parts have to be removed.

333
wrijs
Contributor
tva
Contributor
tvaContributor
asked in Data
FME Form (desktop) best practices and style

Hi,I was wondering if there are some best practices and or style guides / examples on for workspaces. Some questions that I have about this (besides the obvious: avoid crossing as much as possible, annotation, clear transformer names etc) Like is there an agreed style for public and private parameters (casing, under scores, ...)Is there an advise on how many transformers can be put in the main part and how many in a custom transformer?How many custom transformers can be nested into each other or is it advised to not do this to deep (performance impact)Any other good to implement styles?

23613
redgeographics
Celebrity
shamoneau
Participant
shamoneauParticipant
asked in Data
FME Form Ubuntu 22.04 synology file server permission problem

Hi,We are trying to use a FileCopy Transformer to transfer files from one location to our server (synology) / other computers (other computer).If we do it in Ubuntu GUI / SystemCaller command it works perfectly.But the file copy transformer always give error messages as we don’t have sufficient permission. Even if I put our NAS credential in the transformer (with a featureWriter), it give the error messages. The files are created in the server but there’s no content inside.Does somones already experienced it ?Thank’sSam

160
shamoneau
Participant
joy
Enthusiast
joyEnthusiast
asked in Data
Black screen FME Form 2023 and FME Form 2024

Hello everyone,At our organization we work on a virtual machine. When you log on into a virtual machine, you share the virtual machine with other users. Since a few weeks we encounter sometimes black screens when trying to use FME form 2023 or FME form 2024. I googled and the problem is quite similar to Known Issue: FME Workbench 2023 Opens as a Blank Window on a Virtual Machine – FME Support Center, instead it's a black screen. The article mentions about OpenGL versions, however the specs for all our virtual machines should be the same, so I think the cause is different.When you encounter the black screen, you can only hover over the black screen and can't do much else. When you close FME Form and try to open a new FME Form sesssion, you still get a black screen. The only workaround is to log off and log into a new virtual machine where this problem doesn't occur. We already did some troubleshooting and we know one thing for sureIt's possible that fme form 2023/2024 initially worked f

804
joy
Enthusiast
bkerr
Contributor
bkerrContributor
asked in Data
Attribute Creator - Adjacent Feature not populating more than 1 record

Hi, I am using the attribute creator to populate a new field. I want to fill in the blank rows with the previous master property number from the master prop no field. I have enabled the adjacent features but no matter what I try it still only populates just the 1 record after a value and not all the blanks. I thought I would only need 1 based on what I’ve found on the forums.What am I doing wrong here?I also tried to do that same thing via the attribute manager but get the same result.The end result would also have row 3 (FME row id not the xlsx one) with ‘932590’ and row 6 with ‘933123’ and rows 9-11 with ‘933796’ etc with the number of blank rows varying from 1 to potentially hundreds.The settings I have for the transformers are below.I am using FME 2022.2. Thank you! Belinda

596
crutledge
Influencer
norris
Contributor
norrisContributor
asked in Data
Fill in adjacent empty values in table is not working as I expect?

I am trying to fill in the rows of my table with the previous values. I saw this thread and I am following the instructions but for some reason I am only getting the first empty row to be filled but not any subsequent rows.  I tried to change the Subsequent Features to another value other than 0 and even tried to change the number of Prior Features but I always get the same results.I am on FME Form 2024.2.0.0 if that helps. Any suggestions as to what I am doing wrong?  

355
norris
Contributor
aashnaparikh
Contributor
aashnaparikhContributor
asked in Data
Mimic Excel Formulas in FME

Hi, I am trying to replicate the V-Lookup formulas in FME that I currently have in my excel. Also the current excel formulas refers to a different sheet named Limits and I am looking to create the same sheet in the FME and finally when the writer generates the output the file should have 2 sheets - 1st one where the data is written and the 2nd will be Limits. Writing to the existing template is not an option due to complexities involved  

303
crutledge
Influencer
arenscott
Contributor
arenscottContributor
asked in Data
SQL Count function

Hi all, I’d like to use FME to execute some count SQL functions on SQL server (SDE). I’m having trouble with the syntax of the SQL in FME and I’m not receiving any output from the SQL Executor. SQL I’m running:SELECT COUNT(*)from "SDE".SAMPLE;

7610
hkingsbury
Celebrity
fmewithvasu
Contributor
fmewithvasu
asked in Data
reading spatial data from Oracle to writing to SQL server

I'm using FME to write spatial data to a SQL Server database via an SDE connection, but the table is created without the spatial data. How can I ensure the spatial data is written correctly?Error message below for refernce An error occurred while attempting to create the feature class 'EL.DATA'. The error number from ArcObjects is: '-2147216061'. The error message from ArcObjects is: {Table already exists [42S01:[Microsoft][ODBC Driver 17 for SQL Server][SQL Server]

381
desiree_at_safe
Safer
marionmahot
Contributor
marionmahotContributor
asked in Data
List to attributes

Hello everyone,I performed a feature merger between two shapes and obtained a list from this transformer. My list is as follows:insee: 11232code: PTRSURF: 11insee: 11232code: PPHSURF: 15insee: 11232code: SPLSURF: 23insee: 15500code: PPHSURF: 2insee: 15500code: SPLSURF: 555So I want to assign the elements of the list to an attribute table (in columns). I have a maximum of 96 elements for a single code, so my resulting table will have many columns.So I want to get this: How should I do this?THANKS

322
marionmahot
Contributor
froberts
Contributor
frobertsContributor
asked in Data
MBtiles Writer and forcing layers

I am having maybe a similar but different issue with the mbtiles writer.  For some reasons the Zoom level dialog is not making a lot of sense to me.  In my case I would like to force the writer to write out levels 0 to 11. In my head I am thinking I would set minimum zoom level strategy to “Lower”  and then the Minimum Zoom Level to 11.  My assumption is that when we talk about level it matches up with the table below.  However with the above setting I get a mbtiles file with just one level in i (see results 1 below for log). If I instead set Base zoom to Upper and Minimum Zoom to 0 (run 2 below for log), I get a file with 10 levels.  This is close to what I want, but I want 11 levels. Any ideas on how to better explain the dialog to me and get me 11 levels??  thanks so much!!!  Run 1 log: Feature Caching is ONThe workspace may run slower because features are being output and recorded on all output ports regardless of whether or not there is a connection.     Command line to run this w

421
desiree_at_safe
Safer
aashnaparikh
Contributor
aashnaparikhContributor
asked in Data
Best Approach to Write to Excel Template while preserving the structure, formulas and static values

What is the best approach to write to selected excel cells if I am overwriting using an existing template? How can I make FME understand to apply the formula or the predefined values based on the feature count in my input file ?  The template contains the predefined value in the first row. Item ID Program Full Name Master Certificate Number Address   Dentist Association   XL89                        My Input File / Reader FileItem ID Full Name Address 12 Jon Doe 14 King St 45 Mary Jane 5 Todds Ave 345 Henry Smith 7 Queen St  My expected Output Item ID Program Full Name Master Certificate Number Address 12 Dentist Association Jon Doe XL89 14 King St 45 Dentist Association Mary Jane XL89 5 Todds Ave 345 Dentist Association Henry Smith XL89 7 Queen St

191
nielsgerrits
VIP
julianat
Contributor
julianatContributor
asked in Data
DGN Cell Library in FME Server

Hi, I am having an issue of trying to use a dgn cell library and seed file on FME Flow for point features. The workbench works correctly on desktop but not Flow. Is FME capable to read cell library and seed files on FME Flow? This is the error message I get:Any help would be greatly appreciated! Thanks!

741
julianat
Contributor
cdog
Contributor
cdogContributor
asked in Data
ArcGIS enterprise update

Hi, I am trying to set up a simple update workflow that doesn’t like me. I am trying to update my enterprise portal with GDB updates and will eventually want to have this run everyday but for now I am just trying to figure out how to update one shapefile and repeat this 30 times. Right now I haver set up a GDB reader going directly into ESRi ArcGIS portal feature service writer to update an existing feature in enterprise and I have got it to write the features but it doesn’t want to publish the feature into portal.Can someone help me figure this out?    

548
cdog
Contributor

Badge Winners

  • Integrate Spatial Data
    laisgouldhas earned the badge Integrate Spatial Data
  • Student (Grey)
    wthallhas earned the badge Student (Grey)
  • Teacher (Grey)
    evoteck-gchas earned the badge Teacher (Grey)
  • FME Certified Professional
    j.botterillhas earned the badge FME Certified Professional
  • Expert (Grey)
    carlwhas earned the badge Expert (Grey)
Show all badges

Community Stats

31,238
Posts
118,443
Replies
38,833
Members

Latest FME

Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.