Skip to main content
  • EN

Data

Using readers and writers to extract and load data.

  • 6,929 Posts
  • 25,490 Replies
6929 Posts
scottpool
Contributor
scottpoolContributor
asked in Data
Can't get a dynamic URL to work properly in the OGC WFS FeatureReader

Hi,I’m trying to use the OGC WFS FeatureReader to read in features based on a dynamic URL stored as an attribute. This is because in the URL link, I want to have a bounding box parameter that changes based on the extent of another dataset. I.e., I only want to read in the features that are within the bounding box of another dataset.I’ve set up an attribute “URL” that contains the bounding box parameter.When I hard-code in the URL with the bounding box parameter into the FeatureReader, it works as expected and reads in only the features within the bounding box. If I replace the URL with the dynamic “URL” attribute, it ignores the bbox parameter and reads in everything. I  And yes, I’ve confirmed that the dynamic URL attribute is exactly the same as what I’ve hard-coded into the FeatureReader that works. Does anyone have any idea how I can fix this? Or any other ways I can read OGC WFS features with a dynamic bounding box?I’ve tried the Spatial Filter parameter in the FeatureReader too a

937
oliver
Contributor
arcvancouver
Contributor
arcvancouverContributor
asked in Data
DateTime Conversion from character field to proper UTC Time

Hi all. I am reading in an Excel sheet where DateTimeUTC is formatted like this as text, but I need to convert this into a proper datetime for UTC… thoughts? DateTimeConverter is out, or perhaps I’ve not formatted it correctly.   Otherwise, I can use attribute splitter to get the date, but the time part is tricky… not sure what to use to get that formatted properly

181
arcvancouver
Contributor
lcecil
Contributor
lcecilContributor
asked in Data
Write Subgroup Lists of SDE Feature Class to Google Sheets

Hi, I’m an FME newbie. I have an ArcGIS sde (SQL) feature class of rooms with about 15000 rows across several hundred buildings. I need to write a list of rooms, by floor to directories for each floor in Google Sheets. I’m not sure how to go about generating each list. I’ve messed around with list builder some, but I’m stuck. I could do this in Python, but I’d like to take advantage of the FME Google Sheets writer. 

384
lcecil
Contributor
jeetz93
Contributor
jeetz93Contributor
asked in Data
Excel sheet flattening for UID Generation

Hi all. I’m trying to generate UIDs for helping out with site inspection reports for structures. The structures are comprised of “Components” and each “Component” is comprised of “Sub-Components”. Each “Sub-Component” has a inspection lists (“Lists”) that are checkboxes that need to occur for each “Sub-Component”. Each checkbox is a coded item (“Code”) that must be inspected. There are multiple “Codes” for each of the “Lists”.The UID format that is desired is this:“Site ID”. “Component ID” . “Sub-Component ID” . “Code”The problem I am running into is that the data have been provided in Excel. There is a “SiteVisits” Tab, and a “Lists” tab. The SiteVisits tab has 350+ records, with columns containing Site IDs, Component IDs and Sub-Component IDs, and a column on the “Lists” required for each Site.SiteVisits Tab, with 350+ recordsThe “Lists” Tab contains a descriptor of each checkbox and the associated “Code”. What I am trying to do is concatenate the easy parts of the UID (“Site ID”. “C

615
crutledge
Enthusiast
bwhite
Contributor
bwhiteContributor
asked in Data
ECW Header information

HI, I have a problem to solve where I need to extract the coordinate system within the header information of the ECW image (not the sidecar files).  Our internal systems only work when the ECW header has “EPSG:7855” as the coordinate system, and I need to identify images that sometimes possess “GDA2020” & “Zone 55” information (as shown below).  The software “ECW Header Editor” can distinguish between the two, but FME seems to use the two values interchangeably, so my workbench can’t identify potentially problematic images.  The FME output for both “EPSG:7855” and the below entries are always “MGA2020-55_FME”.  Is there any way for FME to identify the below values hidden inside an ECW header? 

181
ctredinnick
Supporter
asali
Participant
asaliParticipant
asked in Data
ESRI Geodatabase Reader Not Working

Hi,I am not able to use the ESRI GDB reader, as I am not able to open a layer from my database.So I created a reader reading a layer from GDB and then later this layer was removed from GDB, now I am not able to use the GDB reader at all, even if I open a new workbench and press the three dots button in the parameters of the reader, used to select the “Tables” , it gives me a message “Failed to retrieve feature types” “Layer not found”, but the layer mentioned in the error message is the old layer which is already removed from the GDB. And i didnt even got a chance to select a layer. What to do? Please do help.

481
saraatsafe
Safer
charry
Contributor
charryContributor
asked in Data
3dtiles writer support front and back texture?

hello,3dtiles writer support front and back texture? How can I achieve it？

193
charry
Contributor
benvk
Contributor
benvkContributor
asked in Data
Is there a simple way to add a 'total' row to a Excel Writer?

I am outputting many rows to an excel table, some of my columns are numeric, I am hoping to add a total row at the bottom of my table, to sum these columns. I have found the Statistics Calculator, but I have no idea how to apply it (if at all possible). Any thoughts?

1046
saraatsafe
Safer
galigis
Enthusiast
galigisEnthusiast
asked in Data
Why is the FME area (Area Calculator) different to the ESRI GDB area?

Hi All,I have read a few posts on the community already but can’t understand why the area calculated using Area Calculator is different to the SHAPE_Area in the Feature Class after exported to a GDB.An example below:Any ideas why does this happen?Thanks :)

535
galigis
Enthusiast
tomf
Contributor
tomfContributor
asked in Data
Using functions such as @round() inside the XMLTemplater drops data

This is probably a simple question, but I'm having difficulty rounding values within the xml editor of the XMLTemplator.With the root template set to:<feature>    <number>{fme:get-attribute("NUMBER")}</number></feature>The NUMBER value is written in the output. But when I use:<feature>    <number>@round({fme:get-attribute("NUMBER")},3)</number></feature>or even:<feature>    <number>@Evaluate(@round({fme:get-attribute("NUMBER")},3))</number></feature>the xml returned contains an empty string:<?xml version="1.0" encoding="UTF-8"?><feature>    <number/></feature>I could round with an AttributeRounder prior to the XMLTemplater, but although that rounds the values in Visual Preview, the xml output can and does contain the floating point representation errors.I'm guessing this might be a syntax thing but I've had no joy yet implementing this in the editor.(FME 2020.1 b20620)

1056
debbiatsafe
Safer
marta.podsiad
Supporter
marta.podsiadSupporter
asked in Data
Civil3D drawing settting information

Hi, I have a Civil 3D CAD model and need to extract information from the Drawing Settings. However, the available data comes from the Drawing Units window instead, which can be very different from what’s in the Drawing Settings   

463
daveatsafe
Safer
asali
Participant
asaliParticipant
asked in Data
Preserve Domain

Hi,Could anyone please guide me on how to preserve the domain as it is when a feature class with domain for multiple fields is read using SDE reader, and then when it is written to a SDE Writer.When i look at the fields of the output feature class, the domain is empty. How can i overcome this?

444
asali
Participant
jiri.petrak
Contributor
jiri.petrakContributor
asked in Data
DGNV8 writer throws AutoCAD2004Writer error

Hi,I am converting SHP data to DGN using DGNV8 writer dynamically (via FeatureWriter). Everything works fine, but I don´t understand why it always uses ACAD writer first, throws this error and warning:AutoCAD Writer: An error occurred for dataset 'dgn\DI\vydej_di_L1.dgn', in function AutoCAD2004Writer::write. Unknown exception thrown. Aborting methodBADNEWS: ACAD writer: An error has occurred. Check the logfile above for details(D:\code\fme\source\engine\engine_api_impl\cpp\pluginbuilder\dynawch.cpp:129) class StatusInfo __cdecl DynaWriteChannel::putFeature(class STFFeature *&)DESIGN WRITER: Failed to write '3D elements' data through a proxy. Skipping such data for all elements writtenThen it writes everything using DGNV8 writer and the output DGN looks good, nothing seems to be missing. The only issue is that it throws quite big number of errors like this. I publish and use this workbench in FME Flow and the number of errors doesn´t look good in the finished jobs.Any idea how to a

564
jiri.petrak
Contributor
rzel24
Contributor
rzel24Contributor
asked in Data
How to INSERT '0' value as date using MariaDB/MySQL format

Hello,So I have been trying to INSERT a ‘0’ value date using a FeatureWriter with MariaDB/MySQL compatible format into our MySQL DB, but I can’t seem to be able to do it. I keep getting the message the row is skipped because an “Invalid date/time value”.I have tried using many different values for the ‘0’ date like ‘0000-00-00’ , ‘00000000’ with and without the single quotes but I keep getting the same behavior. I’ve also tried using different types of fields (varchar, date, datetime) in FME (but always keeping datetime in the DB) and still the same.I tried using 00010101 and that one was accepted and it did actually loaded into the DB without problems. In the DB the field is Nullable and also accepts cero values, but I can’t seem to get it to work from FME.Any ideas?.Thanks in advance.RZ.-

474
rzel24
Contributor
J
jecollins
asked in Data
MDB_ADO reader: An unknown exception occurred

Hi All,I am attempting to walk an FME script (using desktop 32 bit) over a large folder of specialised access databases called gINT files (.gpj) and move the data to various CSVs. However it will terminate after several hours with "MDB_ADO reader: An unknown exception occurred", and only write several of the CSVs for the files it has read. It works for smaller sample sets, so I don't believe it is a flaw in the script.I have a copy of the log, unfortunately it is a bit large to attach. Any help appreciated.Thanks-Jack Collins

377
neilmyoung
Participant
simeon.simic
Contributor
simeon.simicContributor
asked in Data
Monitoring directory

I need help to make like something called monitoring directory.I have folder, its network based folder, and using FME form or FME Flow or both, i need to make next: Users will add CSV files in that folder, then that file or files will be add in postgreSql database.When users change file, it need to be changed in database too.When users add file , if the file with that name already exists in database  , it needs to be replaced with new one  

553
nielsgerrits
VIP
simeon.simic
Contributor
simeon.simicContributor
asked in Data
Copy multiple CSV files on PostgreSQL database

Hello, i am working on project and i need help.I want to copy multiple csv files from my pc, to postgreSql database.I have a folder on my computer and i want all files from that folder just to copy on postgre.I dont know how many files will be in  folder on pc, so i cannot predict, but they all have some column names sames, some diferent, and some of them even have diferent  number of columns  So basically, i have set up a reader in  FME workbench, in reader i selected which folder i want to read all csv files from, then reader is connected to schemaValidator , and that is connected on writer,  that wrriter should write in postgresqlMy problem is that, when i run that workspace , in my postgre database writer writes right names in right schema  with right data, but with columns from all files I need help, what can i do that if its exactly 5 files in folder that i read , that all 5 files in exactly how they are in folder to be copied on postgresql ? I have been try transformer FileCopy,

514
simeon.simic
Contributor
zeliesch
Contributor
zelieschContributor
asked in Data
Problem reading data from ArcGIS REST feature service

Hi all!I am relatively new to FME but I work with ArcGis Pro since a while now.I am trying to add an “Esri ArcGIS Server Feature Service” reader, which actually contains multiple map servers hosted online on the open government website (public data). When I try to add the reader in FME, I get absolutely no visual preview of the data, it does not open in my window and therefore cannot interact with the data. Even though I get multiple warning messages, my translation log seems to indicate that mapping file generation was successful…Can anyone help me? Has anyone already dealt with that kind of issue? Is it an issue with authentification and authorization?Do you have pertinent resources concerning ArcGis features services as reader in FME? Thank you!

624
zeliesch
Contributor
kanteyemaodo
Contributor
kanteyemaodoContributor
asked in Data
DGN LInes câble TO SHP

 j'ai un fichier dgn contenant les câbles de type polyligne avec leurs extrémités. -je veux extraire les câbles et les convertire en shapefile (.shp) avec les noms des extrémités.??? 

1146
kanteyemaodo
Contributor
tbarker1986
Contributor
tbarker1986Contributor
asked in Data
Extract DWG properties into FME (the equivalent of DWGPROPS in AutoCAD)

I'm trying to extract DWG properties into FME, the equivalent of the DWGPROPS command in AutoCAD. I have added an example screenshot below. The aim is to use this information wthin a workbench on FME Server. I couldn't identify anything in the AutoCAD reader that shows these properties. Does anyone have any solutions that would expose these DWG properties in FME? Any help would be greatly appreciated!

1815
emmaatsafe
Safer
fme_hans
Contributor
fme_hansContributor
asked in Data
save Geometry to a jpg file

Hi,I read sdo_geometry and make a cutout of a small area. I want to save this cutout as a jpf file. Is that possible?I do see jpg writers, but I suspect that some preparation needs to be done before you can save geometry data as jpg.

333
fme_hans
Contributor
offermann82
offermann82
asked in Data
Switch Esri Geodatabase Reader/Writer from traditional Versioning to Branch Versioning

Is it possible to switch the Versioning in an Enterprise Geodatabase (on SQL Server) from "traditional" to "Branch Versioning" without touching FME Workspaces containing Esri Geodatase Readers and Writers (GEODATABASE_SDE)? In a first attempt in the test environment, everything seems to be running like before, after we switched to Branch Versioning of our feature classes.  A quick look in the geodatabase tables seems to prove that the internal fields like GDB_ARCHIVE_OID, GDB_FROM_DATE, ... are filled correctly.  Can it really be that simple? Does FME support the switch to Branch Versioning while keeping the Workspaces untouched? Of course, this would be a dream, but this seems too good to be true.     

3542
bruceharold
Contributor
rmpayaby
Contributor
rmpayabyContributor
asked in Data
How to add coordinate system to an IFC2x3-file?

I'm trying to update an IFC file by assigning it a coordinate system. However, FME doesn't seem to recognize the coordinate system used by the IFC2x3 file.I understand that the IFC file operates with a local coordinate system, but I don’t have information on which specific system it uses.I've attempted the following approaches:CoordinateSystemSetter Reprojector Setting the Writer’s coordinate system to WGS84 UTM33Despite these efforts, the file always ends up with the coordinate system IFC_COORDSYS_0.Does anyone have a solution or a recommended approach to resolve this?

312
rmpayaby
Contributor
mattwilkie
Supporter
mattwilkieSupporter
asked in Data
How to update available EPSG coordinate systems?

The Coordinate Systems Gallery is missing EPSG spatial references we use all the time. How to update the gallery list so that more EPSG codes are available? (e.g. anything on https://epsg.io/)

2559
mattwilkie
Supporter
olivier
Contributor
olivierContributor
asked in Data
Uploading a file geodatabase to FME Server from the web interface

A user is supposed to use FME Server's web interface for running a FME workspace that has been published on it. That FME workspace takes a file geodatabase as input. The user should upload his/her file geodatabase at runtime in the web interface for that workspace. The problem is that the browser will not let the user select the folder defining the file geodatabase. Furthermore selecting all subfiles of that geodatabase will lead to a failed run. What is it that I am overlooking? Thanks in advance!

453
xanduqe
Contributor

Badge winners

  • FME Form Basic Training
    a.hamannhas earned the badge FME Form Basic Training
  • Welcome Back
    jerrylim735has earned the badge Welcome Back
  • Arabic Speaker
    insetmehas earned the badge Arabic Speaker
  • French Speaker
    insetmehas earned the badge French Speaker
  • German Speaker
    insetmehas earned the badge German Speaker
Show all badges
30,837
Posts
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.