Skip to main content

Data

Using readers and writers to extract and load data.

  • 6,963 Posts
  • 25,632 Replies
6963 Posts
oskar
Contributor
oskarContributor
asked in Data
Reading multiple WFS with FeatureReader issue

I have a list containing 93 different WFS URLs, my goal is to be able to read these in one workbench without clutter and for the workflow to handle changes in the URLs for example, a new WFS is released and gets added to the list, I want it to work without having to change or add more to the workbench. Im using FeatureReader to read the format OGC WFS (Web Feature Service) but it wont accept an attribute value as the Dataset, I mean I can put one in there put it does not know what to do with it. If I put the URL directly as the Dataset it knows exactly what to do and I can use the Feature Types to Read and select all an they end up as text separated by a space between each Feature Type. This theoretically means that I could just write a list of every Feature Type I need, use the Attribute Value as Dataset and get results. However, what I really need now is a way to get every Feature Type from the WFS List so that I can ad it in place of Feature Types to Read. Dataset = Attribute ValueF

143
oskar
Contributor
sveeraa9790
Participant
sveeraa9790Participant
asked in Data
GMLfile reading issue

I have tried to read GML file using OGC GML reader but I got some errors. I Tried different version reader still the issues coming all those files.

121
takashi
Influencer
cossy27
Contributor
cossy27Contributor
asked in Data
Creating CityGML Solitary Vegetation Object

Hello all, I am currently working on the conversion of FBX tree model to CityGML SolitaryVegetationObject. I have made reference to the template workflow for creating SolitaryVegetationObject (ref: https://hub.safe.com/publishers/con-terra-lab/templates/citygml-create-solitaryvegetationobject-with-implicit-representation). However, my case is slightly different from the template. First of all, what I current have, is a CSV file containing the tree information (XYZ, DBH, etc) and an FBX file containing one 3D tree model with correct transformation and dimensions (DBH). This FBX file was originally a prototype in FBX format and was transformed into a correct XYZ and DBH. Since not all my data contain crown spread and tree height, they would be estimated by scaling from the prototype with reference to DBH. This estimation would not generate a value for the crown spread and tree height but only a dimension change. What I want to do is to convert this FBX into a CityGML file, appending the

367
cossy27
Contributor
geovision
Contributor
geovisionContributor
asked in Data
Create one dataset from three

I have three datasets containing points which I want to bring down to one complete set. Dataset A contains most of the points but not very accurate in terms of location Dataset B contains less points but is more accurate then dataset A Dataset C contains the least points but is the most accurate of all three sets. So I am looking to create something like the following: When there are three points (dataset A, B and C) within one meter of each other merge the A and B with the point from C. When there are two points, for example dataset A and C, merge the point from dataset A with the one from C. And so on.. A way of ranking I would say. Now I have tried using the NeighborhoodAggregator but I cannot specify which point I want to merge which point. I have also looked at the NeighborFinder but I cannot seem to get that working either. Is it possible to have all three datasets analyzed together? Or do I have to do two at a time?

424
max_h
Contributor
marta.podsiad
Supporter
marta.podsiadSupporter
asked in Data
Expose Feature Attributes from KM

Hi, I have error while running the workbench from How to Expose Feature Attributes from KML Tags or HTML Tables – FME Support Center It failes at point Method 2: XML> 2. Fragment XML Any idea why it does not work. I have not changed any of the settings in that workbench

476
marta.podsiad
Supporter
marta.podsiad
Supporter
marta.podsiadSupporter
asked in Data
KML to Gdb with saving hierarchy folders names

Hi, I have a very complex KML file with many folders and subfolders. I would like to save the folder name or full folder path where each placemark is located — something similar to the "FolderPath" field when converting KML to GDB in ArcGIS Pro. How can I achieve this?

212
marta.podsiad
Supporter
jakethepainter
Contributor
jakethepainterContributor
asked in Data
Error reading KML: Could not determine the root kml file of

I'm trying to read some USFS wildland fire data. There's a link to the kml on their website: http://www.wfas.net/google-earth/wfas_nfdr.kmz When I use the url in the kml reader, it downloads it to a temp file location successfully (I can see it there) and it's unzipped. It's got a folder structure about 5 deep. Then I get FATAL |KML: Could not determine the root kml file of `C:\\Users\\xxx\\AppData\\Local\\Temp\\FME_OGCKML_KMZ_1562607566639_14376' If I go to the temp file location and navigate down to the kml, I can read it in no problem. C:\\Users\\xxx\\AppData\\Local\\Temp\\FME_OGCKML_KMZ_1562607566639_14376\\fsfiles\\office\\wfas2\\dir-obs\\wfas_nfdr.kml

418
saraatsafe
Safer
joy
Enthusiast
joyEnthusiast
asked in Data
Black screen FME Form 2023 and FME Form 2024

Hello everyone, At our organization we work on a virtual machine. When you log on into a virtual machine, you share the virtual machine with other users. Since a few weeks we encounter sometimes black screens when trying to use FME form 2023 or FME form 2024. I googled and the problem is quite similar to Known Issue: FME Workbench 2023 Opens as a Blank Window on a Virtual Machine – FME Support Center, instead it's a black screen. The article mentions about OpenGL versions, however the specs for all our virtual machines should be the same, so I think the cause is different. When you encounter the black screen, you can only hover over the black screen and can't do much else. When you close FME Form and try to open a new FME Form sesssion, you still get a black screen. The only workaround is to log off and log into a new virtual machine where this problem doesn't occur. We already did some troubleshooting and we know one thing for sure It's possible that fme form 2023/2024 initially worke

393
crystalatsafe
Safer
cdog
Contributor
cdogContributor
asked in Data
ArcGIS enterprise update

Hi, I am trying to set up a simple update workflow that doesn’t like me. I am trying to update my enterprise portal with GDB updates and will eventually want to have this run everyday but for now I am just trying to figure out how to update one shapefile and repeat this 30 times. Right now I haver set up a GDB reader going directly into ESRi ArcGIS portal feature service writer to update an existing feature in enterprise and I have got it to write the features but it doesn’t want to publish the feature into portal. Can someone help me figure this out?

377
crutledge
Enthusiast
fme_junior
Contributor
fme_juniorContributor
asked in Data
Summing the population data from several rasters for overlapping polygons

I have two layers: one raster with population data for each cell (blue) overlapping polygons (buffer of 100m) around existing road segments (red) I would like to know the total population contains in each polygon. How do I do this in FME?

854
takashi
Influencer
geckosolutions
Contributor
geckosolutionsContributor
asked in Data
FME Ingesting AGOL json

Hi Everyone, I have this “open” dataset provided by the City of Toronto. I am using a few from the city that are provided in this manner. The only way to bring it into ArcPro and then extract it to a geodatabase is to use the json address and add by url, then use the ‘feature to point’ geoprocessing tool to extract the data to points in gdb format, then use as needed. I need to get this layer into FME from the live feed so I can analyze it and then push it out via other format(s), one of which is AGOL feature layer. For some reason I have been unable to bring it into my FME script via any JSON reader or HTTP caller and I’m wondering if anyone else here will have more luck. The link I am working with is as follows: https://gis.toronto.ca/arcgis/rest/services/cot_geospatial2/MapServer/64 The json link is: https://gis.toronto.ca/arcgis/rest/services/cot_geospatial2/MapServer/64?f=pjson If anyone has the steps to bring the content into FME (geometry and attributes) to have it available for

192
geckosolutions
Contributor
B
bramvr
asked in Data
Optimizing i3s (slpk) LOD

I'm converting 3D data from Postgres to an Esri Indexed 3D Scene Layer (slpk). I'm converting this to an i3s service on Azure (not ArcGIS Online). When loading this as an external layer in ArcGIS Online, the LOD seems totally random, buildings are being visualised but not the way I'd like to see (not based on distance). Is there a way of optimizing this in FME?

44815
vince_j
Contributor
creinier
Contributor
creinierContributor
asked in Data
Data sources in an ArcGIS project (aprx)

I’m working on a data inventory project and need to figure out what data sources are being used in a large number of aprx’s. Is there a way to determine the data sources of an ArcGIS project by directing a workbench to an aprx file path?

273
creinier
Contributor
simeon.simic
Contributor
simeon.simicContributor
asked in Data
Coordinate system

Hi everyone, i have few questions about coordinate systems1. How can i know in which coordinate system is file that i upload2. And how can i reproject that coordinate system into 32634(I have been trying with reprojector but it’s not exactly right when i add project in qgis on my map)3.Which ever coordinate system i tried, it never is 100% match So i need help if someone can tell me what should i do? Thank you in advanced

151
redgeographics
Celebrity
adnanyazdani
Contributor
adnanyazdaniContributor
asked in Data
Field Type - Time only

Hi Experts I would like to suggestion and Guidance Does FME Form has Time as field type I’m using FME 2024 Thanks

326
crutledge
Enthusiast
joy
Enthusiast
joyEnthusiast
asked in Data
[FME Form 2024.2.] Losing Database- and webconnection on new machines

Hello everyone, We just upgraded to FME Form 2024.2.1 and we currently encounter one problem. For the users, the personal database- and web connections were inititally not migrated. Thus we sent first a mail to the users that they coud export the database- and webconnections in FME Form 2023 and then import the exported file in FME Form 2024. It looked like it solved the problem, however when the users logged in the next day on an new machine, they lost all their personal database- and webconnections. We read the following link: Changes to Connection Storage in FME Form 2024 – FME Support Center and it became clear to us that's due to the new way that personal database- and webconnections are stored. They are stored in fme_userconnection.data and the machineID is hardcoded in this file. In our organization we work on a virtual machine and thus also everytime we log into a new machine. It's now clear why the database- and webconnections are not visible in FME Form 2024. I was wondering

1037
joy
Enthusiast
jugoslaviaa
Enthusiast
jugoslaviaaEnthusiast
asked in Data
Accessing Annotation Class Names and Text Symbols from GDB

would like to know if FME can read annotation classes and text symbol information that are defined in the Feature Class properties (e.g., in a File Geodatabase or Enterprise Geodatabase). Specifically: Can FME access Annotation Class Names? Is it possible to read text symbol properties like font, size, color, or placement defined in the annotation feature class?

290
jugoslaviaa
Enthusiast
lorinda
Contributor
lorindaContributor
asked in Data
Using the Metadata Updater for Revision Date

Hi, I’m finding that the date shown for data updates on ArcGIS Online Hub is misleading and not showing the date that we are actually updating the data, has anyone used the Metadata Updater to update any of the dates in metadata? I don’t see it listed in the dropdowns for the updater. Or should I be using the XMLUpdater? Thanks, Lorinda

132
hkingsbury
Celebrity
aashnaparikh
Contributor
aashnaparikhContributor
asked in Data
Duplicate Features Division

Hi, Based on the sample table, I want to implement a logic. Same name, different policy # and same address - Goes to file 2 Same name, different address and different policy number - The first record goes to file 1 and rest of the records go to file 2. All the manual should again go to the file 2. Policy Number New/ Renewal/ Manual Name Address To process Split 123 New John Chris 123 Oxford St Y File 1 110 New Mary Jane 45 Caron ave Y File 1 111 New Mary Jane 360 Caron ave N File 2 112 Renewal Kent Bryan 333 Queen St Y File 1 113 Renewal Kayla Jan 333 Queen St Y File 1 114 Renewal Jung Chen 5 Queen St Y File 1 115 Renewal Jung Chen 5 King St N File 2 116 Manual Ray Ban 5 abc st N File 2 116 Manual Kan Can 6 xyz st N File 2 116 Manual Harry Potter 8 niagara st N File 2 116 Manual Rob Smith 9 India St N File 2 116 Manual Tony Stark 10 peter st N File 2 116 Manual John Snow 11 sandwich st N File 2 2000 New Melanie Joy 275 Rankin Avenue N File 2 2001 New Melanie Joy 275 Rankin Avenue N Fil

594
aashnaparikh
Contributor
crutledge
Enthusiast
crutledgeEnthusiast
asked in Data
fme_geometry undefined when writing to fgdb

I am having trouble writing a few rasters to a raster fgdb. Most are fine but a generate errors of undefined. But previous definition seems fine. bit of a stumper. don’t think it’s a projection issue. Any thoughts? Error on write for 1 or 2 rasters Previous step has all as fme_aggregate and fme_type as raster.

462
crutledge
Enthusiast
dollargis
Contributor
dollargisContributor
asked in Data
FME 2021 Compatibility with Postgres 14

We are using FME 2021 and planning to bump postgres version to 14. Are there known issues with that combination? I have tried to find a compatibility sheet for FME Desktop/Server and Postgres and/or other Software, but did not succeed.

2045
natalieatsafe
Safer
jlw_tyrens
Contributor
jlw_tyrensContributor
asked in Data
STAC Asset Reader

Hi guys! I'm trying to understand how the STAC Asset reader works. In my case, I have a URL to an item, but when I try to access it, it complains that I don't have authentication. When I then check the "Use authentication" box, I only get options related to AWS, and not the service I created. I've used the URL to retrieve data via Radiant Earth Stac browser with my credentials and it works there. Has anyone used this reader to extract data from sources other than the default ones?

944
kailinatsafe
Safer
sunita22
Contributor
sunita22Contributor
asked in Data
data source CSV file each day populated with date stamp, my requirement need to load CSV each day to SQL server

data source CSV file each day populated with date stamp, my requirement need to load CSV each day to SQL server

252
takashi
Influencer
ifawzi
Participant
ifawziParticipant
asked in Data
Reading CSV files with different schemas and writing to SQL server.

Hi, I am new to FME, and I am trying to build an FME workspace that reads multiple .csv files from a folder and write each file to separate table in SQL server. Each CSV file: has different schema should became it is own SQL table Here is what I tried so far: I used Directory and File Pathnames reader I used Feature Reader The <Generic> port of Feature Reader shows no data, I am assuming because the schema of each is different. the <initiator> port give me the correct file name and path.

192
crutledge
Enthusiast
dsign-world
Contributor
dsign-worldContributor
asked in Data
What are valid values for a custom schema's data types in a dynamic wokspace?

The website Dynamic Workflows: Destination Schema is Derived from a Lookup Table > Creating a Table-Based Schema > Geometry Type tells me that valid values for the `geometry_type` column are the values from the `fme_type` format attribute. It provides the full list of values under this link. The link is incorrect. This is the list of FME internal data types. These are NOT valid values from the `fme_type` format attribute and these are NOT values from the `geometry_type` column in the screenshot above in the tutorial. Where is the correct list of values for the `geometry_type` column or the `fme_type` attribute?

351
AliAtSafe
Safer

Helpful Members This Week

Badge Winners

  • Teacher (Grey)
    oskarhas earned the badge Teacher (Grey)
  • Welcome Back
    pgammiehas earned the badge Welcome Back
  • Welcome Back
    casperroelofshas earned the badge Welcome Back
  • Student (Grey)
    oskarhas earned the badge Student (Grey)
  • Welcome Back
    marloeshas earned the badge Welcome Back
Show all badges

Popular Tags

Community Stats

31,079
Posts
117,827
Replies
38,837
Members

Latest FME

Terms & ConditionsCookie settings

Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.