Skip to main content
  • EN

Data

Using readers and writers to extract and load data.

  • 6,701 Posts
  • 24,703 Replies
6701 Posts
L
lundse
asked in Data
adding suffix to list attributes in json-file

Hi, I want to add different suffix to different attributes, 'Filename' in a jason-file. FilenameSuffixABC_123DEF_456GHI_789JKL_102MNO_103PQR_104I have managed building a list with the added suffix, and tried unsuccesfullty to replace the array value using JSONUpdater.  Another approach I tried was to merge the generated list using Featuremerger. I get the correct lists, but are unable to write them as a JSON-file. Any suggestions? {"name":"MAP_SUFFIX","type":"FeatureCollection" ,"crs":{"type":"name","properties":{"name":"EPSG:25832"}} ,"features":[ {"type":"Feature","geometry":{"type":"Polygon","coordinates":[[[1,5],[1,2],[1,2],[1,2],[1,2]]]},"properties":{"filename":"ABC"}} ,{"type":"Feature","geometry":{"type":"Polygon","coor

1
takashi
Contributor
6 years ago
G
geob
asked in Data
Is there any way to convert light-weight Polylines to 3D Polylines?

Hello, I have a 2D City Map .*dxf file. I draped my city map on digital terrain model to obtain a 3D city map. It works fine but when I try to use .*dxf/.*dwg writer, ? see this problem. "AutoCAD Writer: Encountered a feature with inconsistent Z values, and type 'autocad_lwpolyline'. For this type, only geometry with constant Z values, or 2D geometry are supported. Dropping Z values"In my output, the geometry which consist of 'autocad_lwpolyline' does not have Z values. Is there any way to solve this problem?

4
G
6 years ago
R
rajibtechlabs
asked in Data
How to download Google Map in KML .

How to download Google Map in KML .

5
david_r
Influencer
6 years ago
jatoxa
Contributor
jatoxaContributor
asked in Data
Directory reader on Cloud

Hi, I have a wokflow that reads PDF files and validates them. The workflow works great on Cloud as well. However the PDF reader doesn't read all PDF files and seems to be dependent on what software they were published in. So if I try to read invalid PDF's there's simply no output. This is a problem if you want to at least get a list of what files are invalid.   The problem was solved by using a directory reader instead along with a feature reader that reads a list of PDF's. The problem is now that it doesn't work to upload a folder to Cloud in the same drag n drop manner I used before.  Is there an alternative way to know what you can't read?

3
G
6 years ago
boubcher
Contributor
boubcherContributor
asked in Data
write to PDF

How can I write attribute MSG to a PDF as text ID MSG1inside the polygon2outside the area3close to the event area4comment

6
G
6 years ago
S
sebastianblock
asked in Data
OSM PBF Reader: Get nodes in a way

I'm using the OSM PBF reader in FME 2018 to extract highways. I want to split the highway lines into segements, so that each line only contains 2 vertices (the "from node id" and "to node id"). What I am interested in are the underlying node ids.I've tried reading the PBF, both as OSM feature types and primitive types, but neither seems to expose the underlying node ids.I can get to the node ids using the OSM XML Reader, but the XML file is far larger and I will need to process the whole road network for Europe, hence the use of PBF.Looking at the forum seems to suggest that either it's not currently supported or to use Python.Is there a simple way to expose the underlying node ids using the PBF reader?

1
S
6 years ago
G
grahamkeencadba
asked in Data
Connecting to ArcGIS Online via FME Server

I'm unable to run a Workbench on FME Server 2017.0.1.1 that reads and writes to a feature service at ArcGIS Online. I've read the other streams related to this but can't work out the correct workflow, or if we have some problem with the server accessing the online resources for OAUTH2 from inside the corporate network. The workbench works from FME Desktop 2017.0.1.1 with no a problem.I've created an app at ArcGIS Online to obtain a Client ID and Client Secret with the redirect Uri of urn:ietf:wg:oauth:2.0:oob and this all works in Desktop. These same elements were entered into the "Client Information panel" of the Web Service page in FME Server for service "Esri ArcGIS Online". I then created a new Web Connection "g9k Esri ArcGIS Online02" using the  "Esri ArcGIS Online" Web Service defined above. I can then use the "Authorize" button and enter the AGSOL credentials...After clickiong on Sign In", we get a code string back from OAuth2 Approval with "Please copy this code, switch to your

9
N
6 years ago
eric_armitage
Contributor
eric_armitageContributor
asked in Data
Feature reader WFS where clause greyed out

Hi. I'm trying to read features from a WFS service. I can read the features ok based on a spatial filter but the option to use a WHERE clause on an attribute does not seem to be available. Any idea how to fix this?

3
danilo_fme
Evangelist
6 years ago
gazza
gazza
asked in Data
ArcGIS Online Reader Feature Service name

With the ArcGIS Online reader I can get the layer name from fme_feature_type but is there any way to get the name of the feature service that contains that layer?

1
lars_de_vries
6 years ago
jdh
Contributor
jdhContributor
asked in Data
HTML ancestors

I have some html data that's in the structure  <h2>Status</h2><h3>Place</h3><p><a name="1">Name</a></p> <p><a name="2">Name<blockquote><p>Description</p></blockquote></a></p> <h3>Place2</h3><p><a name="3">Name<blockquote><p>Description</p></blockquote></a></p>but the Line Feeds are entirely erratic.  I need to have one feature per name anchor (which is easily enough done with the HTMLExtractor) but I also need to have the corresponding contents of the h2|h3 tags stored as attributes. Normally I would read in the data line by line and use a TestFilter and variables to do so, but since the lines breaks don't match the data structure in any way, I'm not sure as to the best way to proceed.

2
jdh
Contributor
6 years ago
A
alicepalmer
asked in Data
gz to shp problems

I am struggling to get 113.gz os mastermap files to convert over to .shp. I am not that experienced with FME but am finding that the workbench keeps crashing when its reading the files.Does anyone have any tips? I have found previous docs regarding a translator called Fanout - but doesn't seem to exist in my software.I would be grateful for any tips as would like to start moving all of our processes over to FME.Thanks,Alice

4
M
6 years ago
F
florianbenoit
asked in Data
Convert AutoCAD text to SHP attribut

Hello,I have a DWG file with two layers. In the first one I have lines representing a nework of pipes. In the second one I have autoCAD text describing the pipes diameters. I would like to create a shape file representing the pipes with their diameter as an attribute. I linked a DWG reader containing the first layer to a SHP writer to create the shape file representing the network of pipe. How can I extract the text (diameter information) from the second layer of my DWG file and how can I link the diameter to the right pipe in the SHP file ?Thank you for your help,

4
C
6 years ago
F
firsttube
asked in Data
Write DWG with coordinate system and units

Using FME Professional Desktop 2012 SP3, Windows 7 Pro, 64-bit.  I have a workbench that I created to extract geometries in WKB format from SQL Server and write them into an AutoCAD Map DWG file.  I noticed the resulting DWG file has no coordinate system assigned, and the units are set to "unitless".  Although the objects are correctly positioned and the coordinates are correct, the coordinate system is not set.  Is there any way to write the AutoCAD Map DWG file with the coordinate system and the units set?  I have posted this question in the Google group FME Talk.  I am just trying to increase exposure to my question.  Thanks,

5
jneujens
6 years ago
J
jkubacka
asked in Data
How to convert ZFS directory to LAS?

I'm trying to convert ZFS directory to LAS but it's not working. There's an error warning: 'Data type of the file not supported, please update library'. What went wrong?

3
lars_de_vries
6 years ago
H
hallkbrdz
asked in Data
Oracle Spatial to PostGIS conversion speed issue

We are attempting to convert Oracle Spatial databases to PostGIS, and find it is taking a very long time to convert large 4269 SRID lines and polygons at roughly 1-2 per second, with smaller ones at around 10-20 a second. I am also very puzzled while for very long periods it will seemlingly do nothing but burn CPU (no new rows). I've tried both 9.6/2.3 (Azure for PostgreSQL PaaS) and 10.3/2.4 (on-prem) with the same results.For instance, it took 2 hours 54 minutes to convert 2136 rows, or about 12.3 a second.Is the overhead for converting the SDO_GEOMETRY number based geometries to the geography binary type really that high? Any suggestions for getting better conversion speed? Currently using 2017.1 - will a newer version run significantly quicker?

6
lars_de_vries
6 years ago
soothsayers
Contributor
soothsayersContributor
asked in Data
When data is missing, I want to output an empty feature class anyway

Hi, I have an issue which could potentially be solved in different ways, and I want to put this out to see if people have better solutions. It is probably best explained with a simple example:A user requests output of a collection of data - X-points, X-polylines, X-polygonsSo we have a list of what they want. The data will be read from three Postgis tables and output dynamically using the Postgis schema to a FGDB. The user will then be sent the FGDB along with one layer file showing the three datasets.However the geographic area the user has requested data for contains no X-points data, so currently all that is exported is the X-polylines and X-polygons and the layer file contains a broken link for X-points. I am looking for the best way to write an empty X-points feature class to the FGDB so this link remains but isn't broken.My initial thought is too look at the list the user has requested, then compare it to what is read from Postgis. Then I'd have to create a dummy feature

4
lars_de_vries
6 years ago
philipwhitten
Contributor
philipwhittenContributor
asked in Data
How do you get the fme_basename parameter to update for each filename opened in a workspace runner?

I need to copy all of the features in a folder containing many zipped shapefiles to tables in a SQL Server.  The table names in the SQL Server need to reflect the shapefile names.  Currently I am using the Table Name Expression: NatTitl_@Value(fme_basename)When I run the workspace runner that I created, the same table name is applied to every shape file.  Hence, the fme_basename parameter is not updating when a subsequent shapefile is opened.

2
philipwhitten
Contributor
6 years ago
R
ravioli
asked in Data
"directory and file path names" - source folder

Hello,I am at very early stages of incorporating FME into my work, and I am trying to figure out how to unzip all zip files in a folder (and all zip files in subfolders).I was able to unzip files with the "directory and file path names" transformer (**/*.zip) by passing folder and file locations to ZipExtractor. However, That transformer asks me to provide a "source folder" before it can start running. Is there a way to define the source folder to the folder of my .fmw file?I plan to use this same file with different folders and I would like users to be able to copy the .fmw file into another folder and run it there without providing a source folder location everytime.Thanks!

4
R
6 years ago
S
stgt
asked in Data
The Transformer Emailer causes error message on our mail server

The Transformer Emailer causes the following error message on our mail server:  2018-05-17T22:22:23.651384+02:00 warning: Illegal address syntax from xxx.xxx.de[xxx.xxx.xxx.xxx] in RCPT command: <> Unfortunately I do not have any further information about our mail server. I don't care about the error message, but the guys from the central IT pressure for this error message to disappear. Is it possible to make changes to the email settings and if so, where? With REST?

2
S
6 years ago
oliver.morris
Contributor
oliver.morrisContributor
asked in Data
SQL Server JDBC writing error

Hi,Has anyone had an error when writing to sql server from FME Cloud, when I run locally it runs fine. Transferring input attributes onto output feature, this may take several minutes...COM.safe.fme.jdbc.TranslationFailureException: com.microsoft.sqlserver.jdbc.SQLServerException: com.microsoft.sqlserver.jdbc.SQLServerException: SQL Server did not return a response. The connection has been closed.COM.safe.fme.jdbc.TranslationFailureException: com.microsoft.sqlserver.jdbc.SQLServerException: com.microsoft.sqlserver.jdbc.SQLServerException: SQL Server did not return a response. The connection has been closed.Destination Feature Type Routing Correlator(RoutingFactory): COM.safe.fme.jdbc.TranslationFailureException: com.microsoft.sqlserver.jdbc.SQLServerException: com.microsoft.sqlserver.jdbc.SQLServerException: SQL Server did not return a response. The connection has been closed.COM.safe.fme.jdbc.TranslationFailureException: com.microsoft.sqlserver.jdbc.SQLServerException: com.microsoft.s

13
oliver.morris
Contributor
6 years ago
venu
Contributor
venuContributor
asked in Data
Relationship feature class creation for Polygon featureclass with Table error?

I have file geodatabase which contains polygon featureclass and Table with unique ID and common field. While creating relationship class in fme the following error is coming... Error :-------------------------------------------------------------------------------------------------------------------------------- Unable to create table/feature class/relationship class of type 'geodb_invalid'. Creation of this type is not currently supported by FME, and must be done through Esri toolsGeodatabaseRelationshipFeaturesPipeline::GeodatabaseRelationshipFeatures(SortFactory): A fatal error has occurred. Check the logfile above for detailsA fatal error has occurred. Check the logfile above for detailsSome error has occurred closing the pipeline factory, see previous warningNone of the edits made during the edit session were savedSome error has occurred closing the pipeline factory, see previous warningSome error has occurred closing the pipeline factory, see previous warningAbnormal termi

2
bhanu_vd
6 years ago
A
andypcatbcc
asked in Data
HELP: Detecting aggregate features and setting writer type

Please help, at a complete loose end. I have a process that's copying data from one SDE database to another. So far it's working almost okay. My main problem is a featureclass which uses multipoint geometry, and I'm dynamically setting the geometry on the writer (intended as this process will be a child workspace run against different datasources). The only way I could initially get it to work was to tell it to set the geometry based on the first feature, but I have one dataset that has 4-5 multipoint features towards the end of the dataset, so the process bombs out trying to put an aggregate into a simple feature type.

8
davideagle
Contributor
6 years ago
F
frazucca1
asked in Data
Complex KML: how convert them to vector formats?

Hi at all,at https://sentinel.esa.int/web/sentinel/missions/sentinel-2/data-products you may download Sentinel-2 tiling grid in kml format (Sentinel-2 tiling grid kml) (nearly 100 Mb). I tried to export it to SpatialLite and Shape polygons in order to work with inside a GIS more easily.But till now, I haven't success to export geometry Placemark (no polygon nor lines).By inspection them inside FME DATA Inspector, I see a complex structure --> IFMEAggregate made up by polygons and points. What I'm looking for!! But sure due to my illiteracy I can't find a manner to explode and extract and I just get or error on output or strange void features.Any hints?Best,fz

2
jdh
Contributor
6 years ago
K
kat
asked in Data
Use color ramp for pdf output

Good day, I have a position (coordinates) for each day of a month. I would like to display them on a map with a color ramp. e.g. Light yellow for the first day of the month and then getting darker with each day. The output should be a pdf. The pdf styler has no color ramp option and I couldn't get GradedColorizer working properly. Any other solution available?

1
davideagle
Contributor
6 years ago
soothsayers
Contributor
soothsayersContributor
asked in Data
Writing to a GDB using a UUID or Timestamp

I would like to create a folder for output that is named by either a UUID or Timestamp. Within that folder I want to output multiple feature classes within one geodatabase, plus add other items such as layer files loose into that folder.So for example it might look this this:UUIDFolder/ (containing loose lyr files etc.)PLUSUUIDFolder/export.gdb (containing multiple features classes)The problems I'm facing are as follows:- I can create a parameter for UUID/Timestamp, using @UUID() for example, then link my writer to that parameter, but whenever it tries to write out it creates a brand new stamp.- I can create an attribute from the parameter and use a fanout expression with the attribute value but this always fails to write. And it seems impossible to populate a new parameter from the attribute.I'm trying to do this all in one workbench rather that export attributes as parameters using JSON to a second workbench.It is failing in both Desktop and Server.Any ideas?

3
takashi
Contributor
6 years ago

Badge winners

Show all badges
29,721
Posts
Terms & ConditionsCookie settings