Skip to main content
  • EN

Data

Using readers and writers to extract and load data.

  • 7,089 Posts
  • 26,099 Replies
7089 Posts
T
timewatcher
asked in Data
Read multiple file and save on multiple file.

Hello, I have points on many file on a side.  On the other side i have a directory with multiple cad file each cad have 1 boundarie  I make a workspace with a buffer/clipper and a reader to read all points.  I don't want to insert each boudary and insert writers with the same name.  I want to scan the directory for cad file, For each file this write a csv file with the point.  All of this work perfectly for 1 file but is a way to make it for each file ?

20
gio
Contributor
A
aleksandar
asked in Data
Unable to inspect GML file

Hello,    I got GML file trough translation of SQLServer Spatial into GML, but I can not inspect it, or open it to graphicaly see features. XMLValidator passes the file against schema. Here is a link to my resulting GML file:  https://drive.google.com/file/d/0B8kUf-xMW81XY09lV2prQ3VUVlE/edit?usp=sharing

50
A
L
longhornfan
asked in Data
I have an Excel reader with counties. I want to filter another layer based on the counties in the Excel file

Reader 1: Excel Data with a County column  Reader 2: Ownership Parcels for Northeast United States with a County column  I want to filter out all counties that are not listed in the Excel Data. This will greatly reduce processing time since the Excel file never has more than a 100 records daily. While the Ownership always has millions of records.  If today's Excel Data has County A, B, and C then I only want Ownership Parcels that are in County A, B, and C. I hope that makes sense.  What's the best way to filter the Ownership Parcels with the everchanging Excel Data by the county field in the Excel Data?

90
L
B
bas
asked in Data
FME script terminates/crashes when attempting to write text.

Hi All  I have made a FME script that loads data from an SDE Oracle database and from various csv files. The dat are manipulated and finally summarize in an attribute as a text in XML format . Finaly the text is written to disk using a text writer.  This script has been running in a weekly batch job succesfully until 2 weeks ago. Since then, when the script is run in batch or in the workbench, as soon as the writer attempts to write the the text file, a windows message appears saying that FME.ex stopped working and needs to close. After that another error appears with a runtime error.  When I run the script in the workbench the last lines in the logwindow are like this.  unable to realloc 73638411 bytes This application has requested the Runtime to terminate it in an unusual way. Please contact the application's support team for more information. i = 72 String http://www.safe.com/fme/xquery/functions is still in the pool Zorba Internal Fatal Error in ..\..\src\store\naive\string_pool.c

80
B
E
erin
asked in Data
Exclude records based on another dataset

Good morning, I am using SQL Creator to query a table in SQL Server. This table has about 45M records. My VPN only stays connected for 24 hours, and my workspace needs more time to run.  Last night, I was up to 27M records written to text. In lieu of starting over, I was hoping there is a way to run the workspace again, but exclude the 27M records already written. There is a unique field in the table, so it would just be a matter of excluding based on that one field. I cannot figure out how to do this. I do have a list of the unique IDs that were already written in a MS Access database table.  Is there a transformer that would allow me to do this?  Thanks! Erin

280
david_r
Celebrity
R
robyn
asked in Data
Enterprise Geodatabase reader/writer failing to load client libraries

Hi All, We have received a number of cases in support recently where a Geodatabase_SDE direct connection to an enterprise Geodatabase sitting on SQL Server has failed due to the inability to load the client libraries. Once we have determined that it isn't related to a 32 bit vs 64 bit issue the problem manifests as either the error message  "fme.exe - System Error The program can't start because sdemssqlsrvr102.dll is missing from your computer. Try reinstalling the program to fix this problem."  or  "Could not open the Enterprise Geodatabase. The error number from ArcObjects is: '-2147155644'. The error message from ArcObjects is: {Cannot connect to database because the database client software failed to load. Be sure the database client software is installed and configured correctly.} "  In all cases these workspaces also contained a reader/writer that was accessing some other Microsoft format - e.g. Personal Geodatabase, MS SQL Server non-spatial,

80
R
J
jroge087
asked in Data
Leica DXF to GIS feature class

Hello,  I am working with DXF files which are outputs from Leica GPS units.  When I bring the DXF into FME as a reader it displays the point data and attributes as seperate individual features.  What is the best tranformer to use to combine these into one point feature with an attribute from each one.  My goal is to end up with a point feature in a file geodatabase.  Thanks in Advance        

120
gio
Contributor
R
robertdbuckley
asked in Data
HTTPFetcher to pdf format

How can I use the HTTPFetcher to download pdf documents from a URL? I have successfully managed it with Images using the Imagefetcher and writing to JPG, but what do I do for PDFs?   

280
takashi
Celebrity
F
flyinace
asked in Data
How to change CSV column type

I have a large CSV and apparently FME tries to identify the column type when a CSV reader is added to a workspace. I believe this is causing a failure in a writer when FME identifies a column as "number" but there are a few null values in the column. It appears the CSV writer crashes when it wants to write a "number" but the value is "null". How do I get FME to stop identifying columns that have null values as "number"? I am doing no calculations on this column; it just needs to get pushed through to the writer as it is found in the reader. As a last resort (although painful) how do I change the column type manually to successfully deal with null values?

640
F
A
aleksandar
asked in Data
Make single feature member instead of many of them

Hello,  I'm translating MS SQLServer Spatial to GML file by mapping SQLServer views I created for translation. But I have one problem. I need to have something like this (just an axample, not real code): FeatureMemberCollection   FeatureMember     Town     School     Class     ...   /FeatureMember /FeatureMemberCollection  But I have FeatureMemberCollection   FeatureMember     Town   /FeatureMember   FeatureMember     School   /FeatureMember   FeatureMember     Class   /FeatureMember   FeatureMember     ...   /FeatureMember /FeatureMemberCollection  Because of this my xslt creation goes wron way. What can I do, or where I'm wrong?

70
gio
Contributor
R
robertdbuckley
asked in Data
How to download images from API?

 I have to download images which are made accessible using a password protected api interface.  Using the following URL I can retrieve an xml dokument showing the element id´s which contain image data:  https://www.THESERVICE.com/api/media-get.php?&user=?????&pw=?????&crs=EPSG:4236&version=current&bbox=9.3131,52.5912,10.2535,53.2265&type=1  From the list, I can then call the images using the following query:   https://www.THESERVICE.com/api/medium-file.php?&id=73298&user=?????&pw=?????   Could anyone offer a way to enable me to download all the images to a given directory and rename then according to the id?   

760
david_r
Celebrity
jamal
jamal
asked in Data
Choosing a particular layer when converting CAD file to ESRI feature class,

Choosing a particular layer when converting CAD file to ESRI feature class,  I couldn’t figure out how can the end user choose a particular layer when converting CAD to feature class.  It appears the “feature type” can’t be set as “user parameter”  Then how the user can select a particular layer when converting CAD to feature class?              Thank you  Best  Jamal

40
jamal
P
pvaziri
asked in Data
Invalid Schema Directive Value - CSV Reader

I'm getting the following message:  CSV Reader: Invalid schema directive value 'number<comma>15<comma>0<comma>number<comma>4<comma>...The expected syntax is SCHEMA <typename1>,<width1>,<precision1>,...<typenameN>,<widthN>,<precisionN>.  Using schema scanner instead  As you can see, no typenames are being furnished.  This happens reading a pipe- or tab-delimited CSV file.  User Attributes are parsed and assigned correctly when opening the reader properties, so I don't understand the error.  Also why all the <comma> indicators instead of a literal ','?  Later on, I'm getting many attribute could not be written errors associated with certain attributes. The containing feature has been dropped...outputs, writing to a file geodatabase as a dynamic writer.  Where can I go from here and/or what am I doing wrong with my simple csv -> file geodatabase translation?

60
gio
Contributor
K
kflaw
asked in Data
Converting raster KMZ to other image format

I have a bunch of raster KMZs that I want to conver to another raster format like geotiff. I inserted a geometry filter to just filter out the raster to write to geotiff. When I write out one file as a test, I am getting 17 different raster images as output. Why is this and how can I just get one image file as output? I tried to use mosaic raster before writing out to geotiff but it did not give a good result.

690
M
V
vikram
asked in Data
Error from Fme Server Notifier

Hello I am using FME Server Notifier in my workbench . The topic is subscribed to email .  But it is throwing below   BODY @EvaluateExpression(STRING_ENCODED,<at>Value<openparen>_job_id<closeparen><lf><at>Value<openparen>_timeStarted<closeparen><lf><at>Value<openparen>_timeFinished<closeparen><lf>,FMEServerNotifier_2) OUTPUT OUTPUT FEATURE_TYPE FMEServerNotifier_2_OUTPUT OUTPUT REJECTED FEATURE_TYPE FMEServerNotifier_2_<REJECTED>' is incorrect.  The parameter to TOPIC must be a constant value, a value-of (&) attribute, or an FME function call  Below is my content for the notification @Value(_job_id) @Value(_timeStarted) @Value(_timeFinished)  Any ideas why this might be happening ?

70
M
T
takashi_huylo
asked in Data
Saving shp file

Hello guys,  Could you please tell mem how could i sva shape file in some folder with another name? The trouble is that while i try to do this the programm creates the folder with defined name but the name of the file stays the same.  Thx, Siber-DEV.  

110
gio
Contributor
V
vikram
asked in Data
test coordinate system between 2 feature class

Hello,   I am trying to compare corrdinate system between 2 feature class . If corrdinate system is different then I want to terminate the process   In this below workflow my tester always gets failed even when both feature class are in same coordinate system .   Can anyoone point out what am I doing wrong .    

80
gio
Contributor
R
ricky
asked in Data
how to convert a tiff and a world file into a geotiff file - the sequel

I saw the post about "how to convert a tiff and a world file into a geotiff file". Call me lazy but I'm no FME hotshot. I have a lot on my plate and I need to convert a tiff and world file into a geotiff in a short amount of time. Any chance you could provide me with the actual solution to do this conversion?    

430
fmelizard
Safer
K
kam
asked in Data
Reading metadata from an ArcSDE Feature Class

I want to map the version last updated date from the metadata to a new field in the feature class.  When you save a feature class as an ESRI layer file it also creates an xml file in the same location.  Previously I would locate the feature class as a layer file saved in my local directory then read in the XML file using an XML reader, flatten the metadata and set the elements to match to 'metadata'  Then I can easily locate the attribute that contains the version date etc - and map this right over to a new field in a feature class.  But now where I have feature classes in SDE not saved as layer files I was wondering if there was any way of following this same process? How do I read that XML and after reading it, how would I carry out the xml flattening and elements to match when it is not using an XML reader?   Thanks

480
Y
B
barani
asked in Data
Writing DTM as TIN surface into CityGML

Dear FME users.  I want to write TIN surface into CityGML as TINRelief (tin) and I am not sure how to do that.  I am using in the CityGML GeometrySetter the following parameters:  CityGML Lod name: extent Feature Role: reliefComponent  Since there there is no option to select lod I set citygml_level_of_details to '0'. Hope that is fine.  Finally I am setting CityGML writer Feature Type Name as ReliefFeature.  When I run the workspace CityGMLWriter gives me this worning:  CityGMLWriter registered 1 messages of type:  The 'extent' citygml_lod_name trait is invalid for 'ReliefFeature' features, the geometry will be ignored, valid values are {}.  and there is no geometry being outputed into CityGML file.  I would appreciate any help very much.  Thank you.  Regards, barani

840
B
J
johng
asked in Data
User Parameters from file

Hello,  Is it possible to read user parameters from a text/xml file. I need to store user parameters in a file which it turn read by a workbench to set a connection info which will be used to connect to db.  Regards,

1460
takashi
Celebrity
A
Anonymous
asked in Data
FME Server Login from localhost

Is there a way to protect the login page for FME Server 2014? I would like to allow login only from localhost, that is, on the machine itself. You can do stuff like this with Apache. Is it possible with FME Server 2014?  Thanks

130
redgeographics
Celebrity
P
pierre
asked in Data
google fusion error 503 & 443

Hi,  I have a workspace runner calling some workspace to export Google Spreadsheet data to Google Fusion. But, sometimes, I have an error like this :   Google Fusion Tables: Unknown SSL protocol error in connection to www.googleapis.com:443 Google Fusion Tables Writer: Unable to create the connection.  Please check your login information, table names and table IDs.  (Error was: 'Unknown SSL protocol error in connection to www.googleapis.com:443 ')  Or like this :   2014-07-15 10:14:12|   4.8|  0.2|ERROR |Google Fusion Tables: HTTP error code : 503 2014-07-15 10:14:12|   4.8|  0.0|ERROR |Google Fusion Tables: HTML Response: 'Internal error. Please try again.' 2014-07-15 10:14:12|   4.8|  0.0|ERROR |Google Fusion Tables: CommitTransaction failed 2014-07-15 10:14:12|   4.8|  0.0|ERROR |Google Fusion Tables Writer: Error inserting features to table 't_facturation'  The next workspace will run without problem. There is no network problem (tested on 2 servers).

20
P
L
l_marsvolta
asked in Data
Rename material and texture from Wavefront OBJ to OBJ

Hi everbody,  i just want to rename the textures associated to my OBJ files. I have for example a file called AB0001.obj and four textures :  noname001.jpg,noname002.jpg,noname003.jpg,noname004.jpg  I want to rename my textures in order to have :  AB001_1.jpg, AB001_2.jpg, AB_003.jpg,AB_004.jpg  So i used "Appearance extractor" and "Appearance setter" but without success.  Thanks!

1020
gio
Contributor
Y
yamabushi
asked in Data
Read in feature class from output of a transformer

Hi, I am trying to read in a feature class, the name of this feature class depends on the output of another reader/transformer.  I may be missing something simple (like schemas) but I can't seem to find this.    More specifically I am reading in feature classes from a database, I want to compare the names to a diffTable in the database to see if there are new feature classes or edits that have been made.  I am stuck at the part where I have say 2 new feature class (names) and I want to read those and write them out to another database (just as they are, in 2 seperate feature classes).  I have access to desktop and server.  I was thinking of chaining or using server to run them.  Thanks for any advice.

130
gio
Contributor

Badge Winners

Show all badges

Community Stats

31,801
Posts
120,812
Replies
39,424
Members

Latest FME

Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.