Skip to main content
  • EN

Data

Using readers and writers to extract and load data.

  • 6,697 Posts
  • 24,694 Replies
6697 Posts
jfd553
jfd553
asked in Data
PostgreSQL connection: Unknown type with OID xxx

I am writing features in a PostgreSQL database. Everything seems OK except for the following warning...  INFORM|Truncating existing table 'public.user_locations'... INFORM|Appending to existing table 'public.user_locations'... WARN  |PostgreSQL connection: Unknown type with OID 24635, treating as char(-1)  What does that warning means and what should I do to get rid of it?  Since I am about to load hundreds of GB of data, I just want to be sure everything will run smoothly after the upload...  

3
david_r
Influencer
9 years ago
D
djabx
asked in Data
EsriReprojector FME Server

I've made a workbench witch project some geometry with EsriReprojector.  It work very well on my desktop (FME 2013 SP4 build 13547 WIN32) but on the server (FME Server 2013 build 13547 win32 on a Window 2008 R2 Enterprise x64) it crash without any log (with the exact same geometry).  Any help will be appreciate !

2
D
9 years ago
G
george
asked in Data
can an FME server host an OGS service?

Can an FME server host any map services (e.g. OGC's wms services)?

1
david_r
Influencer
9 years ago
B
buniekwawa
asked in Data
access mdb - Provider error `(-2147467259)

Hello, i've have problems with reading some access mdb files.  I have win7 x64, office x32, fme x32  when i try to read mdb received from client, i get error:  MS Access Reader: Connection failed.  Connection string `Provider=Microsoft.Jet.OLEDB.4.0;Jet OLEDB:Engine Type=5;Data Source=C:/mbu/database.mdb;User ID=admin;Password=********'. Provider error `(-2147467259) Unrecognized database format 'C:/mbu/database.mdb'.'  I can manualy open mdb in access and save as accdb, then it will read in fme workbench, but i'm going to do it many times, so i'll be greatfull if someone can provide solution. i've tried reinstalling fme, access database engine - didn't help.

0
B
9 years ago
J
johng
asked in Data
Excel Reader

Hello,  I would like to read excel files containing one sheet only. But columns headers may not be same apart from 2 columns Lat and Long. I am using VertexCreater to create point from Lat,long.  e.g. Excel1.xls contains A,B,C,Lat, Long Excel2.xls contains C1,C2,C3,C4,Lat, Long,C5,C6 Excel3.xls contains Lat,Long,A1,B1,C1  How can I read excel file with dynamic scheam and retain all attributes for destination format.  Cheers.

1
takashi
Contributor
9 years ago
jamal
jamal
asked in Data
Choosing parameter value from a list stored in an Excel field,

Choosing parameter value from a list stored in an Excel field,  I couldn’t figure out how the a parameter value can be selected from a list saved in an Excel table  For example, in the screenshot below, I wanted to choose the “PlanType” value from the shown Excel table     How this might be accomplished?   Thank you  Best  Jamal

2
jamal
9 years ago
J
johng
asked in Data
FME KML file name (bit strange behavior)

Hello, I have observed that FME behaves differently if we run a workspace in Workbench and after publishing to server.  I have a workspace with Shape Reader reads multiple files and converts to KML using GENERIC writer. If I fanout the dataset on fme_basename it works perfectly in workbench, but if same workspace is published to server and run as service it creates output kml as "output.kml"  e.g. Shape reader reads Test1.shp & Test2,shp Creates output by fanning out on fme_basename Output KML : Workbench creates Test1.kml in Test1 folder & Test2.kml in Test2 folder - PERFECT On server : "output.kml" in Test1 folder and "output.kml" in Test2 folder  Could anybody explain this behavior?  Cheers John

1
davideagle
Contributor
9 years ago
G
george
asked in Data
How to publish workspaces to FME server without using FME desktop

It is encouraged to automate the deployment proces for production in our organization. Deployments are usually done by designated people especially for production development.  Can you give me any suggestions on automatically publish workpsaces (many of them) to FME servers?  like using scripting, .......

1
davideagle
Contributor
9 years ago
F
frost
asked in Data
Calculate builiding elevation from lidar

Hi,      I'm looking to calculate a building elevation and output the attribute to an sdf.  What I have is xyz files for bare earth, xyz of full feature (includes all elevated points) and an sdf of building footprints.      I need to somehow get the difference between the full feature point elevation and the bare earth elevation, then clip out all points that are outside of the building footprints, and lastly create the building footprint sdf.    I've never done this in FME before, so it will be a new process.    thanks.   

1
davideagle
Contributor
9 years ago
jelle
Contributor
jelleContributor
asked in Data
Views Oracle to SQL Server

Fellow FME'ers,   we have migrated data from Oracle Spatial to SQL Server. It works like a charm, without any transformer in between.   Does anybody have an idea how we could convert the views, defined in Oracle, to SQL Server using FME?   best regards, Jelle

3
G
9 years ago
G
george
asked in Data
FME engine licenses vs. ArcGIS licenses

Hi All,  If I have two engine licenses, does it mean I need two ArcGIS licenses in order to run jobs (connecting to SDE) in parallel (make full use of 2 engines)É  Thanks,  GS

0
G
9 years ago
jamal
jamal
asked in Data
Accessing published workbench from client machine,

Accessing published workbench from client machine,  I have published simple workbench that converts CAD files to shape files but couldn’t figure out how this workbench can be used from the client side.  The link below doesn’t work from the client side  http://10.110.62.82:8080/fmedatastreaming/Samples/CAD_TO_Shape.fmw?SourceDataset_GENERIC=%24(FME_MF_DIR)Ayman_02.dwg&FEATURE_TYPES=Parcels_Geometry&DestDataset_SHAPE=D%3A%5CS1  What might be the issue here?   Thank you  Best  Jamal        

2
jamal
9 years ago
jamal
jamal
asked in Data
The ArcGIS FME keeps the old local IP,

The ArcGIS FME keeps the old local IP,  We have switched the local IP to real IP. It appears that the FME Servers still generating URLs with the Local IP.  What might be the issue here?  How to set the FME Server to generate URLs with the Real IP?   Thank you  Best  Jamal    

1
K
9 years ago
G
gschleusner
asked in Data
Postgre Read and Write SSL

Is there a way to connect to a Postgre server  using an SSL connection?  Thanks,  Greg

3
fmelizard
Contributor
9 years ago
M
matthewbrucker
asked in Data
ESRIJSON - How to Query more than 1000 records from ArcGIS Feature Server

I am trying to query more than 1000 records from a public, online ArcGIS Feature Server. I've used the help reference showing the way to use the ESRIJSON query method here (http://fmepedia.safe.com/articles/How_To/How-to-read-data-from-an-ArcGIS-Feature-Server-with-FME), but all the documentation I can find says that the limitation from ArcGIS server is limited to returning 1000 features. Note, I was successful in pulling using the ESRI-JSON reader in FME to create the first 1000 features.  My web services knowledge is a little weak and I'm wondering if someone knows how to loop/iterate the request to keep returning more than the 1000 records?

2
M
10 years ago
K
kelldell
asked in Data
Geodatabase Writer - adding fields/attributes

I am writing to the esri Canadian Municipal Data Model (geodatabase) and cannot get the attributes I added to user attributes to write to the geodatabase tables and populate. Can anyone help with this?

0
K
10 years ago
M
mattyryan
asked in Data
WFS Doesn't Run Workspace Transformers or Post SQL

Hi Everyone,  I have a workspace which contains a reader for a WFS and a writer for Oracle. I have an issue where if the WFS doesn't read any data then it does NOT run any of the transformers or the pre/post sql scripts for the writer.  This is a problem because if the data has changed we want out database to reflect that. This means that if there is NO data in the WFS we want our table to have no data in it. I can write python to have it delete the data from Oracle but then I also have to kick off some procedures and run some other SQL and realistically that should be in the post SQL. Additionally this would add a third party dependency on a python module which we don't want.  My questions - Is this a bug or is this how it is meant to function? Is there possibly a setting I'm meant to change for WFS to run the workspace even when no features are read?

2
K
10 years ago
J
johng
asked in Data
KML filename in Zip Output

Hello,  I have a very simple workflow which reads ESRI Shape file  and converts it to either KML, MIF, MITAB etc. using generic writer. To create a compressed output file I appended .zip to DestDataset.  I am facing an unusual behavior in case I try to convert to KML. When I select KML as output format it creates zip file with kml but the file name inside the zip is comping as something _auto_zip_<some numbers>.kml. This is the case only for KML for MITAB, MIF file names are same as source shape file.  e.g. If source shape file is test.shp and output format is MITAB then it created zip with test.tab, test.id etc but if the output is KML then it create zip with _auto_zip_1256789651236_1234.kml Could anybody explain is this behavior as expected?  Cheers John

1
takashi
Contributor
10 years ago
B
brookphillip
asked in Data
No response when trying to read an http link with CAT as the reader

I read txt, shp, and csv files zipped a url all the time. For some reason, when I set Column Aligned Text (CAT) as the reader for fixed width files, FME cannot see the file. What's going on here?   btw, it's a .dat file inside a zip.   Thanks,

0
B
10 years ago
A
ali
asked in Data
Text file reader

Hi Everyone,  I've a little question to ask, How do I read a list of files from a text file (Throught text file reader), which is placed in the parent folder of another data reader in the same workspace.  I can do it manually, but I want the text file reader to automatically read a certain file from the parent folder. The attributer creater is not willing to expose my created attributes as private/public variable, which i wanted to use as an input to Text file reader.  Thanks

6
A
10 years ago
jamal
jamal
asked in Data
Enlarging the “text” that appears in the “FME Data Inspector” from the “workbench”,

Enlarging the “text” that appears in the “FME Data Inspector” from the “workbench”,  I couldn’t figure out if there is a way to enlarging the “text” that appears in the “FME Data Inspector” from the “workbench”.  For example, the “text” output below appears in small size, how to enlarge its size from the “workbench”?     Thank you  Best  Jamal

3
jamal
10 years ago
J
jonathan
asked in Data
Generic writer and Overwriting datasets

Hi Folks, So I've inherited a workspace that has a generic writer in it. One of the formats it writes is File Geodatabase (ArcObject).  One thing we've noticed is that FME seems to append to the GeoDatabase rather than overwriting it. As it's a Generic writer, is there a way to specify Overwrite = true? I.e. with an attribute? The doc's don't seem to say what the default is so I'm guessing it's False based on what we're seeing.  Using FME 2013.  Thanks, Jonathan

2
J
10 years ago
hoa_le
Contributor
hoa_leContributor
asked in Data
Import raster image to Postgis

Hi everybody,  I have a raster image(TIFF file). I want to import it to Postgis. How can I do? I tried it : - Reader: read tiff file , - Writer: PostGIS Raster. But I got error. Could you tell me please how can I do? Thank you     

9
hoa_le
Contributor
10 years ago
P
phil
asked in Data
Generic (Any Format) Reader and Schema

Can the Generic (Any Format) Reader handle an arbitrary schema or is it limited to a fixed schema?  This quote from the FME 2014 documentation suggest that it should handle arbitrary schema: "... and is very useful for solving problems when the input format and schemas are not known at design time."  However I have a workflow with Generic (Any Format) Readers that throws an error when I change one or more of the input datasets when testing. The error is:  "During translation, some features were read that did not match a reader feature type in the workspace. This can happen if the reader is changed, or a reader feature type removed or renamed."  "The following unexpected feature types were read:" .....  My workflow checks for a number of data problems.  There are two datasets: a reference dataset and a dataset to check. The reference dataset and the one to check are each used by a Schema (Any Format) Reader so I can check schema differences and a Generic (A

3
M
10 years ago
A
aquamarine
asked in Data
How do I create contours from an XYZ file of 1m regular horizontal intervals?

Hi, I have an XYZ file with 1 million points at regular 1m horizontal intervals.  I want to convert this to contours, however when I tried ContourGenerator and SurfaceModeller transformers, all I got was a square polygon showing the extent of the XYZ points.  Does anyone know what I'm doing wrong or if I should be using a different transformer?  Thanks,

3
M
10 years ago

Badge winners

  • Integrate Spatial Data
    oisinhas earned the badge Integrate Spatial Data
  • FME Form Basic Training
    oisinhas earned the badge FME Form Basic Training
  • I Dig JSON
    bwnhas earned the badge I Dig JSON
  • Teacher (Grey)
    amelchiorrehas earned the badge Teacher (Grey)
  • FME Form Advanced Training
    ashokme99has earned the badge FME Form Advanced Training
Show all badges
29,703
Posts
Terms & ConditionsCookie settings