Skip to main content
  • EN

Data

Using readers and writers to extract and load data.

  • 7,082 Posts
  • 26,077 Replies
7082 Posts
M
mike707
asked in Data
FME doesn't find existing Feature Service in AGOL then fails because it already exists / Update Token

I have a series of WorkSpaces that run in sequence (via WorkSpace Callers). I have the main WorkSpace scheduled to run a few times a day, and over the last few weeks they've been running fine.Today I noticed some of them began to fail, and throw the same error that I would sometimes get when I set them up the first time, a few months ago:Encountered an unexpected error. The error code from the server was 'none' and the message was: 'Service name 'SERVICE' already exists for 'xxxxxx''.  Of the 5 Workspaces that write to AGOL, 4 started to fail. One is working fine. All have the same settings, use "Insert" and "Truncate".Looking deeper in to the logs, I found that for all of the writers that are failing the following is happening (some names redacted):  ArcGIS Online Feature Service Writer: Requesting portal informationArcGIS Online Feature Service Writer: Getting list of user content items in 'User_org'ArcGIS Online Feature Service Writer: Searching for all groups in the organization ac

3176
M
X
xavier_49
asked in Data
Error "edgeCurves_[edge].has_value()" with reading or writing a IFC file

 Hello everybody, i try to integrate an IFC File of a building into a gdb File.Someone from ESRI France has read the file correctly and put in gdb file with Data Interoperability and ArcGIS Pro 2.5.I don't know very much this BIM format but i have some problems to read it in FME 2020.1 (build 20596). step1 : all the file is ok except the class IfcWallStandardCase wich has a problem just in reading it with an inspector.with 2D, no problem, but 3D reading, it failed : Construction de la commande OSG Graphe de scène caught exception: stk::ex::FatalError[0] Expression: edgeCurves_[edge].has_value()[1] auto __cdecl cycletopology::Graph::extractCircuit::<lambda_62cd9dca1bd617e3c133d0725ef0952f>::operator ()(const struct stk::ex::index<unsigned __int64,struct cycletopology::Graph::Edge>) const(C:\\code\\fme\\foundation\\framework\\geometry\\cycletopology\\src\\graph.cpp:167) step 2 :i have tried to write the 1400 items into a gdb file (multipart) with a geometry filter on soli

290
X
geo-x
Supporter
geo-xSupporter
asked in Data
Create a published parameter with date

Dear FME community.I've to use the dynamoDB connector with a partition key like a specific date (for example : today - 5 days).For that I only can using a published parameter :But in this case I don't have access to the date function :Do you know how can I do that ?For information, I've FME Server.Thank you for your help.

1772
geo-x
Supporter
lazarlubomir
Contributor
lazarlubomirContributor
asked in Data
Upload media files to ORACLE

Hello everyone,is there any possibility how to upload mediafile to ORACLE (e.g. PNG, JPEG etc.) via FME?Thank You so much!Lubo

200
lazarlubomir
Contributor
umapper1
Contributor
umapper1Contributor
asked in Data
12d xml write error

Hi all!I'm testing the new 12d xml reader and writer transformers. I've started with a simple static reader to writer workspace to see if a basic conversion works with no manipulation but hasn't been successful. I believe it requires some extra manipulation. Had the following error when trying to get back in to 12d model software:I don't know how to read the xml so I don't know what's going wrong. Below is an image of my workspace and attached fme template file. I'm also curious whether/if FME can write back to 12d vertex attributes as well as string attributes. It certainly reads them however puts them in to a list. I wonder if this complicates things on the 'write' side of the process. Any help would be appreciated.Using FME 2020.

480
jovitaatsafe
Safer
L
luke.giles
asked in Data
S3Connector System Authentication

I am trying to connect to S3 on FME Server using the S3Connector System authentication. Our FME server is using the 6th method as per https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-quickstart.html#cli-configure-quickstart-precedence : Instance profile credentials – You can associate an IAM role with each of your Amazon Elastic Compute Cloud (Amazon EC2) instances. Temporary credentials for that role are then available to code running in the instance. The credentials are delivered through the Amazon EC2 metadata service. For more information, see IAM Roles for Amazon EC2 in the Amazon EC2 User Guide for Linux Instances and Using Instance Profiles in the IAM User Guide. I have confirmed I have access on the server to reach the S3 bucket specified directly via the CLI, but when I run through FME Server I get the following:Attribute(encoded: UTF-8) : `fme_rejection_message' has value `S3Connector: Required Amazon S3 credentials were blank or could not be found. Please note 

1763
G
N
nfink
asked in Data
AutoCAD 2021

The reader documentation for acad list 2020 as the most recently supported version, just checking in to see if this is true for the FME 2020.1 release and when AutoCAD 2021 will be supported. 

70
jovitaatsafe
Safer
J
jmacalister
asked in Data
Create a Geotagged Photo?

I have some legacy images (jpg) with a SHP file with points for each image.Is there anyway to create geotagged photos from these?

430
J
M
mladenamovic
asked in Data
Read all tables from POSTGIS schema

I need to read all tables from Postgis scehma. With reader I got only tables which has geometry or which has tables related to geometry tables with foreign keys.Is there some option to read them all, or some Python script which can help me with this?

381
david_r
Celebrity
B
brianapeters
asked in Data
GUID Generator assigning Primary key automatically

When using the GUID generator i would like to assign that column as a primary key without opening the writer and assigning it? Is there a transformer i can use for this? (Would be a great addition to that transformer.)

1770
B
L
lola
asked in Data
overwrite existing attribute

I have a .dxf with points and lines within different layers and blocks. For one layer of data, I need to delete the attribute value within the block and insert in it the geometric value z. Which transformer is best to use?

660
L
B
bshiro
asked in Data
Converting KML generated from Google Earth to Cesium 3D Tiles

I follow step 1 of https://knowledge.safe.com/questions/101352/write-b3dm-from-kmz-with-textures-to-cesium-3d-til.html, selected 'Placemark' feature and run.It generated 'tileset.json' and 'data/data0.b3dm' and I tried loading it in Cesium but I couldn't see anything. What is the right way to do to convert KML to 3D tiles?

930
B
C
cardiofaz
asked in Data
Converting AGS Files as a GIS Layer

Hi all,Just been on working on converting AGS Files to GIS format.This is the first time I have gone through this and it is a difficult file to convert.Just wondered if anyone has any experience with this and are there any steps that have to be done before using FME to convert.@daveatsafe - I have seen a similar thread where you have done some fantastic work building the AGS FME file. I have played around with it but struggle with the output. Below is the FME file:I have also used the AGS transformer which FME now has but not getting the output I was after. 

1950
M
A
andrewteg
asked in Data
Add to Multiple Feature Writers at once?

I'm going from ESRI Shape Files to PostGIS and trying to add a primary key to all my tables so that Mapserver and other queries are happier with the unique Primary Key. I see from https://knowledge.safe.com/idea/74374/postgres-writer-add-unique-primary-option-for-inde.html that I can add a Serial and that works well but I'd like to add that to 20, 30, or even 50 shape files at a time. Is it possible to do this as a batch?Thanks!

410
B
geo-x
Supporter
geo-xSupporter
asked in Data
Compare and update Json into DB

Dear FME community.I would like to compare to database who contains json (with subobjects).The aim is to UPDATE, DELETE or INSERT json into the target DBInforamtion : the base 2 may be to json format or "classical" attributes format.What is the better way to do this ?Thanks.

590
geo-x
Supporter
jorge_vidinha
Contributor
jorge_vidinhaContributor
asked in Data
POSTGIS Reader different geom column

Is there any possibility to change the default geometry column 'geom' on postgis readers to some other column with a different geometry like geom2 ?Thanks

2090
ponceta
Contributor
D
davidbroadstock
asked in Data
Featurewriter

Hi Guys, this might be a simple question - I am writing data to a TAB file. every time I run the workspace it over writes the TAB file with the new data. How do I append data being written to the TAB file.Dave

230
jenmulligan
Participant
S
spiller
asked in Data
points to topojson

I I am trying to convert the attached shape file to topojson. it is a shapefile of points. It seems some of the points are causing a problem when converting because I can take a sample and create the topojson file. I can't seem to figure out the problem with the offending points. Appreciate any help. Thanks.David

70
S
M
mamaieron
asked in Data
Unexpected SurfaceOnSurfaceOverlayer ouput

Hello everyone, I am converting CAD data to CityGML. To create the LoD2 class of buildings with hierarchical integrity, I used GeometryCoercer to transform the extruded solids into surfaces, followed by the Deaggregator, separating the roofs, walls and grounds. I tried to use SurfaceOnSurfaceOverlayer to identify shared walls between buildings but the transformer is not identifying all the cases. When I set the tolerance, it identify more cases but it ends up disturbing the geometry of the surfaces, generating non-flat surfaces, with duplicate points, etc ... all verified by GeometryValidator and by CityGML validation tools like Val3dity. As you can see in the prints, a case of shared surface is identified (Surface 1 and 2) and another one with the same characteristics (Surface 3 and 4) no. As I used the vertical direction in the Extruder and have the same X and Y coordinates, they should be coplanar- I also used the PlanarityFilter before the process and the value of the _surfaceNorma

120
M
dms2
Contributor
dms2Contributor
asked in Data
GML reader and 'cannot convert null string into an integer' error

We recently upgraded FME Desktop 2007 to 2020. Now I'm upgrading all of our old workspaces and I've having an issue after upgrading GML reader to the latest version.GML files are read without any error with the old reader but with the new version I get the following error for every feature in every GML file:Cannot convert null string into an integer class StatusInfo *__cdecl STF_stringToUInt(const class ObsoleteString &,unsigned long &)(C:\\code\\fme\\foundation\\framework\\util\\stf\\stfutil.cpp:358) - class StatusInfo *__cdecl STF_stringToUInt(const class ObsoleteString &,unsigned long &)(C:\\code\\fme\\foundation\\framework\\util\\stf\\stfutil.cpp:358)I checked the GML version for all my files in the xsd file and they all say it's GML 3.1.1. Didn't see any limitation on this in the reader's help.Any idea of what's going on?Edit: the old GML reader works as expected before saving my workspace with Desktop 2020, but the error is there after sav

1192
D
rudystricklan1
Contributor
rudystricklan1Contributor
asked in Data
Read AutoCAD files to be processed from a text file

I have a text file that contains an AutoCAD pathname on each line, e.g., D:\\project4033\\dwg\\p1234.dwg D:\\project515\\dwg\\p642.dwg X:\\dwg\\a556.dwgHow can I set up a Reader to process each file (the output format will be Adobe GeospatialPDF)? I see answers like a "TextFileReader", but on my 2020.1 system there is no such transformer.

130
rudystricklan1
Contributor
G
groschialeux
asked in Data
How to order tags in OSM Writer

I am working on a workspace that reads Tourism Information Office data from an XML file, converts the attributes to OSM tags and writes it to an OSM file. This OSM file is then processed by a program called OsmAndMapCreator to create an obf (OsmAnd Binary Format) file for the OsmAnd mobile application. Each office has a collection of characteristics and services so there is more than 1 tag for each node. The order in which those tags are written to the OSM file is important for how the feature will appear on the map. A tourism information office that has parking or picnic tables still needs to appear as a tourism information office no matter what else they offer. The OSM writer does not maintain the order of its reader or User Attributes list. In my reader and writer the tourism tag is at the top but for some reason, it is always the last one in my OSM file.  Here's 2 nodes where it creates a problem.This node shows up as a picnic table on my map but it should appear has a tou

270
G
B
bigapplerizzi19
asked in Data
How to convert OpenDrive (.xml) to 3D SHP

Is it possible for FME to convert OpenDrive (.xml) to 3D SHP?

540
virtualcitymatt
Celebrity
J
jdgvm
asked in Data
BIM (revit) to Cesium3dTiles, Reprojection Failed

Hi everyone,I have a BIM (revit file) that i would like to transform to Cesium3Dtiles. I used the "Revit Geolocation" Tutorial of @daveatsafe and exported the model into IFC. When I look in the FME Data Inspector the IFC file looks fine (I think). But when I add a Cesium3Dtiles writer I get an reprojection error. The coordinate system of the IFC file is EPSG:3857 and the coordinate system of the Cesium3Dtiles schould also be EPSG:3857. So I don't understand why there is a problem with the reprojector.

708
jovitaatsafe
Safer
dms2
Contributor
dms2Contributor
asked in Data
How to handle a geopackage received as base64 binary

I need to publish a workspace to fme server that will receive a geopackage file as base64 binary and will do some schema and data validation before inserting geopackage data into a number of Oracle tables. Actually an API will call the FME service providing that base64 binary and some other information.Now my question is how to read geopackage schema and data into fme. Any idea? I don't see how to get the binary back into a geopackage file.Thanks!

310
dms2
Contributor

Badge Winners

  • English Speaker
    jamatsafehas earned the badge English Speaker
  • Student (Grey)
    hdnguyenhas earned the badge Student (Grey)
  • Teacher (Grey)
    hdnguyenhas earned the badge Teacher (Grey)
  • FME Certified Professional
    rune.van.sevrenhas earned the badge FME Certified Professional
  • English Speaker
    nasirhas earned the badge English Speaker
Show all badges

Community Stats

31,788
Posts
120,751
Replies
39,405
Members

Latest FME

Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.