Skip to main content
  • EN

Data

Using readers and writers to extract and load data.

  • 7,082 Posts
  • 26,081 Replies
7082 Posts
N
natasha
asked in Data
Deleting records from a relational database

Hi everyone,I having trouble working with a Postgres relational database. Basically what's happening is I'm reading from one (or many) XMLs files, transforming/combining/editing data then writing to a number of different Postgres tables which all link together via a primary key in the 'master' table. When the XML file(s) get updated and a record drops off, I need to delete that record from the master table but because of the related records in the other tables, I am unable to do so. Obviously the related records in the other tables need to be deleted as well (and apparently before I can delete the 'master' record). Does anyone have some logic around how to make this work? It seems I need to find the deleted 'master' records (I'm using Update Detector at the moment), then somehow get a list of all of the related records in the other tables, delete them, then delete the 'master' record. Any tips or advice would be greatly appreciated!Nata

2394
N
D
davidalvarez76
asked in Data
Surface (LAS) output to LandXML

All, I am trying to export a surface created by TINGenerator from LAS to LandXML. Does any one knows how should I structure the output so my LandXML come out correctly (valid) ?I am using http://www.landxml.org/webapps/LandXMLValidator.aspx to validate my output. Thank you,  

530
mark2atsafe
Safer
webservicessia
Contributor
webservicessiaContributor
asked in Data
GML extract geometry CircleByCenterPoint instead of posList

Hello,How to extract a GML geometry from feature with keeping the original CircleByCenterPoint type and the UOM instead of a posList of points ?

476
mark2atsafe
Safer
G
grishmash
asked in Data
How to filter excel file with formulas without messing up the output numbers using FME

I'm new to FME and I'm creating a workbench using two excel files. One has all the data with formulas from information column (Region, Strat, Just Code) to formulas -- simple Sum (A+B), Percentage to complex IF statements AND the second excel file contains info which helps me filter using certain categories (Just Code in this case). I used Feature Merger to merge the two excel files, then Tester and then Test Filter to help me create output based on Strat using Just Code (For eg: If Just Code (in 2nd excel) is 1 or 2, Output that as Poles). I opted for 'Keep formulas' while first creating the Reader but when I checked the output, all my excel numbers with formulas are messed up). In the excel file 'original sheet', Column G (row 10) is = F10+AI10-E10 but after the filter the output which is now in Column G (row 8) is still reading as F10+AI10-E10 disregarding the filter when it should be F8+AI8-E8. I was advised to use "Counter" and "String Repla

1300
1
fgiron
Supporter
fgironSupporter
asked in Data
ARCHIBUS

Hi everybody!Have you ever heard of ARCHIBUS? As far as I understand it should be a web platform for managing Real Estate, Infrastructure & Facilities assets. According to their website (https://www.archibus.com/):"ARCHIBUS reduces complexity through native business intelligence. ARCHIBUS turns BIM, GIS and Mobile into powerhouses that accelerate understanding of your real estate portfolio, workspaces, and operations, and speeds business processes" Any idea about what kind of data they use? Would it be possible to integrate that with FME?Thanks a lot in advance for any tip you can give!

150
fgiron
Supporter
danilo_fme
Evangelist
danilo_fmeEvangelist
asked in Data
Problem to connect transformer S3ObjectLister

Hi FME user's,My customer doens't connect the transformer S3ObjectLister. I maked the test in my machine and company and the results is OK, all files are display.I believe my customer wants to contact the IT Team to disable any port.Attached the log file.Thanks in Advance,Danilo de Lima

50
darkspatiallord
B
bfegyverneki
asked in Data
Why is there a 10 Character limit to V8DGN Cell Names in the V8DGN Writer?

Why is there a 10 Character limit to V8DGN Cell Names in the V8DGN Writer when the V8DGN file format allows for a 512 character cell name? The old igds days are long gone and I don't understand why FME V8 DGN writers would not use the V8DGN file format attributes sizes? 10 Characters is not enough in the V8 DGN world for a cell name.

60
fmelizard
Safer
krostir
Participant
krostirParticipant
asked in Data
?KML with ExtendedData to Shapefile

I am trying to convert a Google KML file to ESRI Shapefile and to include the ExtendedData as attributes.European Space Agency is publishing planned satellite observation scenario as a KML file that has several attributes in the ExtendedData tag (see ). These attributes are always the same, as here:<ExtendedData>   <Data name="SatelliteId">      <value>S1A</value>   </Data>   <Data name="DatatakeId">      <value>1C269</value>   </Data>   <Data name="Mode">      <value>EW</value>   </Data>   <Data name="Swath">      <value>NA</value>   </Data>   <Data name="Polarisation">      <value>DH</value>   </Data>   <Data name="ObservationTimeStart">      <value>2017-06-06T20:19:16</value>   </Data>   <Data name="ObservationTimeStop">      <value>2017-06-06T20:20:32</value>   </Data>   <Data name="ObservationDuration"> 

697
geosander
P
philippe33
asked in Data
How to obtain a good structure

Hello,It's hard to explain. I'm french guy and new in FMEI give you a "fmw" file and try to explain my problem. I try to transform an AIXM 4.5 file to an AIXM 5.1 file. This concern the "AirportHeliport" feature. I follow different examples on the FME website and some examples in my office.I have a problem with the final schema. I try to obtain the "servedCity" structure at the end of the transformation but in the final output file (XML File) the structure "ARP" disappear ... I don't want that, but i don't understand why ...So, if i connect the bookmark "servedCity" the result with ARP disappear and if i disconnect "servedCity" the result with ARP appear.what is wrong with the "geometryPropertysetter" transformer ? I saw it's necessary to use it with the XML_Geometry type but i don't understand why the structure in the xml file disappearPlease some helpMy first Zip : airportheliport_without_se

160
geosander
whkarto
Contributor
whkartoContributor
asked in Data
Flipped triangles at OGCKML Writer output

Hi everybody!I'm facing a problem with the ocgkml Writer. When I route 3D surface geometry of building models to the writer, I'm getting flipped surfaces for my facade features. Unfortunately this behaviour doesn't remain constant for all facade triangales, some of them are oriented correctly and some are not. The input features look correct to me and exports to collada prove this to be wright.An export with FME 2015 also shows a correct export with all walls oriented correctly. So this has to be an issue with FME 2016 and 2017. Has anybody else encountered this behaviour and found some workaround for this? Find attached the sample data together with a snipped of the workspace and the exported KML filesThanks in advance to everybody who can help me!

90
geosander
helmoet
helmoet
asked in Data
Directory and path reader does not support nul

Hi, I came across the following issue: using a directory and path reader (filtering for files only) and supplying an empty folder works fine (it doesn't produce any files, but that's just fine). However, say I want to skip any processing from this reader, as an innocent user I supply the Windows nul: device. Now my workspace crashes, how is that? Of course, there are a lot of ways around this problem, but wouldn't it be nice if the directory and path reader behaves in the same way like, say, the Windows dir command?

599
helmoet
f.kemminje
Contributor
f.kemminjeContributor
asked in Data
how to create line string from json point array

Hi team.I extracted json from http caller. extracted json file attached. json.txt.in this json there are array of points. the path of these points is. ucan see these points after "paths"(text string)routes{0}.routeOptions{0}.optionAttributes.spatialReference.geometry.paths{0}.array{0}.array{0} routes{0}.routeOptions{0}.optionAttributes.spatialReference.geometry.paths{0}.array{0}.array{1} routes{0}.routeOptions{0}.optionAttributes.spatialReference.geometry.paths{0}.array{0}.array{2} routes{0}.routeOptions{0}.optionAttributes.spatialReference.geometry.paths{0}.array{1}.array{0} routes{0}.routeOptions{0}.optionAttributes.spatialReference.geometry.paths{0}.array{1}.array{1} routes{0}.routeOptions{0}.optionAttributes.spatialReference.geometry.paths{0}.array{1}.array{2} routes{0}.routeOptions{0}.optionAttributes.spatialReference.geometry.paths{0}.array{2}.array{0} routes{0}.routeOptions{0}.optionAttributes.spatialReference.geometry.paths{0}.array{2}.array{1} routes{0}.routeOptions{

1024
david_r
Celebrity
F
felixderglueckl
asked in Data
Deleting the template sheets after writing Excel file with a template file

Hello everybody,I want to create an Excel file with the FeatureWriter. To get the output well formatted I created a new file before that all as a template file. Well, the FeatureWriter does something I don't want to have. It creates a copy of the template file and creates a new sheet in this copy with the output.. This sheet is exactly that what I want to have but the additional sheets of the copy are disturbing. Is it possible to remove the sheets of the template and just keep the sheet which contains the output?Best regards,Felix

540
takashi
Celebrity
D
danellelaurin
asked in Data
Can you send a link toTutorial: KML Transformations

I'm an entry level FME user and I'm struggling trying to transform a file geodatabase to kml. The feature class consists of 67k culvert line features with 38 attributes. Both FME and ArcGIS seem to generate the kml but Google Earth encounter an error whey trying to open it. I thought the tutorial could help me discover what I need to do to fix this.

210
D
A
Anonymous
asked in Data
Read and Write Geometry in Oracle Spatial Database

Can FME read and write features that are in ESRI's SDE.ST_GEOMETRY in an Oracle Spatial database? I have a table that is in ST_GEOMETRY, but the Oracle Spatial reader cannot show it because there is no entry in the user_sdo_geom_metadata table. I don't see an ESRI reader that would be able to read from Oracle. Likewise, when I try to write from a file geodatabase that is in ST_GEOMETRY, FME converts the geometry to SDO_GEOMETRY.Thanks

1360
runneals
Supporter
4
40_eme
asked in Data
Write automatically to a subdirectory of the source data

Hello,  I have to read sources that have the same structure in n paths all structured in the same way that I therefore designate to read by ..... dep \\ com ??? \\ shape \\ parcel.shp. ??? Is 001, 002, 003, etc ... No problem for reading. However, I would like the destinations to be written each time in a subdirectory (example: ..... dep \\ com ??? \\ shape \\ l93 \\ parcel.shp.  Can you help me ?  Thank you.

380
revesz
Contributor
danilo_fme
Evangelist
danilo_fmeEvangelist
asked in Data
Insert color in Attribute Names in XLS

Hello FME Users,I need to put the color in Attributes Names in my XLS file. I saw the configuration in Output XLS, but it change the color all my columm:The Line 1 is the Attribute Name The Line 2 is the Value.Thanks

550
mark2atsafe
Safer
pcheng
Contributor
pchengContributor
asked in Data
Excel Exact Position Placement

Hey Safers,I was wondering if it was possible to position an attribute to an exact row/column position!Using the image below as an example I want to designate an attribute 'LICENSE' to the cell indicated by the red arrow, let's say its at G-5Is there a formula I could use? The spreadsheet can't reference a raw data table as its a fan out of many pipelines in one workbook (a,b,c,d)Thanks,Patrick

1220
pcheng
Contributor
R
ravi255210
asked in Data
The error number from ArcObjects is: '-2147216072'. The error message from ArcObjects is: {FDO error: -2147216072}

Hi All,I am facing an issue while translating data from Oracle Spatial to ESRI Geodatabase. while translating the data using FME workbench getting error message "Unable to drop the table/feature class 'GD_EXPANSION_JOINT_point'. The error number from ArcObjects is: '-2147216072'. The error message from ArcObjects is: {FDO error: -2147216072}". i did google search for this FDO error number but did not find anything. Could anyone help me out in this error.Thanks in advance.Thanks & Regards,Ravi Kumar

27480
philsc
Contributor
pschout
Participant
pschoutParticipant
asked in Data
read dat from $(FME_SHAREDRESOURCE_DATA)

Is it possible to read data from $(FME_SHAREDRESOURCE_DATA) with the feature reader? In a workspace i write excel datasets to $(FME_SHAREDRESOURCE_DATA), I want to read these because they contains attributes which i need to dicede to send a mail (or not) with another job.(The full path to the data is : $(FME_SHAREDRESOURCE_DATA)/Informatie_Management/Vaarwegmeldingen/Meldingen_borden.xls)I use FME server 2015

310
david_r
Celebrity
Y
ygutfreund
asked in Data
Sending Count over Total of files written by FeatureWriter as the are emitted?

I currently have a custom Transformer that gets a parameter MaxTileCount (the value of _num_tiles) from a WebMapTiler and it counts the number of features and sends a HTTP message every 10 messages to show progress.That is all working fine.However, now I have issue. The PNGwriter is no longer a PNGRaster writer, but has changed to a FeatureWriter that does a fanout. Well that takes a lot of time, So I want to move the Notifier to later in the pipe, after the "pngWriter (FeatureWriter). The issue I have is that I don't know how to count anymore the number of files written by the FeatureWriter as they are emitted, nor do I have access to _num_tiles which is the total number of tiles coming from the WebMapTiler.In simple words, I want to send a HTTPPost message with count/total as the files are written by the FeatureWriter any idea who to do this?The second notifier fails. So I disabled it.

610
takashi
Celebrity
jordiew
Participant
jordiewParticipant
asked in Data
Excel writer creating XML errors

I'm using FME Desktop 2017.0.1.1. I have a csv file that I need to merge with 6 different csv files. The 6 different csv files are derived from MapInfo table that Intersects with another table and the selection is exported as a csv. I have AttributeManager, AttributeCreater, DateFormater and FeatureMerger for each csv. These csv files are FeatureMerger with another csv and the writer is a 6 sheet excel workbook. When "Translation was SUCCESSFUL with 0 warning(s) (81575 feature(s) output" the excel xlsx file contains errors: "Removed Part: /xl/sharedStrings.xml part with XML error.  (Strings) Unexpected end of input. Line 1, column 1614248. Replaced Part: /xl/worksheets/sheet1.xml part with XML error.  Unexpected end of input. Line 1, column 1614250. Replaced Part: /xl/worksheets/sheet2.xml part with XML error.  Unexpected end of input. Line 1, column 1614250. Replaced Part: /xl/worksheets/sheet3.xml part with XML error.  Unexpected end of input. Line 1, column 1614250. Replaced Part: /

940
mark2atsafe
Safer
J
jonathanlepage
asked in Data
Acad Block Rotation

Hi,   I'm trying to insert block in a DWG file with a specified rotation.    If I use feature and replace them with a block they all appear with the wrong rotation.   Here I set the rotation at 0 for all of these :  

1010
jneujens
takashi
Celebrity
takashi
asked in Data
Restriction on Deleting Questions

I noticed that some users delete their questions, even though answers helpful to other users have been posted. I think it would be better if there were a restriction that a question having answers cannot be deleted unconditionally.

80
takashi
Celebrity
R
rob14
asked in Data
Generate list of all unique characters contained in (huge free text) field.

Hi,Can anyone provide some advice as to the most efficient way to explode a huge free text field or fields into all of its character elements retain a single instance of each. I am essentially trying to complete a pre-flight check in order to understand whether there are any ‘odd’ or ‘unexpected’ characters in an ever expanding data set, over which I have no control. I have created a process below which completes the task; however, it is very inefficient and as the number of records increases it will become too slow. 1. Derive string length of free text field 2. clone by number derived in 1 (clone number created in process) 3. substring extract using clone number to obtain character at that position 4. Duplicate remover to create my list. 5. Expose character code. 6. Output listThanks in advance,Rob

1775
R

Badge Winners

  • Welcome Back
    mjjhas earned the badge Welcome Back
  • FME Certified Professional
    _jacques_has earned the badge FME Certified Professional
  • Integrate Spatial Data
    madihas earned the badge Integrate Spatial Data
  • Webinar Watcher (Bronze)
    bradmakesmapshas earned the badge Webinar Watcher (Bronze)
  • English Speaker
    bradmakesmapshas earned the badge English Speaker
Show all badges

Community Stats

31,792
Posts
120,765
Replies
39,408
Members

Latest FME

Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.