Using readers and writers to extract and load data.
Recently active
Running an automated script that gets data from another source to map. Have table in SDE (10.3.1) and set to Insert with Truncate Existing. The table is not truncating. It appears to work with Drop and Create. However, we want to keep the permissions and such intact, therefore using Truncate. This feed started today with about 15 rows, changed to 8 rows, then to 0 rows, back to 8 rows. This data is being used in a Dashboard on ESRI Portal for real time reporting. FME 2018.1.0.2 64 bit Build 18547 ArcDestop 10.3.1Thanks,
I generated some 3D tiles data using Cesium 3D tiles writer.Then shader effect is appeared in all of 3D tiles data.The Cesium 3D Tiles writer seems to write the shader effect in b3dm data by default.I would like to remove the effect from those 3D tiles data.Could you tell the solution?
Created a workbench to download and save on local disk, from S3 to local disk and read csv using featureReader works fine with FME Desktop (20181203 - Build 18578 - WIN64). Publishes same workbench to FME Server with additional parameters like file name etc and run through post call, failed to process csv file with following message: CSV reader: The keyword _DELIMITER has an invalid value: 'auto'. It must be a single character that is representable by one byte in utf-8BADNEWS: A fatal error has occurred. Check the logfile above for details (dynaread.cpp:199)A fatal error has occurred. Check the logfile above for details(dynaread.cpp:199). How do I track this fatal error ?
Trying to capture network traffic when FME Server writes to and reads from REST Services on ArcGIS Online to debug an issue with readers/writers and arcgis online. I followed all of these steps to setup and configure the proxy, but didn't seem to be working completely.Additionally, do we need to run fiddler on the account that FME server is ran on or can we do it through an admin account on the machine? We tried both Fiddler 2 and 4. In version 2, the FME account could see some of the traffic that was coming from jobs that were being ran in addition to traffic from the admin account. My admin account couldn't see any of the traffic from the FME user, but did see mine.
I am trying to read attachments from an AGOL feature service. Specifically, it is a jpg image. I have the reader feed into a BinaryDecoder set to Base64, outputting Binary(fme-binary), then going into a rasterreplacer. Error occurs on the BinaryDecoder. The error is:BinaryDecoder(TextEncoderFactory): Invalid Base64 character 'ÿ'Does anyone know how to handle or avoid this error? Thanks!
I have a MicroStation dgn file that I need to change the levels names, line styles, colors and weights to match our standards. Is there a way to read the attributes in one file and cross reference them in a excel spreadsheet or csv file and then write a new dgn file with the cross referenced attributes?
I have got a kmz file and I want to convert it to a shapefile. I also want to change the data from being a 3 coordinate system to a 2 coordinate system or I could keep the 3 coordinate system and ensure all the z coordinates are 0. There is not much help on how to transform kmz files to esri shapefiles in fme. Can someone give me some tips? I have attached the file here.
I'm trying to convert a NetCDF file generated by a climate model to be able to use in a GIS software. I have no specific software preferences.
Hi Safe.I seem to have stumbled across a bug in the WFS reader, that may also affect other OGC readers.Our local WFS services typically have a few extra parameters, e.g. "servicename", "login" and "password", or "ticket". These are given in the stub that the reader receives, and normally it works just fine.However, one of our services returned these parameters in the service stubs in the "capabilities" document, but had lowercased all the parameter names. I.e. the FME stub contained "Login=" and "Password=", but the returned capability-stubs had "login=" and "password=".Unfortunately FME handles this wrong, as it appends the camelcased parameters to the lowercased stub, because FME considers the parameter names to be different bacause of case sensitivity.The reason I say it's wrong, is because the OGC documents clearly state that "Parameter names shall not be case sensitive, but parameter
Hi,I have an Excel reader and one column of the Excel files contains merged cells.For an unknown reason, the option "expand merged cells" doesn't work well for some merged cells.For some merged cells, I got all values in the expanded cells and for some others, I got only 2 values for 5 cells that were merged (for example).(I tried to convert the Excel file in XLSX, XLS,.... the problem persists) So, I try to find a solution to fill empty values with previous values from the same column (attribute).I searched with the NullAttributeReplacer transformer or with a NullAttributeMapper but I don't find how to map to the previous value with this way... One other difficulty is that it must work in "cascade": for example the empty value from row 3 must get the value from row 2 and then the empty value from row 4 must get the value from row 3, and so on... Thanks for any help.Christian
Hello,I am using FME to create several zipped shapefiles (Not SHZ). And when I set my FeatureWriter to "PATH\\@Value(Tbl_Name).zip", the output folder is zipped but only contains the DBF file and nothing else. I am open to other suggestions on how I should be doing this. Thanks,Ben
I'm building quite a lot of flows where the amount of input features should always be the same as the amount of output features. During the processing of the data a lot of transformers merge, delete, split, and aggregate the data but in the end it should always result in the same amount of features as i started. I now use an UniqueValueLogger to list all the ID's in between and then compare it and at the end i do a manual check to see if the correct amount of features are written to the writer. But i was wondering if this could be tested with the tester and if it does not match can stop the translation raising an error that the input data is not correctly formatted.
Hi,I have a problem setting up a dynamic excel writer. If an input attribute has leading zeroes, they are removed in the output excel file. Can this behavior be prevented somehow? The input excel files are read via a FeatureReader.
H?,I want to do a wfs query with fme. as normal url =>http://cbsservis.tkgm.gov.tr/tkgm.ows/wfs?service=WFS&version=1.0.0&request=GetFeature&typeName=TKGM:parseller&cql_filter=durum=3&maxfeatures=10 if the result =>I want to query according to a certain value . I want to get values that are only 3 .<TKGM:durum>3</TKGM:durum>I've tried using xml filter. Is there any idea? Thank you.
Hello,In some cases I get out of memory exception when using writers. The reason is that FME needs to get all results for example from postgis into memory and then save it (for example in GML or DWG format).When there is a lot of data then problem appears. One soluton for that would be to use paging. First execute count() query to check how many records are we expecting and then for each part run workspacerunner with some limit and offset parameters. That works nice but:In my case I need to first take a polygon by which I will do filtering and then take all rows from another table that intersects with this polygon. Using FME Reader with intersects in works quite well but when I want to execute that query on my postgres DB then it takes forever even though I have geo indexes set on both tables :/ Do you have any experiences with such a problem?
I would like to detect from a postgis database the number of records and the filesizes (mb) of the tables in a schema so i can write it to xlsx. I can't find the solution.
We store a large number of files in a Bytea type as base64 ecoded strings in our database. If i use the pgAdmin to retrieve the data as Text it works fine.If you excecute the statement below in PgAdmin you get a different result then when you run it in sqlCreator to a Postgres database.How do i setup FME to get the same results as PgAdmin?What am i missing here?select ('\\x55457344424251414141414941474e7a4b6b3535706a744957514141414873414141414b414141414d584a6c593256774c6e4a6a6343584679776d414d4252453062316744314f427650796553545642564e796f415546527133636b44476675666d416674686c33486b7335384e514d31354a6677565332764a355954304f5748486b4b704e5254704e5132677567546a427276416d7a534b414c704e495a6b4453774571727736352f393855457342416a3841464141414141674159334d71546e6d6d4f30685a414141416577414141416f414a41414141414141414141674141414141414141414446795a574e6c634335795933414b4143414141414141414145414741444956446774364b6a55415276594e43336f714e5142473967304c65696f314146515377554741414141414145414151
I tried to convert a gml file to cesium 3dtiles. But the conversion didn't went well.What could be the issue.When i converted the same gml file in other software it got converted correctly.What wrong i have done in the workspace . I have attached the workspace , gml file and 3dtiles. Please help me with the issue.
Has anyone else had issues reading items that aren't shared to a group, but are shared to your organization? The only items that show up when I try to read them in are groupname/servicename or username/foldername/servicename, which indicates they are in a group or the user you are using owns them. We've been using FME for over 7 years, and I'm just now noticing this (which I guess either shows how important it is or how rare it is) :P
I am a first time FME user. I want to analyse the closest LiDAR points that are to my point data (pit). I tried using NeighborFinder but I guess it does not like LiDAR data and produces a empty .shp file or .gdb file. I want my .shp or .gdb with the nearest 5 LiDAR points that are averaged to produce a z value for my point vector data (pit data). It will be very kind if someone could help me with what code I would need to produce the required result? PS: plz disregard the aerial image in the background.
When I try to save a database connection. In Tools -> FME Options -> FME Database Connection. The dialog is displayed: Unable to Store Connection.Can anyone tell you why this happens?
Hi allIn a GIS to CAD transformation, I'm trying to make all autocad linear entities with line type generation attribute = true per entity.I added a DWGStyler to my workbench with "linetype generation" property set to "continuos" for lines. However, it's not working so far.Could someone please point out what I'm missing?Many thanksAlan
Hi,I have to read Excel files with FME 2017 but the first column of the table can be at different position in each file.(but there are always the same number of columns)So, I'm searching a way to link the first column position to a published parameter like the screenshot attached (which doesn't work...).I can't believe that with such a powerful soft like FME we can not do this...Thank you for any help !Christian
I am trying to connect to database using Presto JDBC driver. Need to know how to define and set parameters for this driver e.g. SSL = true, SSLTrustStorePath etcjava.sql.SQLException: Authentication using username/password requires SSL to be enabled
i am trying to update xml column in oracle and getting errors which i am not able to resolve . need some help hereif the data exists there and $t/row/c178[@m=82]='USA' then only update the value or do nothing update USER.TABLE t1set t1.xmlrecord=XMLQuery('copy $t := . modify (if fn:exists /row/c2 then replace value of node $t/row/c2 with concat("firstname",$t/row/@id) else(),if fn:exists /row/c3 then replace value of node $t/row/c3 with concat("lastname",$t/row/@id) else(),if fn:exists /row/c178[@m=83] and $t/row/c178[@m=82]='USA' then replace value of node $t/row/c178[@m=83] with ("NEW YORK") else() ,if fn:exists /row/c4 and $t/row/c178[@m=82]='USA' then replace value of node $t/row/c4 with ("Manhattan")else() ,if fn:exists /row/c5 and $t/row/c178[@m=82]='USA' then replace value of node $t/row/c5 with concat ("NY 10036, USA.",$t/row/@id) else(),)return $t'passing t1.xmlrecordreturnin