- - Community Resources
Ideas
Product Updates
Events
Blog

Home
Forums
FME Form
Data

Data

Using readers and writers to extract and load data.

6,811 Posts
25,090 Replies

When you subscribe we will email you when there is a new topic in this category

6811 Posts

Hi Everyone,I am trying to convert Esri .gdb file to PostgreSQL v 11.5 with PostGIS extension of v 2.5,an error message appears (Error executing SQL command ('SELECT AddGeometryColumn('public', 'UtilityNetwork_EDD', 'geom', importfromesri, 'GEOMETRY', 3)'): 'ERROR: column "importfromesri" does not exist)Can anyone help in solving this issue ´, I have searched for the same problem but nothing helped me, thanks in advance

Can I create PGDump file like ogr2ogr

I have a remote postgres / postgis db that I'd like to import data using .sql file resulting from an FME translation. Something like ogr's PGDump format using the COPY methodKindest regards...

Workspace didnt work on server due to DatimeConverter Error

Hello,I have a simple workspace. I read a xml service, use datetime converter then write DB. It works perfectly in FME Dekstop but when I publish and schecule job to FME server I got error below. DateTimeConverter_Rejector (TestFactory): The clause 'TEST @Value(fme_rejection_code) != FEATURE_TABLE_SHIM_SUPPORT Yes' within 'FACTORY_DEF * TestFactory FACTORY_NAME DateTimeConverter_Rejector INPUT FEATURE_TYPE ___TOREJECTOR___ TEST @Value(fme_rejection_code) != FEATURE_TABLE_SHIM_SUPPORT Yes OUTPUT PASSED FEATURE_TYPE DateTimeConverter_<REJECTED> @RemoveAttributes(___fme_rejection_code___) OUTPUT FAILED FEATURE_TYPE DateTimeConverter_OUTPUT @RenameAttributes(FME_STRICT,fme_rejection_code,___fme_rejection_code___)' is incorrect. TEST must look like: TEST <value> <operator> <value> [ENCODED]I can not find how to configure test parameters of datetimeConverter or solve this issue.I look into comminity and knowledge sections of safe and on the internet bu

140

4 years ago

jessContributor

asked in Data

Parsing JSON array from HTTPCaller

Hello!I'm trying to parse JSON data returned from the API for Oracle Service Cloud. I've figured out the HTTPcaller piece and the data is returned like this:{ "count" : 10000, "name" : "OpenData REST export", "columnNames" : [ "Address", "Channel", "Date Closed", "Date Created", "Date Last Updated", "Disposition ID", "Latitude", "Longitude", "Product Hierarchy", "Product ID", "Service Request ID", "Status Type" ], "rows" : [ [ null, "Email", "'2018-06-20 12:46:36'", "'2018-06-19 17:36:19'", "'2019-08-20 18:22:39'", null, null, null, "Parking", "Valid monthly permit", "2296", "Solved" ], [ null, "Email", "'2018-09-06 10:13:19'", "'2018-06-19 18:28:00'", "'2019-08-20 18:22:38'", null, null, null, "Parking", "Disputing information on offence", "2299", "Solved" ], [ null, "Email", "'2018-06-28 14:45:50'", "'2018-06-19 19:07:40'", "'2019-08-20 18:22:38'", null, null, null, "Parks", "Residential new tree request / replacement tree", "2303", "Solved" ]...etc} I need to

Convert DTED to CesiumJS 3D Tiles

i have some sample DTED (*.dt0 and *.dt1) files that contains grids of elevation data. I like to know if its is possible to convert this data in to 3D tiles and display them in a CesiumJS map? i tried some ways as seen in a post "How to Bring 2D Data into CesiumJS"from @dmitribagh but my inputs are different from that. so i appreciate if anyone can share any ideas. Thanks

FeatureWriter slow writing back to FGDB

I have a workbench that is reading from an already populated FGDB with 140 Feature Classes. I am trying to add an extra column to the majority of those feature classes and then populate the column for each row with a URL and ID to match the row. I am finding that it is taking 6 minutes to write back 60,000 rows and considering there are over 10 million to update that is an unacceptable timescale.I am creating the new attribute and a non spatial index with a python caller. Feature Reader to get the ID of each row. AttributeCreator and Manager to create the URL and merge it with the ID. Finally an AttributeFilter to the FeatureWriter. In the FeatureWriter I am doing an Update and using PrimaryKey columns for the row selection. I would appreciate any thoughts on how I might quicken the process.

Hi, sorry im struggleing to convert KML to shp, i can get the shp files converted but i cant seem to transfer any names of the files

How to parse an xml file

I have tried to meka a flat file out of an xml file but I can't get the parent data onto every row of the child data. How can it be done? here is snippet of the xml file: <?xml version="1.0" encoding="UTF-8"?> <event Type="Relay"> <eventname>Lappee-Jukola 2016, Jukolan Viesti</eventname> <class> <classname>JU</classname> <team> <teamid>9</teamid> <teamname>Koovee</teamname> <teamnro>1</teamnro> <result>8:03:46</result> <tsecs>29026</tsecs> <placement>1</placement> <leg> <legnro>1</legnro> <nm>Topi Anjala</nm> ,....so I'd need to get all this data on the same row for each result:eventname, teamid, teamname, teamnro, result, tsecs, placement, legnro, nmJukola,9,Koovee,1,8:03:46,29026,1,1,Topi Anjala

can I import git project into fme workbench?

Hi,is there any sources to use the git in FME unlike Eclipse, pycharm….

4 years ago

koenvdwContributor

asked in Data

copy images from folder to another folder based on their name in a csv

HiI have a csv file with the name of an electricity pole and the oblique images where the pole is visible on.Now I want to pull these images from a folder with a large collection of images containing the images I want and copy them to the folder with the csv. I dont know how to import a folder or a path. I have seen them but cant seem to find in the readers... I don't know how to copy. Please be elaborate with the explanation, because that's something I never have done with FME.

1020

4 years ago

gabriel_hirschContributor

asked in Data

Pricing and documentation for the GoogleVisionConnector transformer

FME 2020 supports the Google Vision API with the transformer GoogleVisionConnector. The user just have to enter their Gmail address in the transformer (and then approve the connection with FME) to make use of the functionality. The documentation of the transformer informs that The GoogleVisionConnector uses your Google Cloud account credentials to access the Google Vision client services.With this forum question I just want to clarify the pricing of using the Google Vision API with the GoogleVisionConnector transformer. Is it correct that the transformer uses the API so that this pricing table applies? https://cloud.google.com/vision/pricingWhat happens when the amount of API calls exceeds 1000 units/month? I can assume that Google will send an email to the connect Gmail account but what happens in the GoogleVisionConnector transformer? What is the reply from the API and how is it interpreted in FME? Will it come out of the Rejected port?

4 years ago

ssisman06

asked in Data

.Gdb and sde write problem

H? all,I am trying to write my data over gdb and sde by processing. Some columns containing integers write null.fme is working fine. I'm not getting an error. but when I look at the database it looks null. How can I solve this problem. Thank You.

100

4 years ago

oliver.morrisContributor

asked in Data

cesium terrain

Hi, we would like create some terrain models for cesium - is it possible to create TIN quantised meshes that are directly compatible with CesiumJS?Thank you

810

4 years ago

cbalmbra1Contributor

asked in Data

Automate updating documents on CDE

I am currently updating a number of documents and models on a CDE (currently trying to pull the API information) that require manually logging into the system as an administrator, deleting the files in question and then uploading the updated file.I wanted to see if there was a way to automate this procedure through an FME workbench?

4 years ago

naren

asked in Data

Any Use case for ArcGIS and FME desktop to anlasis the TOID, Topoligical points, sites, polygon's......

Hi All, I am really excited to learn new tool and work with new tool, new project new task really challenging !AS, I said, I am new to FME data integration Platform, I am really struggling where can I start. AS an Automation tester came from python background, I need to automated the TOID lines, sites..., My project include with Technologies ArcGIS and FME. Could you please suggest me any tutorial or any use case related to Geo spatial Maps.(I have excel, .fme, .gpkg files). Apologies for lengthy question, I really nee to run the test and develop more based on them ThanksRegards

4 years ago

kana

asked in Data

Read blocks of txt file and search for data

Hello. I have a txt file which contains "blocks" of data, separated by line containing #####. I need to extract information from each "block", and each time the #### appears that means the new epoch starts. For each block I check if it contains eg. "Station point:", "PPM", "MWF". Each Feature is then written in DB. Not all blocks give results for each Tester.Here is an example of such file:2020-03-23 08:01:13.8810 Info ############################################################################################################################################### 2020-03-23 08:01:13.8810 Info Processing pointgroup 'MWS200_Fix' (Type:'Freie Stationierung') measured by 'MWS200'. [GON/M] 2020-03-23 08:01:13.8810 Info Station point: 'MWS200' 2020-03-23 08:01:13.9122 Info Station coords: 2660392.8147,1251984.6680,416.3654 2020-03-23 08:01:13.9435 Info Reference coordinates of 'MWF204': E=2660397.8542,N=12

4 years ago

sivaram81082Participant

asked in Data

-2147024893 Error when reading esri file gdb in compressed folder..

I get the following error message when reading esri gdb in zip folder. Reader not functioning as expected in both Desktop and Server when pointed to zip file. When extracted and reader pointed to gdb, it works. Error -2147024893 The ArcObjects license 'Standard' is being selected from the Windows registryFME has checked out an Esri license. The product checked out is 'Standard'Installed ArcGIS version is '10.8'Unable to connect to the File Geodatabase at 'D:\\xxx\\xxx\\Compressed\\FileGeoDB.gdb.zip'. Make sure the correct filename was specified, and that the geodatabase wasn't saved with a newer version of ArcGIS than the one installed locally. The error number from ArcObjects is: '-2147024893'. The error message from ArcObjects is: {} Is it mandatory to extract the files manually before executing the workbench. Or do we have any work around. ThanksSiva Bulusu

190

4 years ago

thijsknapenContributor

asked in Data

Why does a JSON reader convert JSON boolean values (i.e. true/false) to Yes/No boolean values?

I hope the title is already quite self explanatory.As an example, consider the following JSON document;{ "boolean_1": true, "boolean_2": false}When this is being read with a JSON reader, it returns the following output;I would prefer the data to be non altered, so that comparisons can be made later on.See also the following attached sourceFile and workspace; To me it reminds me of the following item I addressed earlier on;https://knowledge.safe.com/questions/90726/extracting-nested-object-with-jsonextractor-issue.htmlin which a JSON Extractor used to convert JSON boolean values (i.e. true/false) to 1/0 boolean values, where I'm glad to see that that got fixed.To me it only raises some concerns that the handling of these values doesn't seem to occur in a very consistent manner; whereas a JSON fragmenter does seem to keep JSON boolean values intact, initially a JSON Extractor converted these values to 1/0, and now I find that a JSON reader converts these values to Yes/No. :(

9825

4 years ago

gogopotter90Contributor

asked in Data

How could export only specific area and the intersected of this area direct from oracle reader

i have reader oracle spatial and i would like to export only specific area and the intersected of this area only .i do not want to export all areas from oracle reader because it spends too much time so i would like to export the interested areas only from oracle . i did already spatial filter after that and it works great but the problem it takes too much time to export all areas then execute spatial filter so is it possible to write code inside where clause and as a result as give me rrequired areas without upload all areas .Thanks alot

4 years ago

stefanhContributor

asked in Data

How can I create a nested JSON from an Excel- or CSV-file?

I have a Excel-file with the following content: group1group2group3nameinfoGeneralNation Philinfo1GeneralNation Kareninfo2GeneralMunicipality Billinfo3GeneralMunicipality Paulinfo4SpecificProvince Patrickinfo5SpecificProvince Maikelinfo6SpecificProvinceGovernanceMikeinfo7SpecificProvinceGovernanceLukeinfo8SpecificDistrict Mariainfo9SpecificDistrict Davidinfo10 The attributes can be adjusted it is a Excel-file; I create the file myself.I would like to automatically create a nested JSON file. This should look like this: "name": "Overview", "children": [{ "name": "General", "children": [ { "name": "Nation", "children": [ {"name": "Phil", "info": "info1"}, {"name": "Karen", "info": "info2"} ] }, { "name": "Municipality", "children": [ {"name": "Bill", "info": "info3"}, {"name"

8125

4 years ago

benvkContributor

asked in Data

Is there a simple way to add a 'total' row to a Excel Writer?

I am outputting many rows to an excel table, some of my columns are numeric, I am hoping to add a total row at the bottom of my table, to sum these columns. I have found the Statistics Calculator, but I have no idea how to apply it (if at all possible). Any thoughts?

PointCloudeExpressionEvaluator Parameter type

I tried to fuse the shapefile information with the point cloud. When I clipped the point cloud based on the shapefile polygon boundary, the information of the count number of each polygon is supposed to be attached to the point cloud attribute. I saved this information in the userdata attribute but it is Uint8 data type, my count number should use int16 to cover the range. Is there any solution that can set the userdata data type as int16 without converting by the application automatically ?

4 years ago

swachContributor

asked in Data

ESRI Geodatabase Raster Dataset Footprint

When you view an fgdb mosaic dataset in ArcMap, the footprint and boundary are features that are shown and editable. I can read the fgdb raster dataset in FME but I can't figure out how to access the footprint and boundary datasets. Is this data accessible?

How to update single layer in existing CAD dwg with gdb feature class, while maintaining all other layers that already exist in dwg?

Hello, I'm still a relative newbie to FME, so bear with me. I'm trying to update our topographic CAD dwgs with our updated LiDAR derived contours. I only want to replace the existing contour layer and maintain all other layers in the existing CAD dwgs. I'm just not sure how to proceed. The contours are already clipped to the same index grid as the dwgs. Any guidance to the correct path of transformers would be most appreciated. Thanks everyone, be safe out there and wash those hands!

Writing Autocad text features creates block

HI, I looked at this great article: https://knowledge.safe.com/articles/38908/autocad-workflows-reading-and-writing-text-mtext-f.html And tried applying it for my project. By creating label of my feature id then using DWG styler:label = @Value(ID)gThe problem I have is that these do not text as the example article but creates a block with text tags. If I manually add text in autocad I get this: RIght TextBut instead I get this: Block ReferenceI wrote the text in a separate layer called LayerLabels, I even added a AttributeManager adding a new attribute autocad_entity = autocad_text and activated format attributes "autocad_entity" in the writer to maybe force it to read as text but its still created as a block.My writer:IN bentley it says its an FME block, but I never set any blocks and my only reader is a shp readerThis below is also supposed to say text with a T sign next to the Acad and supposed to show the text Any ideas?

230

4 years ago

Page 78 / 273

Badge winners

dmccullagh.npcahas earned the badge FME Form Basic Training
sdcp73has earned the badge Welcome Back
andybarnes1978has earned the badge Welcome Back
ibrucehas earned the badge FME Form Basic Training
ibrucehas earned the badge FME Form Advanced Training

Show all badges

30,372: Posts

Terms & Conditions Cookie settings

Sign up

Already have an account? Login

An FME Account is required to contribute

Login to the community

An FME Account is required to contribute

Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.

Enter your e-mail address

Back to overview

Scanning file for viruses.

Sorry, we're still checking this file's contents to make sure it's safe to download. Please try again in a few minutes.

This file cannot be downloaded

Sorry, our virus scanner detected that this file isn't safe to download.

A product of

Company Products Giving Back Careers Contact Privacy Feedback

Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.