Using readers and writers to extract and load data.
Recently active
Hi all, So this I thought would be a really simple task, but I am having issues bringing in an Excel spreadsheet into ArcGIS Portal. My end goal for this was to update an excel spreadsheet on our server, and get FME to send them changes through on a schedule to Portal, in turn updating a dashboard we have. Bear with me as I have tried lots of different things but to no avail, this is what I tried in a nutshell:Firstly I tried reading an excel spreadsheet, and then adding it as a new feature to Portal. The parameters I used were layer name as default and set geometry to arcgisonline_no_geom, as it isn't spatial data, and set writer mode to 'INSERT'. The two tables (geodatabase and hosted table) gets added to my Portal, but there is no data there, if I click on data tab or fields it just says "error". Looking at the FME log the only warning which might be the cause is in relation to SSL certificate "SSL certificate verification failed for host... HTTPS connect
Hi!I have a road network in a shapefile (polygons). When I read the shapefile, all the roundabouts become multipolygons. I assume this is because of a error with island detection. When I read the same shapefile using Civil3d (Autodesk) the roundabouts are perfect single polygons. I already tried it with a sdf and a gml, but the problems persist.Can someone please tell me what I am doing wrong.. This is a problem I have faced a lot of times, any help would be much appreciated!
I am trying to write raster image to Azure SQL, I need to store four images in each record - the following formats: PNG, JPG, JP2. The writer works well - no errors during the writing, however, the stored images cannot be read in another program (no FME). Any comments regarding the correct way to import images into Azure SQL database... ?Thank you!
Hey, I am currently reading a feature layer from AGOL, processing the data and writing the edited data to a new feature layer. I have been able to extract the associated image attachments from my original table, but I am wondering if there is a way for me to insert these images to my output table? Can we write these images, i have a GUID of the feature i need it to be attached to, I just don't know if it is possible through FME.Any ideas?
Any ideas how would FME power be able to convert a 'decapitated' CAD file back to text?I have a PDF file which used to be a CAD drawing (not sure if Autodesk or Bentley) and it contains only a 'dropped/exploded' line segment geometry. I have managed to read it into FME as line geometries, rotate the page so that it's 'level' and use LineCombiner to join segments back to 'character shapes'.Now I have geometries as shown below (some are perfect characters, some are in 2-3 parts, like 'd or b') and have no idea how to turn it back into text. I tried exporting it to DGN but Microstation doesn't seem to have a function like that either (i.e. once you drop/explode a text to lines, there's only 'Undo' to help, no function to 'characterise' it again, that I found).Also tried rasterising it with ImageRasterizer and then calling a Tesseract to attempt OCR, but it notoriously returns 'No text found' regardless of resoluti
I'm using Geodatabase_SDE reader and writer with different connection files. I have one database "Operations" with user1 and another database "Data_Sources" with user2. They both support geometric data used in ArcGIS. I try to read from Operations.user1.featureclass1, and write to Data_Sources.user2.featureclass2. Operations.user1.featureclass1 is in a dataset "ops". I'm tryng to just write a standalone featureclass in Data_Sources, so i am leaving the Feature Dataset parameter as blank. However, seems that no matter what i try, the error keeps getting thrown that I can't do the operation because I'm not the database owner. And the log says it was trying to write Data_Sources.user2.featureclass2 to the Operations.Ops dataset.The message is:Geodatabase Writer: Creating feature class `User2.Featureclass2' in feature dataset 'Operations.User1.Ops'An error occurred while attempting to create the feature class 'User2.Featureclass
I'm trying to sort through huge amounts of data. I have already run a directory file reader and have a huge CSV full of all the possible files in a directory including their name, path and extension etc. I now would like to push those file types into a new folder. This is for non-spatial, so a different folder for csv's, xsl, xlsx, doc, jpeg etc etcBy doing this I can then investigate these futher. Work so far: Reader: Read all files in directory and sub-directory that are, for example, a .csv I get an error and it won't create the reader, so I create it as a single CSV and then change the directory to the folder with \\**\\* at the and so it looks through all sub-directories. Writer is set to Dynamic and the same file type. I get errors from any of the above workings. Alternatively I feel like the original csv I have with all the directories included (including file path and extension) could be utilised for this?When I pull that CSV in it shows up like this instead of bro
A customer has FME Cloud. But now they need to write to GDB (ArcObjects) for raster writing.How should we add ESRI support?
Hi, I have some WFS that each contain multiple layers. When I add them with a WFS reader I have to check the parameters. When I want to write them in a GDB I have to choose the parameters. But then I have 2 Writers.Is there a possibility of writing several layers in one GDB?Additionally I am interested if it is somehow possible to write the parameters/lLayers from different WFS in one GDB?
Hi Everyone,I am trying to convert Esri .gdb file to PostgreSQL v 11.5 with PostGIS extension of v 2.5,an error message appears (Error executing SQL command ('SELECT AddGeometryColumn('public', 'UtilityNetwork_EDD', 'geom', importfromesri, 'GEOMETRY', 3)'): 'ERROR: column "importfromesri" does not exist)Can anyone help in solving this issue ´, I have searched for the same problem but nothing helped me, thanks in advance
I have a remote postgres / postgis db that I'd like to import data using .sql file resulting from an FME translation. Something like ogr's PGDump format using the COPY methodKindest regards...
Hello,I have a simple workspace. I read a xml service, use datetime converter then write DB. It works perfectly in FME Dekstop but when I publish and schecule job to FME server I got error below. DateTimeConverter_Rejector (TestFactory): The clause 'TEST @Value(fme_rejection_code) != FEATURE_TABLE_SHIM_SUPPORT Yes' within 'FACTORY_DEF * TestFactory FACTORY_NAME DateTimeConverter_Rejector INPUT FEATURE_TYPE ___TOREJECTOR___ TEST @Value(fme_rejection_code) != FEATURE_TABLE_SHIM_SUPPORT Yes OUTPUT PASSED FEATURE_TYPE DateTimeConverter_<REJECTED> @RemoveAttributes(___fme_rejection_code___) OUTPUT FAILED FEATURE_TYPE DateTimeConverter_OUTPUT @RenameAttributes(FME_STRICT,fme_rejection_code,___fme_rejection_code___)' is incorrect. TEST must look like: TEST <value> <operator> <value> [ENCODED]I can not find how to configure test parameters of datetimeConverter or solve this issue.I look into comminity and knowledge sections of safe and on the internet bu
Hello!I'm trying to parse JSON data returned from the API for Oracle Service Cloud. I've figured out the HTTPcaller piece and the data is returned like this:{ "count" : 10000, "name" : "OpenData REST export", "columnNames" : [ "Address", "Channel", "Date Closed", "Date Created", "Date Last Updated", "Disposition ID", "Latitude", "Longitude", "Product Hierarchy", "Product ID", "Service Request ID", "Status Type" ], "rows" : [ [ null, "Email", "'2018-06-20 12:46:36'", "'2018-06-19 17:36:19'", "'2019-08-20 18:22:39'", null, null, null, "Parking", "Valid monthly permit", "2296", "Solved" ], [ null, "Email", "'2018-09-06 10:13:19'", "'2018-06-19 18:28:00'", "'2019-08-20 18:22:38'", null, null, null, "Parking", "Disputing information on offence", "2299", "Solved" ], [ null, "Email", "'2018-06-28 14:45:50'", "'2018-06-19 19:07:40'", "'2019-08-20 18:22:38'", null, null, null, "Parks", "Residential new tree request / replacement tree", "2303", "Solved" ]...etc} I need to
i have some sample DTED (*.dt0 and *.dt1) files that contains grids of elevation data. I like to know if its is possible to convert this data in to 3D tiles and display them in a CesiumJS map? i tried some ways as seen in a post "How to Bring 2D Data into CesiumJS"from @dmitribagh but my inputs are different from that. so i appreciate if anyone can share any ideas. Thanks
I have a workbench that is reading from an already populated FGDB with 140 Feature Classes. I am trying to add an extra column to the majority of those feature classes and then populate the column for each row with a URL and ID to match the row. I am finding that it is taking 6 minutes to write back 60,000 rows and considering there are over 10 million to update that is an unacceptable timescale.I am creating the new attribute and a non spatial index with a python caller. Feature Reader to get the ID of each row. AttributeCreator and Manager to create the URL and merge it with the ID. Finally an AttributeFilter to the FeatureWriter. In the FeatureWriter I am doing an Update and using PrimaryKey columns for the row selection. I would appreciate any thoughts on how I might quicken the process.
Hi, sorry im struggleing to convert KML to shp, i can get the shp files converted but i cant seem to transfer any names of the files
I have tried to meka a flat file out of an xml file but I can't get the parent data onto every row of the child data. How can it be done? here is snippet of the xml file: <?xml version="1.0" encoding="UTF-8"?> <event Type="Relay"> <eventname>Lappee-Jukola 2016, Jukolan Viesti</eventname> <class> <classname>JU</classname> <team> <teamid>9</teamid> <teamname>Koovee</teamname> <teamnro>1</teamnro> <result>8:03:46</result> <tsecs>29026</tsecs> <placement>1</placement> <leg> <legnro>1</legnro> <nm>Topi Anjala</nm> ,....so I'd need to get all this data on the same row for each result:eventname, teamid, teamname, teamnro, result, tsecs, placement, legnro, nmJukola,9,Koovee,1,8:03:46,29026,1,1,Topi Anjala
Hi,is there any sources to use the git in FME unlike Eclipse, pycharm….
HiI have a csv file with the name of an electricity pole and the oblique images where the pole is visible on.Now I want to pull these images from a folder with a large collection of images containing the images I want and copy them to the folder with the csv. I dont know how to import a folder or a path. I have seen them but cant seem to find in the readers... I don't know how to copy. Please be elaborate with the explanation, because that's something I never have done with FME.
FME 2020 supports the Google Vision API with the transformer GoogleVisionConnector. The user just have to enter their Gmail address in the transformer (and then approve the connection with FME) to make use of the functionality. The documentation of the transformer informs that The GoogleVisionConnector uses your Google Cloud account credentials to access the Google Vision client services.With this forum question I just want to clarify the pricing of using the Google Vision API with the GoogleVisionConnector transformer. Is it correct that the transformer uses the API so that this pricing table applies? https://cloud.google.com/vision/pricingWhat happens when the amount of API calls exceeds 1000 units/month? I can assume that Google will send an email to the connect Gmail account but what happens in the GoogleVisionConnector transformer? What is the reply from the API and how is it interpreted in FME? Will it come out of the Rejected port?
H? all,I am trying to write my data over gdb and sde by processing. Some columns containing integers write null.fme is working fine. I'm not getting an error. but when I look at the database it looks null. How can I solve this problem. Thank You.
Hi, we would like create some terrain models for cesium - is it possible to create TIN quantised meshes that are directly compatible with CesiumJS?Thank you
I am currently updating a number of documents and models on a CDE (currently trying to pull the API information) that require manually logging into the system as an administrator, deleting the files in question and then uploading the updated file.I wanted to see if there was a way to automate this procedure through an FME workbench?
Hi All, I am really excited to learn new tool and work with new tool, new project new task really challenging !AS, I said, I am new to FME data integration Platform, I am really struggling where can I start. AS an Automation tester came from python background, I need to automated the TOID lines, sites..., My project include with Technologies ArcGIS and FME. Could you please suggest me any tutorial or any use case related to Geo spatial Maps.(I have excel, .fme, .gpkg files). Apologies for lengthy question, I really nee to run the test and develop more based on them ThanksRegards
Hello. I have a txt file which contains "blocks" of data, separated by line containing #####. I need to extract information from each "block", and each time the #### appears that means the new epoch starts. For each block I check if it contains eg. "Station point:", "PPM", "MWF". Each Feature is then written in DB. Not all blocks give results for each Tester.Here is an example of such file:2020-03-23 08:01:13.8810 Info ############################################################################################################################################### 2020-03-23 08:01:13.8810 Info Processing pointgroup 'MWS200_Fix' (Type:'Freie Stationierung') measured by 'MWS200'. [GON/M] 2020-03-23 08:01:13.8810 Info Station point: 'MWS200' 2020-03-23 08:01:13.9122 Info Station coords: 2660392.8147,1251984.6680,416.3654 2020-03-23 08:01:13.9435 Info Reference coordinates of 'MWF204': E=2660397.8542,N=12