Using readers and writers to extract and load data.
Recently active
I am working on a workspace that reads Tourism Information Office data from an XML file, converts the attributes to OSM tags and writes it to an OSM file. This OSM file is then processed by a program called OsmAndMapCreator to create an obf (OsmAnd Binary Format) file for the OsmAnd mobile application. Each office has a collection of characteristics and services so there is more than 1 tag for each node. The order in which those tags are written to the OSM file is important for how the feature will appear on the map. A tourism information office that has parking or picnic tables still needs to appear as a tourism information office no matter what else they offer. The OSM writer does not maintain the order of its reader or User Attributes list. In my reader and writer the tourism tag is at the top but for some reason, it is always the last one in my OSM file. Here's 2 nodes where it creates a problem.This node shows up as a picnic table on my map but it should appear has a tou
Is it possible for FME to convert OpenDrive (.xml) to 3D SHP?
Hi everyone,I have a BIM (revit file) that i would like to transform to Cesium3Dtiles. I used the "Revit Geolocation" Tutorial of @daveatsafe and exported the model into IFC. When I look in the FME Data Inspector the IFC file looks fine (I think). But when I add a Cesium3Dtiles writer I get an reprojection error. The coordinate system of the IFC file is EPSG:3857 and the coordinate system of the Cesium3Dtiles schould also be EPSG:3857. So I don't understand why there is a problem with the reprojector.
I need to publish a workspace to fme server that will receive a geopackage file as base64 binary and will do some schema and data validation before inserting geopackage data into a number of Oracle tables. Actually an API will call the FME service providing that base64 binary and some other information.Now my question is how to read geopackage schema and data into fme. Any idea? I don't see how to get the binary back into a geopackage file.Thanks!
I've installed the Google BigQuery package into 2020.1 and I've got a web connection to a BigQuery project. The connection seems to work since I can see the dataset and tables listed. I am sending a simple SQL 'SELECT * FROM...' and I've tried 'SELECT col1, col2 FROM ...' I want the query to return records from a non-spatial BigQuery table to use in FME. I don't get any results and the log doesn't seem to give anything helpful. We've opened all the permissions on the account. Does anyone have any advice on how to get this to work?
Hello! I'm reaching out because I'm in aa bit of a pickle. I have a task to pull a large number of photos and folders from a ProjectWise folder. I've successfully been able to do so through the ProjectWiseWSGConnector and FeatureReader, and then a number of other transformers that modifies the images for what we need. However, because the number of photos is quite large (GB's of data), it's quite the task to run it and to re-download all of the files. Is there a workflow that will skip over previously downloaded files and just download the newly added or updated ones? Thanks for any help!
Hello,I am trying to get the information on files stored in ProjectWise. I am using FME 2020.1.However, the ProjectWiseConnector, seems to be unable to get all attributes, i.e. Version and Status.Also tried custom ProjectWiseNavigationHierarchyRetriver, which looks to be able to work with all attributes, but only works with API v.1.2 and 1.3 and we are on 2.4.Are there any other transformers or a newer version of the above that could read all file attributes?Many thanksMagda
Hi, I am trying the write out using the to a file geodatabase using a feature writer but am unable to remove the feature datasets from the data. I am using OPEN API as i am reading in all the fgdbs in one directory and writing them to a new location and splitting the data back to the original fgdb names, but don't want the feature data sets to be used (one of the main reasons for doing this) and i can not get the feature writer to writer to different fgdbs.Below is the basic work space so far, the schema reader reads all the feature classes in the fgdbs then gets the feature reader to read those tables and send them to be saved in the new location by the feature writer. This will be the first stage of a data update process, eventually the workbench will read new data that has come in, compare it with a previous version and then update the file gdb with the changes (along with details of the change user, date and source). Each time data is delivered there could all or just some of t
I've been given a sample from a CityGML dataset that I want to eventually convert into an FBX 3d model. Now, before receiving this sample, I had built a workflow that did the entire conversion from a CityGML dataset into FBX models, tested it out with different CityGML datasets from open data repositories and it worked just fine. But in all of those datasets, the geometry was made up of surfaces, while on this sample there are just lines.I've been messing around with the AreaBuilder transformer but even though some of the surfaces are generated, I can't seem to get it nowhere near what it should be like.Any ideas?The sample: https://drive.google.com/drive/folders/1JAKSgxgjPMRfwJoDgCaKPValeaZRmO88
I uploaded a table into MSSQL server, but the table is not visible in QGIS.There is no issue viewing the table in SQL Server Management Studio, and the primary key is generated by counter.So far, I tried turn Spatial Index on and off; Change the collection type handling; Change Insert into Identify Column on/off. None of those steps work.Any idea how to modify the writer to let the table visible in QGIS? Update:In ArcGIS Pro, the table is visible, but can't add to map, gives an error message.
Is it free to use or does it consume credits?
I have cad file I am trying to convert to a File Geodatabase. The CAD file has labels that I would like to convert to geodatabase annotations. When I generate a workspace specifying the DWG and output as File geodatabase, I get a few features that are empty and a blank gdb table. Any ideas on how to do this? Thanks! Attached is the cad file and a screenshot of the workspace I have.
I am working on a FME script that is pulling from a raster in a map service: https://maps.nccs.nasa.gov/server/rest/services/global_landslide_catalog/landslide_susceptibility/MapServer When I do so using a Esri ArcGIS Server Feature Service I get a unsupported error. How can I get the raster into a workbench to do spatial queries against? Thank you in Advance!
Has anyone found that FME unprotects excel worksheets when it writes to them? I have a worksheet that I need to keep protected with certain cells being written to only. The rest of the sheet has loads of complex formulae which need to be kept protected.I have protected the sheet but when FME writes it unprotects it without the password. Does anyone know how I can stop FME from unprotecting the worksheet?
I have created an excel reader with the merged feature type because the source dataset as well as feature type (sheet name) changes with each run. But when I am using merged feature types, I cannot get the .formatting. THey are all coming over as "missing". I would have to specify a specific feature type (sheet name), but because it changes each time (not named Sheet 1), I cannot specify that at creation time.Is there not a way to do this? Am I missing something?
Hello everyone,I am currently trying to get a list of existing file names inside a folder. Here are some explanations :- I have a Path reader which reads existing files in a folder (let's call it folder 1). In this folder, there should be one text file;- If the text file exists, I use a FeatureReader to read it. This text file contains several paths to other directories (folders 2, 3, …, n).- Once I extracted those new path, I want to do the same as with the reader : get the names of the existing files inside folders 2,...,n. For now, I know how to read files from within the Workbench, with a FeatureReader. But how can I do the exact same thing as the Path reader without using one ? I do not need to read the files right away, I just want to get the names of the files Inside folders 2,...,n. The idea is to do all those operations in the same fmw script.Any idea ?
I'm running a workspace with a creator→hhtpcaller and I would like to have the response saved into a new csv (excel) file as a regular table format. This is the API as reference: https://datos.cdmx.gob.mx/api/records/1.0/search/?dataset=casos-asociados-a-covid-19&q=&rows=10000&facet=fecha_actualizacion&facet=id_registro&facet=origen&facet=sector&facet=entidad_um&facet=sexo&facet=entidad_nac&facet=entidad_res&facet=municipio_res&facet=tipo_paciente&facet=fecha_ingreso&facet=fecha_def&facet=edad&facet=nacionalidad&facet=embarazo&facet=habla_lengua_indi&facet=diabetes&facet=epoc&facet=asma&facet=inmusupr&facet=hipertension&facet=cardiovascular&facet=obesidad&facet=renal_cronica&facet=tabaquismo&facet=negativo&facet=migrante&facet=pais_nacionalidad&facet=rango_edad&facet=positivo&facet=pendiente&refine.entidad_res=GUANAJUATO Do you guys know what can I do?
The WFS reader automatically chooses the default srs of a WFS layer. Is there any way to make it read the features in a different srs which is supplied by the WFS supplier (and so also listed as option in the WFS capabilities). Changing the coordinate system of the reader doesn't work. It will still request the features in the WFS default srs and then interpret it (wrongly) as the coordinate system set at reader level.
I am attempting to read photos from attachment tables in a FGDB and write them to an Access DB, where we have a workflow for report generation. I can get the photos out easily enough and write them to a folder, but I can't find the proper settings in the Access writer to add the photos.
Hello,my problem is to style a simple Excel table.When cell xy is 1 --> colour background set to red.It is working in many cases only the first row doesn't turn red. What am I doing wrong?Result in Excel: Thanks for your help. :)
After running a feature service reader or writer a bunch of information is logged such as:ArcGIS Online Feature Service Writer: Getting item with ID '2b51386cf7b346d59fda3ff653deccc3'ArcGIS Online Feature Service Writer: Requesting metadata for Feature Service at 'https://services5.arcgis.com/7X0RdDUh866P5Y57/arcgis/rest/services/FeatureServiceName/FeatureServer' Is there anyway to get the item ID and URL of the feature service as attributes to use in FME? I want to be able to get the URL of the feature service that has just been written to, so I can then do another web request, but I don't seem to be able to get those attributes from the writer?
HelloI would really appreciate it if someone could assist me in resolving the below issue as soon as.My AD account shifted to a different OU yesterday and as a result the canonical object name has changed. I can no longer login however the scheduled jobs are still running. I must investigate 2 failed jobs but can't do so anymore.I came across the following: https://knowledge.safe.com/idea/75118/fme-server-active-directory-away-from-distinguishe.html and wonder if that could be related. Sync AD did not work. Checked logs and nothing obvious found.I've moved my AD user account back to the original OU and I can now login but my user account must be shifted back to the new OU. I think what I'll need to do now is to re-assign the jobs to a different "owner" to avoid the above issue. The new "owner" will be a ServiceAccount type.Questions: Is there a way to manually amend DN / CN Names? How to re-assign all jobs to a new "Owner"? I'm planning on
My goal is to read log files from Amazon 33 and parse them into a databaseThe log files look like this:-#Version: 1.0#Fields: date time x-edge-location sc-bytes c-ip cs-method cs(Host) cs-uri-stem sc-status cs(Referer) cs(User-Agent) cs-uri-query cs(Cookie) x-edge-result-type x-edge-request-id x-host-header cs-protocol cs-bytes time-taken x-forwarded-for ssl-protocol ssl-cipher x-edge-response-result-type cs-protocol-version fle-status fle-encrypted-fields c-port time-to-first-byte x-edge-detailed-result-type sc-content-type sc-content-len sc-range-start sc-range-end2020-07-03 13:39:54 LHR62-C3 1571 148.00.00.00 GET d36on651kzt577.cloudfront.net / 200 https://URL/2020/07/02/fooo/ Mozilla/5.0%20(Windows%20NT%2010.0;%20Win64;%20x64)%20AppleWebKit/537.36%20(KHTML,%20like%20Gecko)%20Chrome/83.0.4103.116%20Safari/537.36 - - Hit gIanUGtmvquSunAiRJFbhFdPbexwpIV2DbtYUJ7XtVOKZopkUl1uEw== foo.com https 427 0.001 - TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256 Hit HTTP/2.0 - - 58606 0.001 Hit text/html;%20
I am complete novice to FME. I have a .txt file which I wish to convert to excel. But I only want some of the information to be brought into the excel spreadsheet. I'm not sure how to begin. All I want in the excel spreadsheet (bottom image) is the name e.g HP40 but without the .tif extension and the date e.g 04/03/2016 without the time.I am not sure what connectors to use or how to strip out the unwanted info from the .txt file.
HI EVERYONE. I AM LOOKING TO CONSTITUTE A GEOREFERENTIATED ENERGY DATABASE STARTING FROM REVIT. I AM NEW TO THE THEME, AND I HOPE YOU CAN HELP ME. IF I AM NOT WRONG, I have to CONVERT THE REVIT OR IFC FILE (VERY SIMPLE GEOMETRY) IN CITYGML AND THEN ADD ENERGY DATA WITH ADE? After this, on which tool can I see the mapped information? are there any tutorials to do it or some examples?Thank you very much for your help