Using readers and writers to extract and load data.
Recently active
The input-data are in e57-formate and we want to split them up into a huge amount of small pnts-files for using them in Cesium JS.We need a tileset.json-file for each pnts-file. There should be an overall tileset.json-file in an superior folder listing all children tileset.json-files. Each generated file (pnts and tileset.jsons) should not exceed a specific file size (e.g. 2 MB). We are using version FME 2023.0. Our workbench has an e57-point-cloud-reader, a PointCloudExpressionEvaluator, a PointCloudSplitter and a Cesium-point-cloud writer. The data are splitted by the attribute set in the PointCloudSplitter. How is it possible to get multilevel tileset.jsons? We already created an superior tileset.json-file and many pnts-files. The problem in this case is the big size of the generated tileset.json. In another case many tileset.json- and pnts-file-doublets. The problem in this case is the missing tileset.json-entry-file.
Does anyone know of a way to use FME to generate a .lyrx file with symbology that I could then use in ArcGIS Pro? I think a lyrx file is just XML under the hood? Making and remaking layers in ArcGIS Pro is a pretty huge pain point for me, and I'd love to see someone make a transformer / writer to help with this, so that I can just specify a data source, specify symbology parameters and click 'Run' in FME to generate this > add to ArcGIS Pro, instead of using ArcGIS Pro's laggy interface to click around in the symbology pane. @mark2atsafe thought you might know of a solution? Thank you
I've downloaded a geopackage file from Data Map Wales (in the UK) containing listed buildings and whenever I load it into FME Desktop it seems to miss a field containing a date, all the other fields are there. I've used DB Browser for SQLite to read the file and the missing field is definitely in there and contains data. In FME Desktop I've tried using a reader and a featurereader to load the file, both approaches fail to recognise this field. I've also tried in QGIS and the field does not load in there either. I initially thought that the data might be malformed in some way but the fact that it reads fine in DB Browser for SQLite suggests otherwise. Does anybody have any ideas as to how I can read this date field? The data is available from this website Listed Buildings | DataMapWales (gov.wales)
I need to create a summary slide with multiple sections/placeholder tables on one slide, so I need to have my font size to be 10pt for example. So far it seems that FME hard codes the table font to 18pt. Thanks, Joey
I want to reproject DWG files with Pipe Network features from UTM Zone 17 NAD83 (Original) to MTM 10 NAD83 CSRS using ON83CSv1.gsb.when I use ArcGIS data interoperability and EsriReprojector, it reproject all feature except Pipe Network features.is there any sollution for it?
Hello,I am experiencing difficulties reading an Oracle Spatial table with FME.The table (linestring) has about 250K records.FME starts to log the first 10000 records after about 10 minutes. And then 10000 at a time every 6 minutes.The table is displayed in GeoMedia and also QGis in about 5 minutes.Feature Caching is disabled.I have tried both with the reader and with a SqlCreator. Same results.What checks/configurations can I try?This is the first time it happens and not sure if some metadata could be wrong or corrupted.This sounds strange because of the tests with other products.TIARoberto
Has anyone encountered issues with 4-band jp2000 files written by FME Server crashing in Kakadu? The error is:"A `jp2_channels' object indicates the presence of " "more colour channels than the number which is associated with the " "specified colour space. This may happen while reading a JP2-family " "data source which contains an illegal channel definitions (cdef) " "box, or it may happen while writing a JP2-family file if the " "`jp2_channels' object has been incorrectly initialized."
I have a logfile I need to extract some info from but I'm not sure what to use. Here's an example of what I have.<Msg time="2023-06-05T19:55:23,511" type="INFO" code="9029" source="Rest" process="35236" thread="1" methodName=""
What would be the most convenient technique for querying data stored on Azure Data Explorer (ADX) from FME Workbench? Should this be done via a HTTPCaller?
Hi All,I'm trying to update MDB_ADO reader in existing workspace - it's new data delivery with the same structure as the previous one. I'm getting "An error occurred while updating the reader". I tried also open some random tables in blank workspace, and I was able to do it, but after ran workspace I got "Error running translation" or FME has been closed "automatically".At first I tried to do it in FME 2022.1.1.0 (64 bit), MS Office is 64-bit as well.Then I installed the newest version of FME 2022.2.5.0 - at first everything worked fine - I updated mdb reader and I was able to run the whole workspace. When I closed FME and opened it again I wasn't able to update reader or open any other random tables in blank workspace - so same errors like in older FME version.I tried clean all temp files, I created new temp folder, I tried to repair mdb via "Compact and Repair function in AccessDB. After all of these changes I was able again to work
I would like to be able to use FME to bulk upload data to salesforce. We currently manually upload csv files with awareness that there is a limit of 50,000 rows per file (salesforce import wizard). I have seen it should be possible to automate this using the salesforce reader/writer. I have the API credentials for salesforce and I can see all the table objects available in FME. The issue is that the object we are trying to write to appears to have different fields than expected when read (these look to be metadata fields) - see image. When we run the process to manually import using the wizard the attributes are those in the csv uploaded (see image). Both of these tables are the custom table 'property(_c)', the saleforce URL indicates that this is the case. Do you need to change permissions on the object salesforce side or do anything specific with the reader/writer in FME to get this working? Kind regards, Antony
I am trying to connect to my Smallworld 5 database with FME 2022. I can connect successfully with 2021. I have FME Smallworld edition and have copied the needed connection files to the installation folder. What have changed between the two versions? It says 'This FME edition does not recognize the 'SWORLDSWAF' reader.
I would like a PRJ file written alongside the DWG when using a dynamic format workflow. The bespoke AutoCAD writer has a setting, just looking for how to do with with a dynamic workflow.
Hi When working with full Desktop FME it is possible to set where the log file is written to disk through the GUI (or set it to be a parameter), however when working with the ArcGIS Data Interoperability Extension version setting the log file location is not available in the GUI (none of the workspace parameters are). Is it still possible to set where the log file should be written to on disk through a variable or is this functionality removed for a reason? Ideally, I would like to write the log file to sit alongside the data being processed. ArcGIS Desktop 10.5 which uses FME 2016 Many thanks Duncan
I have an FME workspace in which input from an DWG file is read and exported into an FGBD file. I am using a dynamic FeatureWriter. For the layers, this has the following result: If there is a layer named 'xyz', and this layer contains e.g. text, line, and point features, the FGDB writer will create three layers 'xyz_text', 'xyz_line', and 'xyz_point' for the features. Is there any possibility to tell the FGDB writer to group the layers such that in e.g. ArcGIS Pro, all the layers 'xyz_text', 'xyz_line', and 'xyz_point' are already grouped below a parent layer/group layer 'xyz'? Or is this something which necessarily has to be achieved by another tool, e.g. using ArcPy?
Hi,I read an Oracle Database with "esri SDE" connection file. I need to update the reader. We now have a new Oracle table that contain a new column and the datatype of that column is "xmltype". With a Oracle Reader or SQLExecutor, I can read the "xmltype" data. But if I read the data with "SDE", the column is not there. Like SDE don't understand "xmltype". Any idea ?
I want to be able to read a list of PDFs from SharePoint and publish them on AGOL as PDF.
In QGIS I can open Google Street View coverage using this URL as XYZ tile:https://mts2.google.com/mapslt?lyrs%3Dsvv%26x%3D{x}%26y%3D{y}%26z%3D{z}%26w%3D256%26h%3D256%26hl%3Den&style%3D40,18 The result is following: I can't get it to read into FME Desktop. Any suggestions?
This may be a little simple but it seems not to work for me, I have a workspace that reads a .csv file and runs a process. Ideally i would like to move this csv file once the process is complete and rename it by date. What i have tried so far,After the reader i created a junction and I put one process to run through the whole workspace and another arrow i put to a DateTime stamper, DateTimeConverter and created the attributes filecopy_source_dataset, filecopy_dest_dataset and filecopy_dest_filename, (based on several tutorials i have been reading.) I then use the filecopy writer to move the file. The file gets moved but the name is not changed and i get a warning that --> Ignoring the destination filename .. because the source dataset...was not a file. Destination file names are only used when the source dataset is a file. For some reason the exact name i did set (@Value(Datum)_@Value(filecopy_dest_filename)) is shown in the written summary but not on the folder, anyone with an ide
Hi, I'm using FME workbench 2022.2, Build 22789 - WIN64. I'm trying to process several thousand JP2 files.I'm finding that trying to open just one JP2 file (~1.5GB) in the inspector doesn't seem to work ... by that I mean I stop the process after waiting about 30 minutes. Thanks in advance for help!
I can set an index on an integer or even a unique index, but there is no type for a primary key or autoincrement. How can I specify this? I could do this with an SQL Create expression. Can I use SQL to create a schema?I am adding a changeset with new records using insert but I need an OID column for ArcGIS to be able to use the table/featureclass separate from the real ID.
I was just wondering if anyone has done sunlight / shading analysis in FME? In Rhino 3d + Grasshopper, Ladybug and Honeybee are plugins that can do this. Below a few images. I know that one can do some interesting stuff with the surface footprint replacer, and change the angles, and thereby create shadows. But thought I would ask before diving deeper. Maybe another option would be to go the ESRI route. Ladybug:Ladybug Tools | Food4Rhino CheersFred
FME(R) 2022.2.5.0 (20230329 - Build 22795 - WIN64 Hello Just detected this possible bug when importing Attribute Names / Values using Attribute Manager. The Names are pre-sorted and the "Sorted" option unticked as shown below. Once the dataset has been imported (Select * for Input/Output/Values) the values are set as shown below. Was I not meant to see the actual values or have I misunderstood this part? Looks like the Attribute Names have been set as values. If that's intended then what could be a potential use case? Original Attribute Names order / Table structure as defined in the Postgis Db ThanksSameer
Hi,I have been investigating this for some time, but did not find any solution so far: In order to read sharepoint lists from a predefined sharepoint site, I used the web connection "Microsoft SharePoint Online (Template)". As testing / reauthentication was successful, I felt ready to read a list from this specific sharepoint page so I inserted its URL and indicated my web connection. However, when I click on "Lists" to retrieve the lists I created, I get an error message (see attached). The error in the log furthermore shows:"Failed to obtain any schemas from reader 'SHAREPOINT' from 1 datasets. This may be due to invalid datasets or format accessibility issues due to licensing, dependencies, or module loading. See logfile for more information" I am using FME 2020.2. Do you know what I am missing here?
In both 2022.2.5 and 2023.0.0, I am unable to read DWG files or DGN files. The translation log reads:DWGAutoCAD Reader: An error occurred for dataset 'xxx.dwg', in function AutoCAD2004Reader::open. Exception thrown: 'Extended error' (The file cannot be accessed by the system. SCENEOE_23.9fme_17.tx) Failed to obtain any schemas from reader 'ACAD' from 1 datasets. This may be due to invalid datasets or format accessibility issues due to licensing, dependencies, or module loading. See logfile for more information Translation FAILED.Error running translation.Workspace generation failed. Parameter file was C:\Users\JSDBRO~1\AppData\Local\Temp\FME_1683040838597_37964.DGNFME API version of module 'acad/ACAD' matches current internal version (4.0 20220615) AutoCAD Reader: An error occurred for dataset 'xxx.dgn', in function AutoCAD2004Reader::open. Exception thrown: 'Extended error' (The file cannot be accessed by the system. SCENEOE_23.4fme_16.tx) Translation was SUCCESSFULError running tran