Using readers and writers to extract and load data.
Recently active
Working with esri and the developers for ArcGIS Online, we came up with this script that can be used on shutdown. You will need to write data out to a zipped file geodatabase (or other formats), then configure the script.Only works with 2019.x+ as it requires you to use ArcGIS Pro (tested in 2.3). It does take a while to run (as it has to use the ArcGIS Python library), but is SUPER efficient when trying to write LARGE datasets to ArcGIS Online. It is much preferred over using the FME writer because that one uses the traditional database insert for each record while this is similar to bulk inserts. Can probably be modified for writing to Portal as well.https://github.com/runneals/GISProjects/tree/master/agol/Appendcc @carsonlam @bruceharold @mmccart
Hi All, we are trying to export data from Smallworld to Mapinfo Tab format. In Smallwrold we have objects like Underground Route and Cables, which are having length fields with values available in millimeters, but these values are getting exported as meters to Mapinfo tab, how to keep the values as it is while exporting. Please help. @takashi please help in this regard. Thanks in advance.
I have a pdf that has blank boxes that are editable. Is there a way to write to them? Attaching sample pdf to show how the pdf would look.
I have geometric data that I want to join with non-geometric data. From what I read and noticed, a FeatureJoiner won't keep the geometry during a join. I would use a FeatureMerger, but the cardinality of my features won't make it work. I tried aggregating one of my feature to make it possible for me to use the FeatureMerger, but the aggregations are not supported by ArcGisOnline (that's where I want to write). I ran out of ideas to solve this problem, do you have any solutions? Thank you
One of our customers still uses v2007 of FME Workbench and I'm having a bad time trying to do what I find so easy with latter versions.Workbench help doesn't help much and I can't find examples on the internet so if anyone has a clue on how to do it I'll be very grateful.I want to use attribute values in sql statements but SQLExecutor transformer in FME v2007 looks too basic and I don't know if it's even possible to do such a thing. The image below shows what I tried and doesn't work at all. I get error: BADNEWS: Database Writer: Commit failed -- [NOTSUPPORTED] commitTransaction not supported by core library
Morning,Does anyone know if there is a way of taking monthly excel reports and adding them to a single google sheet with monthly tabs.ie Monthly calls for each month are produced in excel - what we would like to do is when a new report is put into a centralised location fme server will detect a new spreadsheet and then the workspace would run and add the new month to the google sheet. Thanks
Is there a way to retrieve the actual record that fails a database constraint when trying to insert in an oracle table (using Oracle Non-spatial writer) instead of the SQL statement? It seems as if FME only outputs the SQL statement that fails and the oracle error message, but can't seem to find a way to get the actual record that fails a constraint.
Hi @markatsafe, I am trying to read in a Microsoft SharePoint List (from a server not sharepoint online). I have tried in both FME 2017.1 and FME 2018.1 and am having no luck. Can you please review the error message and let me know what the issue may be? I have been told that I have read-only permissions to the list.Also, what do I put in the 'Site:' parameter? Is it the link to the actual list itself that I have read-only access to, or is to the 'Lists' folder or the base URL of the whole site? I am wondering if the issue is related to me only having access to one list on the site....Thank you in advance
Hi, I've a got point layer in shp format and a pointcloud file in las format and I'd like to find the nearest point in the las file for every point in shp file. I tried to use PointCloudCoercer to change las file into mutipoint, but z value of points won't be merged. Then I tried to change las file into individual points, but it was too slow, I got thousands of las files and each of them contains over 1 million points.Can anyone provide a right and fast solution to this matter? Thank you!@takashi @jdh @danatsafe @mark2atsafe
Hello!I have nested blocks on autocad because I need to have the same block with the insert location on a other place. So, on autocad, when you explode the block1 you obtain the block2. I cannot change this configuration because is used for a long time. The 2 blocks have the same name.Is it possible to extract the block nested (block2) without explode it? Or at least extract the insert location of the block 2? When I explode the block on FME reader parameters, FME explode all the elements and keep only the insert location of the block1. There is no option of "hierarchical" explosion.Thank you!
Does anyone know if there a possibility to use FME and ArcGISpro to import files into Smallworld 5? I know that they are different architectures(64/32bit), but you think there might be a medium that could be utilized. Using the data interoperability extension in ArcGISPro doesn't give the potential to create a Smallworld 5 writer.
I'm trying to update an AGOL layer directly from FME instead of overwriting it from a file geodatabase in ArcGIS Pro to automate my workflow.I am getting there following error as soon as the first item reaches the writer:"Reproject: Could not reproject the geometry of the preceding feature: Reprojector: Invalid reprojection parameters%0 Writer: Failed to reproject feature with coordinate system '%1' to '%2'. Verify that the coordinates and coordinate systems are valid, and try againA fatal error has occurred. Check the logfile above for details" I am using an ARCGISONLINEFEATURES writer, trying to write information that is already in British National Grid (BNG), to a hosted feature layer in AGOL that I own and is editable and is also to BNG. I have the source and destination coordinate systems set to BNG, so a reprojection shouldn't be required, but the error message above is suggesting I need to do this.I can get the data to update/publish fine if I
I am writing in regards to an issue when writing features to Portal Hosted Feature Service using FME Desktop. Please see the details below. Version: FME(R) 2018.1.1.2 (20190121 - Build 18586 - WIN64)Operating System: Microsoft Windows 10 64-bit (Build 15063)Issue: The translation log indicates that the translation was successful with all features written (#4622) to ArcGIS Portal Feature Service. When reviewing the portal feature service, it is only writing 4322 features missing 300 features with no errors in the log file indicating the same.Any ideas on why it would not write exactly 300 features to portal feature service? See the attachment for more details.
Since the LogMessageStreamer doesn't have an input, I created a custom transformer with the LogMessageStreamer an emailer and just an input (which isn't hooked to anything). I hooked the last transformer in my workspace (a FeatureWriter) to the input and that way the custom transformer doesn't fire off until after the feature writer is done. That way, at least in initial testing, I get to scan all the logs before I email if I find an error.This seems a bit kludgy so it would be good to know if there's a better way. The errors I'm particularly interested in are logged by the SDE writer which has a parameter to validate features before writing them. I want to log if one fails but move on and not fail the workspace for the rest of the features.
Hi, Do you know if it's possible to make classification of lidar point? Thanks for your help
I am using the oracle nonspatial writer in FME Desktop 2015 and I am trying to write to an Oracle table to update 2 fields. I do not see an error when the process finishes but I noticed that the changes are not reflecting in the table. I don't see any options to commit changes. How can I ensure that the changes are being committed? Do you know what would cause the changes to not be committed? Thanks for your help.
I have a Civil3D dwg file and have tried so many parameters in the reader properties to try to see if I can see the custom "Note" field that is in there to no avail. The creator of the dwg file said it's embedded in the cogo points layer and I see the cogo points layer and associated fields but not that custom field. Is there something that I'm missing? Thanks!
I have a FeatureWriter connected to a Google Sheet. That FeatureWriter's input are files that were just uploaded to a Google Drive (through a GoogleDriveConnector). I want the FeatureWriter to update the Google Sheet with each of the uploaded file's informations but right now only the informations about the first uploaded file appear. How can I make it so all the files are going to be written at some point? I know Google Sheet doesn't support insert but I have a script that deals with that part, I just need the information of the files to be written in the Sheet at some point. Thank you
My problem:I have a FME Server engine running on our ArcGIS Enterprise server (10.6.1).I have a workspace which needs some ArcPy methods only availible in ArcPy-proSetting the Python Compatibility in Workbench to Esri ArcGIS Pro 2.1/2.2/2.3 does not work (can't find...)My attempt to fix:On the Enterprise server, an 3.6 environment lives inC:\ArcGIS\Server\framework\runtime\ArcGIS\bin\Python\envs\arcgispro-py3Hidden, to be used in services published with ArcGIS Pro.I set my custom-python variables toPython: Custom Python Home 64: C:\ArcGIS\Server\framework\runtime\ArcGIS\bin\Python\envs\arcgispro-py3\ Python Interpreter 64: C:\ArcGIS\Server\framework\runtime\ArcGIS\bin\Python\envs\arcgispro-py3\python36.dll Use Custom Python 64: trueNow, when I choose 'Python 3.6+', this python is used. I do reorder my python-path, so the ESRI stuff comes before the FME stuff.Import arcpy yields a complex error, (something with numpy and the version of python 3.6 against which numpy
A customer is planning to deploy FME Desktop 2018.1 to a VM hosted on IBM Cloud (formerly SoftLayer). This is not a shared-tenant VM but it could conceivably be moved to different hardware from time to time. The VM will be running Windows Server 2016.Will a stand-alone license for FME Desktop work correctly in this context?
Hi,can somebody help me with this error?I did run the workspace successfuly and the datas are inside the folder.When I want to inspect the writer, I become an error:"Shape Reader: Failed to open dataset 'C:\\Data\\Adressen_Shape_3_BL\\Hessen\\Adressen_Deutschland.shp' which is neither an existing file nor directory"I has never happened before.
The SharePoint-reader currently doesn't support SharePoint Online with ADFS login method. Our ADFS server is directly accessible by an url that looks like https://ADFSserver/.../SharePointURL, where the user can fill in the login form and submit the login request.Is it possible to access the SharePoint-site and read its contents by sending (several) http requests?
What does this error refer to? Warning: Unable to perform SE_layer_alter operation. (Continuing):SDE_ERROR_CODE(-20) Layer not found(Table: 'TAMISADMIN.T_TRANSMISSIONSTRUCTURE' may be in use.) I have two pipelines off a Feature Merger, and I'm testing just one (I have the connection disabled for the other one.). I'm reading from ESRI using the SDE30 reader and writing to an SDE30 table, T_TRANSMISSIONSTRUCTURE. and i'm updating the data in table, so my writer mode = UPDATE. I have a 2nd writer to the same database with the same T_TRANSMISSIONSTRUCURE feature class. on this writer/feature class i'm inserting records. Could the error be related to the existance of the same feature class being accessed in this translation? Sarah
Hello everyoneI have a directory of PDF files and would like to select some of them for merging.Is there a way of merging a list of PDF files to a multi-page PDF file?Thanks.
Hi,I would like to use the FeatureReader to read the data from the PostgreSQL (PostGIS) and save it to the GML (3.2.1), Esri Shapefile and Mapinfo TAB.The source table contains multiple geometry columns.Do those formats and/or writers support multiple geometries.Best regards, Damian