Using readers and writers to extract and load data.
Recently active
Hello.I am trying to use the Google Drive Connector to download files from a Shared Drive. But when I click on the Google Drive Source in the transformer all I see is the folders and files which are in My Drive. It doesn't show shared drives which I have created as well as drives someone has shared with me.I'm just wondering if this is a bug and if anyone else has the same issue?Many thanks.
Has anyone managed to construct a workspace with the following requirements:Only to be used in Datastreaming serviceGeneric writer with different file based outputs (demo pdf+png)FME Server streams the correct mime type with the response fileMy current experiment was to use a private python parameter which generates the correct mime type string depending on the format. But even with this, the data streaming sends a PDF but the response headers contain "Content-Type ?image/png".Is this possible at all?
I recently installed FME Server to docker using the installation guide. Since yesterday I have been trying to test it using a simple .fmw in order to write some data to a MongoDB instance. I tried in both local MongoDB and in a MongoDB-docker instance. In both cases I am getting the same error: MongoDB writer: Cannot connect: couldn't connect to server <<PORT NUMBER>>, connection attempt failed.Please note that, I have access to both MongoDB instances from local FME Workbench and I can execute the same .fmw. Find attached the file. This seems to be an FME docker network issue, but I cannot be sure what. Any ideas?
I have been trying to report an issue related to a backup file which is bigger as the 5MB. Following https://community.safe.com/s/article/upload-a-file-to-the-safe-software-support-ftp I have tried to upload the respective file through the given `curl` command. But whatever I try, I keep getting "Failed FTP upload: 553". I tried both zipped and unzipped, where the zipped file is around 6GB. Is there anything I can do to overcome this and continue with problem report?
HiI've been given a JSON file containing registrations for 5 properties. We are looking to use other datasets with a single field in the json file to identify the location of the property. I am looking to produce a csv file initially with each record on a separate line Can someone comfirm if the following is correct or not please. Open the JSON file using the JSONFeature Reader Connected a JSONFragmenter, using one of the fields as the JSON Query. I've attached a screenshot of the settings for the JSONFragmenter. When i run the workspace and view the result in the Inspector,i can see the records listed in the right hand pane, but writing out a csv file with each record on a separate line doesn't work. thanks for any assistance
Hi, I am trying to build a mosaic from 400GB of TIFF files - but is continuous failDiskspace Available 4TB - FME Temp file 1TBHas anyone come across this error - can it that the TIFF is corrupted? 445 2020-12-12 14:18:51 | JPEG2000 writer: Completed 21.76% of intermediate processing446 2020-12-12 14:19:13 | TIFF reader: LZWDecode:Wrong length of decoded string: data probably corrupted at scanline 27520447 2020-12-12 14:19:13 | TIFF reader: TIFFReadEncodedTile() failed.448 2020-12-12 14:19:13 | TIFF reader: \\\\mcc\\gis\\SourceImagery\\Taylors\\2020-10-30_Deliverable\\OrthoImage\\B4.tif, band 1: IReadBlock failed at X offset 215, Y offset 228: TIFFReadEncodedTile() failed.449 2020-12-12 14:19:13 | TIFF reader: Failed to retrieve data from the source dataset450 2020-12-12 14:19:13 | JPEG2000 writer: Could not perform Read/Write on file451 2020-12-12 14:19:13 | JPEG2000 writer: Failed to write dataset '\\\\mcc\\groups\\proptest\\Aerial\\Taylors_October2020_2cm_GDA2020z55.jp2'. Thi
The next screenshot is reading a corrupt geospatialPDF. Result: It gets the page points, at the well known zero-island. The extent shows the points (~pixels if you like) of the original pdf. 595x842pt I know the coordinate system. LL84. Therefore I could overrule the coordinate system. But what happens is that the reader thinks there is no coordinate system. Therefore everything is considered page-points(595x842) And these coordinates are treated as lat,lngs. Projecting this geometry all over the ("edges" of the) worldThe settings of the reader:The behavior of the first option would be in my mind.Geospatial (if possible or over ruled). Is this a known limit and is there a way to read such document anyways with the right coordinates?
After completing the setup of the Esri Geodatabase (ArcSDE Geodb) reader for Oracle, the following error occurs: "Unable to read from the table/feature class 'schema.table'. If 'schema.table' contains a class extension, make sure it is registered on this computer. It may be that 'schema.table' contains a feature type that is only available in ArcGIS Desktop or ArcGIS Server. If so, please install required ArcGIS Product, or if required product already installed, update Python Compatibility to 'Esri ArcGIS Python 2.7'. To update setting, go to: Navigator -> Workspace Parameters -> Scripting -> Python Compatibility". Please note that I changed my real database info to "schema.table" for security. I tried changing the Python version to 2.7, but I get the same error. I also tried creating the connection file in both Arc Pro 2.6.3 and Arc Catalog 10.6, but I got the same error. Any help would be greatly appreciated.
Hi, Safers! I read Text Files as Whole text at once but I need convert files not in UTF-8 to UTF-8 before reading. This is because FME needs to know the encoding when it reads the file. I figure one easy way to do this is to store the files to a temp folder and via the SystemCaller run some windows command to convert the files not already in UTF-8 (this can be more than thousands of files so preferably I can convert only the files that needs converting) before reading them with the Text File reader with Character Encoding set to UTF-8. Does anyone have any suggestions on how to solve this in an easy way? Preferably within the FME script if possible... Kind regards, Peter
I am trying to create mtext leaders and I am trying to follow the documentation which has these attributes and I have tried to create text adder at last point on leader line and then write a unique annotation handle to autocad on both leader line and text but the entities are not associated when I open in autocad.Can I not create text in a leader?
I have a contour dataset, which I want to clip into tiles. The contours and tiles are both in shapefile format, GDA2020 datum (Zone 55)Half of the output shapefiles are okay, but half have no geometry.
I'm wondering how I can make a request with SSL using the ArcGIS Online Feature Service reader. No matter what I do, FME continnue sto try to access via HTTP instead of HTTPS. I'm reading a features service hosted in ArcGIS Online into FME Workbench 2020 64 bit. Last week I was able to connect to a service using the Feature Service reader without issue. But our admin team converted the ESRI organization to require HTTPS only, which seems to coincide with the 403 error I'm receiving. I've remade the web connections, tried to reconfigure the feature service URL to use HTTPS, with no luck. I receive this error This may be a temporary server issue, or insufficient permissions. The error code from the server was '403' and the message was: 'SSL Required'Failed to obtain any schemas from reader 'ARCGISONLINEFEATURES' from 1 datasets. This may be due to invalid datasets or format accessibility issues due to licensing, dependencies, or module loading. S
Hi Team,I need assistance in optimizing the FME package. I am new to FME. I have a workbench with reader as OGC Geopackage and writers as csv,shp, json (same data but created different output). I have 5 geopackages . I need to update the reader everytime in order to create the output files. I cannot join or merge these files. Is there any way, after one reader reads 1.gpkg and completed the flow, it pick 2.gpkg and continue execution till last available package. Also I want to update the writer dynamically based on the input files. In SSIS, we have a for each loop container. Is there any way to loop through the files and creates output files. Currently, I am changing the readers and writer to execute one gpkg. So if I have 5 inputs, I am changing reader and writer everytime. This is manual process and high chances of human error.
Hi, I have troubles opening the attached geopackage, and I can't figure out what is wrong with it. It opens nicely in QGIS (where it displays the rasters) and with a SQLite client I'm able to access the tables. The error is : "OGCGEOPACKAGE reader: `IHC Krimpen online - Progress - GRD - Version 1_20200409 204806.gpkg' exists but could not be opened. File contents may be invalid, or the file format may not be supported"Using FME 2020.1
FME(R) 2020.2.0.0 (20201027 - Build 20787 - WIN64) Hello Whilst troubleshooting an automated workflow, I realised that FME was actually reading the Zipped file using the wrong format.It happens when I drag and drop the zip file. (I, unfortunately, cannot share the files.) I tried a different approach, which was to unzip the files then re-compress them again in case it was a "corrupt" archive, but same result. ThanksSameer
Hello FME'ers I am using FME 2013 SP2 to bulk load data into PostGIS 9.2 from Oracle 10 using a very simple workbench. I can set the attribute names to lower case in the PostGIS writer (as well as create the GIST index, sequence and primary key) but I also need to set the table names / feature type name to lowercase. I have several hundred tables to copy over and it would be nice to change case automagically rather than manually editing the workbench. Any suggestions? Thanks Ross
Asking on behalf of a client: they're writing to Oracle and getting an error regarding an UTF-8 string that can't be converted to Windows-1252. I can't seem to find a way to set the encoding on the Oracle writer, is this handled in the database definition itself (i.e. outside of FME)?
Hi, I have a problem where I use DWGstyler to assign different colors to different blocks, but when I open the file in CAD, all these blocks have the same color in red. I am new to FME and hope someone can help me with this problem. Thanks in advance!
HiI am importing Points along with their attachments from ArcGIS Online. I am using sde writer with the new option of preserving GlobalIDs. the issue is it runs smoothly on feature class but with table initially it runs fine and stores the data but at the end it gives me this error.
I am running FME Desktop 2019.1 and am trying to output the results of a Feature Merger to a csv file but I want to add an additional row in the csv file before I have the column headersieI want the output to look like this:but I can only get this:in excel it would look like this: FORMAT ASSET DETAILS , STANDARD 1.0 REGNAMEASSETNBRTMPLASSNBRDESCRLONGDESCRSHORTDESCRASSETTYPEASSETSTATFLEET1000000090000000Sj300 2019 White Ls Colorado 4 X 4 DieseSj300 2019 White Ls Colorado 4 X 4 Diesel (Ranger 4) SJ300Sj300 2019 White Ls SN Is there a way to do this?Preferred output is csv but if i its do-able to excel that will suffice.Thanks in advance
In FME, I have a File Geodatabase Reader (merged feature type). I would like FME to group features by fme_feature_type and output each feature type to a port. I have gotten close with the attribute pivoter, but haven't had any luck writing counts. Does anyone have any suggestions? What I'd Like to accomplish: Almost got it with attribute pivoter:My settings for attribute pivoter: Thank You
I try to read an IFC with IfcReinforcingBars. All attributes etc gets read in FME correctly, however there is no visible geometry. I can not share the data, but wonder if there are certain elements/geometrire FME does not read from IFC? I tried the 3 different IFC-readers, and also different settings on the reader with no luck. Examining the features in FME Data inspector shows no geometry. But it seems to be aggregates and Geometry Instances. I also tried the GeometryInstantiator to see if I could get the geometry. https://www.safe.com/transformers/geometry-instantiator/ An image of IfcReinforcingBars can be found on this page for instance: https://github.com/xBimTeam/XbimGeometry/issues/74
Hi There,As per the title, I am having trouble connecting to an ArcSDE geodatabase using a connection file. I can confirm that I am able to connect to the database in ArcMap/ArcCatalog so my feeling is that there is nothing wrong with the connection file, but possibly some other issue that I am having trouble trying to resolve.I am running FME 2018.1 32-bit and have also tested this in FME 2017.1 32-bit to no avail. I can also confirm that I also have the 32-bit Oracle Instant Client (12.2) installed and configured correctly. I am currently running Windows 7, but I have also tested on a colleagues computer (32bit FME 2018.1 and 32 bit Instant Client) who is running Windows 10, and we both receive exactly the same error.The error that I am receiving is the following:---------------------------Fetching Feature Types Error---------------------------<b>Failed to retrieve feature types.</b><p>Could not open the Enterprise Geodatabase. The error number from ArcObjects is: '
I am having issues connecting to one of our elastic databases. The error I am getting is Failed to obtain any schemas from reader 'ELASTICSEARCH' from 1 datasets. This may be due to invalid datasets or format accessibility issues due to licensing, dependencies, or module loading. Any ideas what how to fix.
I'm specifically trying to find out what versions of PostgreSQL & PostGIS(non-raster) are supported by FME "Desktop professional" version 2020.1.2. More generally it would be good to know if there is a place we can find this information for ongoing versions of various FME platforms. I have tried looking in these places... https://docs.safe.com/fme/2020.1/html/FME_Desktop_Documentation/FME_ReadersWriters/postgis/postgis.htmhttps://downloads.safe.com/fme/2020/whatsnew_2020_1_3.txthttps://downloads.safe.com/fme/2020/whatsnew_2020_0_3.txtbut could only find (in the last link) that some fixes were done to support PostgreSQL 12+