- - Community Resources
Ideas
Product Updates
Events
Blog

Home
Forums
FME Form
Data

Data

Using readers and writers to extract and load data.

6,779 Posts
24,930 Replies

When you subscribe we will email you when there is a new topic in this category

6779 Posts

Recently active

Most replies Most views Newest first

jgarciafdez111

asked in Data

Dropbox issue when retrieving data

Hello! I am having some issues when working with Dropbox both as data input and output. In the case of data input, there is one error that happens sometimes: a retrieval error. At least two different description of this error appears in the log>1. Retrieval error: Unable to create resource with duplicate name on web service2. Retrieval error: "web-service" reporting server-side error. This service may experience issues. (upstream connect error or disconect/reset before headers. reseat reason: local resetI am enclosing printscreens for both cases. Do you have any idea what's the cause(s) and how to overcome this?Thank you very much, Jorge

3 years ago

garydlester

asked in Data

Is there a reason that the Autocad Writer will corrupting a dwg?

I have a workspace (attached to question) that writes leaders to a dwg template. As far as I can tell the template has no errors, but the resulting dwg has 2 errors and must be recovered to be opened. Has any one else encountered a problem like this? I have attached the workspace with the behavior described.

3 years ago

divya

asked in Data

Unable to see FME generated cad file with the attributes in AutoCAD / Revit

I am trying to convert the ESRI format data(feature class) to cad using FME(I have used Extended Entity Data option). When I open the cad output in FME inspector I can see the attributes but when I try to open in AutoCAD or Revit I cannot see attributes attached to the features. Just wanted to know how can I get to see attributes in cad file in AutoCAD

224

3 years ago

aaronContributor

asked in Data

How read Excel data with dynamic schema (sheet name and attributes)?

I am reading an Excel file online where the sheet name and attribute names sometimes change. I therefore need to read the sheet and attribute names dynamically. I can get the schema but I haven't figured out how to read in the data using a dynamic schema. ("Scan Schema in Dynamic Workflow" doesn't appear to work for the Excel Reader.) I figured out how to use a FeatureReader to get the sheet name dynamically using fme_feature_type_name and then pass it to a second Feature Reader but I haven't figured out how to get the attribute names dynamically and then pass them along to the second FeatureReader so I can read in the data. Or is there another way?

Does writing to a MapInfo Extended TAB format take longer than to a MapInfo TAB ?

Does writing to a MapInfo Extended TAB format take longer than to a MapInfo TAB ? Just wanted to check with the community if this is a known/expected behaviour with writing data to MapInfo Extended TAB and also if there are workarounds to quicken the writing process.Although I have experienced FME 2017 take considerably longer time to write a MapInfo Extended TAB for polygon data that is about 1.2 million in count, I failed to take note of the actual time it took, from the log. I do have a stat for a much smaller dataset, that of 1500 polygons and it took about 14 minutes to write to a MapInfo Extended TAB

195

3 years ago

oscardInfluencer

asked in Data

File Copy - Copy empty Folders

Good evening.I need to copy all the content from a folder to another one. That means files and subfolders.I have been able to copy any file or subfolder with files, but if the subfolder is empty, it is not copied. I've been playing around with the filecopy_dest_dataset attribute, but no luck. I bet is something obvious, but I'm stuck. I'm using FME 2019.0.2 (64-bits)Thanks for any help provided!

How to convert DGN with reference files to DWG

Greetings, I have a microstation DGN file that I'm trying to convert to a dwg or dxf and my output is missing all information from the reference files (see attached images). I tried fiddling with the reader to include reference files but it has not made a difference in my output yet. I even tried to include a DWGStyler that references a dwg template but still no change in the output. Thoughts?

1858

3 years ago

peteralstorpContributor

asked in Data

How connect to styles in MS Word?

I have a base word document I want to use and set so I use styles from this base file in the MSWordStyler. But I get error msg when I try to use for instance the style "Tyréns - Brödtext". Tips? Tricks? Partytricks? The styles as viewed in Word.Kind regards, Peter

363

3 years ago

lazarlubomirContributor

asked in Data

Sample of IFC writing

Hello everyone, now I´m working on writing the data to IFC format via FME workbench. It´s quite tough to handle it.... Is there anybody who have small sample of writing IFC via workbench? Thank You so much!Lubo

Unable to see shared drives using Google Drive Connector

Hello.I am trying to use the Google Drive Connector to download files from a Shared Drive. But when I click on the Google Drive Source in the transformer all I see is the folders and files which are in My Drive. It doesn't show shared drives which I have created as well as drives someone has shared with me.I'm just wondering if this is a bug and if anyone else has the same issue?Many thanks.

222

3 years ago

tinoContributor

asked in Data

FME Server Generic Writer and MIME Type

Has anyone managed to construct a workspace with the following requirements:Only to be used in Datastreaming serviceGeneric writer with different file based outputs (demo pdf+png)FME Server streams the correct mime type with the response fileMy current experiment was to use a private python parameter which generates the correct mime type string depending on the format. But even with this, the data streaming sends a PDF but the response headers contain "Content-Type ?image/png".Is this possible at all?

FME Server on docker cannot connect to MongoDB

I recently installed FME Server to docker using the installation guide. Since yesterday I have been trying to test it using a simple .fmw in order to write some data to a MongoDB instance. I tried in both local MongoDB and in a MongoDB-docker instance. In both cases I am getting the same error: MongoDB writer: Cannot connect: couldn't connect to server <<PORT NUMBER>>, connection attempt failed.Please note that, I have access to both MongoDB instances from local FME Workbench and I can execute the same .fmw. Find attached the file. This seems to be an FME docker network issue, but I cannot be sure what. Any ideas?

3 years ago

smolContributor

asked in Data

FMEs FTP server does not allow to upload file

I have been trying to report an issue related to a backup file which is bigger as the 5MB. Following https://community.safe.com/s/article/upload-a-file-to-the-safe-software-support-ftp I have tried to upload the respective file through the given `curl` command. But whatever I try, I keep getting "Failed FTP upload: 553". I tried both zipped and unzipped, where the zipped file is around 6GB. Is there anything I can do to overcome this and continue with problem report?

426

3 years ago

simonhumeParticipant

asked in Data

extracting data from a JSON file

HiI've been given a JSON file containing registrations for 5 properties. We are looking to use other datasets with a single field in the json file to identify the location of the property. I am looking to produce a csv file initially with each record on a separate line Can someone comfirm if the following is correct or not please. Open the JSON file using the JSONFeature Reader Connected a JSONFragmenter, using one of the fields as the JSON Query. I've attached a screenshot of the settings for the JSONFragmenter. When i run the workspace and view the result in the Inspector,i can see the records listed in the right hand pane, but writing out a csv file with each record on a separate line doesn't work. thanks for any assistance

213

3 years ago

rudy_vContributor

asked in Data

Error Building Mosaic from TIFF files

Hi, I am trying to build a mosaic from 400GB of TIFF files - but is continuous failDiskspace Available 4TB - FME Temp file 1TBHas anyone come across this error - can it that the TIFF is corrupted? 445 2020-12-12 14:18:51 | JPEG2000 writer: Completed 21.76% of intermediate processing446 2020-12-12 14:19:13 | TIFF reader: LZWDecode:Wrong length of decoded string: data probably corrupted at scanline 27520447 2020-12-12 14:19:13 | TIFF reader: TIFFReadEncodedTile() failed.448 2020-12-12 14:19:13 | TIFF reader: \\\\mcc\\gis\\SourceImagery\\Taylors\\2020-10-30_Deliverable\\OrthoImage\\B4.tif, band 1: IReadBlock failed at X offset 215, Y offset 228: TIFFReadEncodedTile() failed.449 2020-12-12 14:19:13 | TIFF reader: Failed to retrieve data from the source dataset450 2020-12-12 14:19:13 | JPEG2000 writer: Could not perform Read/Write on file451 2020-12-12 14:19:13 | JPEG2000 writer: Failed to write dataset '\\\\mcc\\groups\\proptest\\Aerial\\Taylors_October2020_2cm_GDA2020z55.jp2'. Thi

731

3 years ago

meijerContributor

asked in Data

How to overrule GeoPDF coordinatesytems

The next screenshot is reading a corrupt geospatialPDF. Result: It gets the page points, at the well known zero-island. The extent shows the points (~pixels if you like) of the original pdf. 595x842pt I know the coordinate system. LL84. Therefore I could overrule the coordinate system. But what happens is that the reader thinks there is no coordinate system. Therefore everything is considered page-points(595x842) And these coordinates are treated as lat,lngs. Projecting this geometry all over the ("edges" of the) worldThe settings of the reader:The behavior of the first option would be in my mind.Geospatial (if possible or over ruled). Is this a known limit and is there a way to read such document anyways with the right coordinates?

3 years ago

phoolish_one

asked in Data

Hello All, I'm trying to add a Esri Geodatabase (ArcSDE Geodb) reader to my workspace using FME Desktop 2020.2, but I'm getting an error. I put the details in the details area.

After completing the setup of the Esri Geodatabase (ArcSDE Geodb) reader for Oracle, the following error occurs: "Unable to read from the table/feature class 'schema.table'. If 'schema.table' contains a class extension, make sure it is registered on this computer. It may be that 'schema.table' contains a feature type that is only available in ArcGIS Desktop or ArcGIS Server. If so, please install required ArcGIS Product, or if required product already installed, update Python Compatibility to 'Esri ArcGIS Python 2.7'. To update setting, go to: Navigator -> Workspace Parameters -> Scripting -> Python Compatibility". Please note that I changed my real database info to "schema.table" for security. I tried changing the Python version to 2.7, but I get the same error. I also tried creating the connection file in both Arc Pro 2.6.3 and Arc Catalog 10.6, but I got the same error. Any help would be greatly appreciated.

761

3 years ago

peteralstorpContributor

asked in Data

Converting character encoding before reading text files

Hi, Safers! I read Text Files as Whole text at once but I need convert files not in UTF-8 to UTF-8 before reading. This is because FME needs to know the encoding when it reads the file. I figure one easy way to do this is to store the files to a temp folder and via the SystemCaller run some windows command to convert the files not already in UTF-8 (this can be more than thousands of files so preferably I can convert only the files that needs converting) before reading them with the Text File reader with Character Encoding set to UTF-8. Does anyone have any suggestions on how to solve this in an easy way? Preferably within the FME script if possible... Kind regards, Peter

How can autocad_anno_handle be created and associated between mtext and leader

I am trying to create mtext leaders and I am trying to follow the documentation which has these attributes and I have tried to create text adder at last point on leader line and then write a unique annotation handle to autocad on both leader line and text but the entities are not associated when I open in autocad.Can I not create text in a leader?

379

3 years ago

nicholasContributor

asked in Data

Output shapefiles have null geometry

I have a contour dataset, which I want to clip into tiles. The contours and tiles are both in shapefile format, GDA2020 datum (Zone 55)Half of the output shapefiles are okay, but half have no geometry.

How to fix 403 'SSL Required' Error while using ESRI ArcGIS Online Feature Service Reader ?

I'm wondering how I can make a request with SSL using the ArcGIS Online Feature Service reader. No matter what I do, FME continnue sto try to access via HTTP instead of HTTPS. I'm reading a features service hosted in ArcGIS Online into FME Workbench 2020 64 bit. Last week I was able to connect to a service using the Feature Service reader without issue. But our admin team converted the ESRI organization to require HTTPS only, which seems to coincide with the 403 error I'm receiving. I've remade the web connections, tried to reconfigure the feature service URL to use HTTPS, with no luck. I receive this error This may be a temporary server issue, or insufficient permissions. The error code from the server was '403' and the message was: 'SSL Required'Failed to obtain any schemas from reader 'ARCGISONLINEFEATURES' from 1 datasets. This may be due to invalid datasets or format accessibility issues due to licensing, dependencies, or module loading. S

Recurring data load with Different files

Hi Team,I need assistance in optimizing the FME package. I am new to FME. I have a workbench with reader as OGC Geopackage and writers as csv,shp, json (same data but created different output). I have 5 geopackages . I need to update the reader everytime in order to create the output files. I cannot join or merge these files. Is there any way, after one reader reads 1.gpkg and completed the flow, it pick 2.gpkg and continue execution till last available package. Also I want to update the writer dynamically based on the input files. In SSIS, we have a for each loop container. Is there any way to loop through the files and creates output files. Currently, I am changing the readers and writer to execute one gpkg. So if I have 5 inputs, I am changing reader and writer everytime. This is manual process and high chances of human error.

362

3 years ago

r_hueskenContributor

asked in Data

Error reading geopackage

Hi, I have troubles opening the attached geopackage, and I can't figure out what is wrong with it. It opens nicely in QGIS (where it displays the rasters) and with a SQLite client I'm able to access the tables. The error is : "OGCGEOPACKAGE reader: `IHC Krimpen online - Progress - GRD - Version 1_20200409 204806.gpkg' exists but could not be opened. File contents may be invalid, or the file format may not be supported"Using FME 2020.1

1703

3 years ago

sameerContributor

asked in Data

Reader detects ESRI Zipped files as EDIGeo format

FME(R) 2020.2.0.0 (20201027 - Build 20787 - WIN64) Hello Whilst troubleshooting an automated workflow, I realised that FME was actually reading the Zipped file using the wrong format.It happens when I drag and drop the zip file. (I, unfortunately, cannot share the files.) I tried a different approach, which was to unzip the files then re-compress them again in case it was a "corrupt" archive, but same result. ThanksSameer

3 years ago

mixedbredie

asked in Data

Lowercase PostGIS table names

Hello FME'ers I am using FME 2013 SP2 to bulk load data into PostGIS 9.2 from Oracle 10 using a very simple workbench. I can set the attribute names to lower case in the PostGIS writer (as well as create the GIST index, sequence and primary key) but I also need to set the table names / feature type name to lowercase. I have several hundred tables to copy over and it would be nice to change case automagically rather than manually editing the workbench. Any suggestions? Thanks Ross

9314

3 years ago

Page 56 / 272

Badge winners

lachlan.thas earned the badge Welcome Back
alanbchas earned the badge Student (Grey)
alanbchas earned the badge Teacher (Grey)
digit42has earned the badge Expert (Grey)
ccantuhas earned the badge Welcome Back

Show all badges

30,121: Posts

Terms & Conditions Cookie settings

Sign up

Already have an account? Login

An FME Account is required to contribute

Login to the community

An FME Account is required to contribute

Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.

Enter your e-mail address

Back to overview

Scanning file for viruses.

Sorry, we're still checking this file's contents to make sure it's safe to download. Please try again in a few minutes.

This file cannot be downloaded

Sorry, our virus scanner detected that this file isn't safe to download.

A product of

Company Products Giving Back Careers Contact Privacy Feedback

Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.