- - Community Resources
Ideas
Product Updates
Events
Blog

Home
Forums
FME Form
Data

Data

Using readers and writers to extract and load data.

6,810 Posts
25,086 Replies

When you subscribe we will email you when there is a new topic in this category

6810 Posts

Most replies

Most views

Newest first

davideagleContributor

asked in Data

Create Hyperlinks in Excel?

Does anyone have any bright ideas. We're trying to write hyperlinks into an Excel worksheet but failing to do so. The old Excel ADO writer doesn't support this (http://fmepedia.safe.com/articles/FAQ/Reading-hyperlinks-from-an-Excel-spreadsheet) but I thought the new one might with perhaps some of the format codes that are now supported in the new writer (http://office.microsoft.com/en-ca/help/number-date-and-time-format-codes-HP005207595.aspx) but it appears not. If you try to be clever and write a concatenated formula string into Excel with either writer like this: =HYPERLINK("http://www.1spatial.com/fme","http://www.1spatial.com/fme") ...FME just writes it explicitly as above... we've even tried adding a return line in the concatenator to no avail. Ironically if you open the results of this and click into the cell and hit return it enables the hyperlink, so it's not a formatting issue either. So, I guess my plea is, has anyone done this or does

How to read XML with GML geometry inside

Hello,I have a XML file with a GML geometry inside (in attach). Moreover I have the XSD file (named: CensusTechSheet).Can I read that file with FME? I suppose: yes. But someone can suggest me how can I do this?Regards Gianpiero

Using variable in a where clause

Just getting to grips with FME, what I'm trying to do is use the where clause in reader to limit the reader to only return records after the last OID of the previous process. Probably simple but I'm struggling a bit

1940

4 years ago

brendannealParticipant

asked in Data

Missing data when writing to FGDB with dynamic schema

Hi All, I hope someone can help me out there. I am writing to a FGDB with dynamic schema, using the "fme_feature_type" to create the feature classes. I am missing 1 feature in every feature class that is written. If there is only 1 "fme_feature_type" record the feature class does not get created at all. Writer parameters image is below, I am taking the Geometry type from the first feature it finds, not sure if this is the issue? but I have different geometry types for 'fme_feature_types", so see it as the only option. Does it use the first feature to define the feature class only? and not write the feature out? Can't see why that should be the case. Image below of; Statistic Summary just before it is written to FDGB of "fme_feature_type", Examples below"- AU1_ADMARE_p has 9 features- AU1_BCNCAR_p has 1 feature- AU1_BCNISD_p has 1 feature- AU1_BCNSPP_p has 22 featuresAfter FGDB is written, image below:- AU1_ADMARE_a has 8 features- AU1_BCN

Excel: Best Way to find the first blank row, then write new data to it / Using an attribute as start row

Hi,I am looking for the best way in FME to find the first blank row in an Excel Spreadsheet. I have a way, but there are likely better ways. For now I am using the AttributeFilter to give me all of the rows that have data and then using a counter to give me the first blank row. If you know of a better way I would like to streamline this:It spits out an attribute named RowToWriteTo and it gives me row 127 which is correct. However, I am not sure how to use this in the writer. I am needing it to write one new row of data to the first blank row in that same spreadsheet and then save it. In this case, row 127. Next time the workspace would be run it would be row 128 etc. etc. So the row number is dynamic each time the workspace it run. My problem has been keeping the existing data and then just adding that one row to it. Perhaps that is a writer issue. Not sure? Any ideas would be helpful!I would think the correct way is to set the Writer Mode to Update? Or should it be Insert?

IFC Readers produce an error ISO 10303-21

Hi folks,Each time I tried to read an IFC file in IFC reader I got an error states that “ISO 10303-21: the input data file has a syntax error in instance ‘’#879 ” I tried several IFC files but I got the same error, how can I fix this? Cheers,Rawan

Insert and update behavior when read information from a spreadsheet to write in a database

I did a POC where I needed to: 1. Read information from an Excel spreadsheet and the insert this fields in certain tables in my postgres database tables. This went fine once. But, when I repeat the process the same "n" numer of rows are inserted again. This is wrong since no change has occured. 2. After the first insert I need to update only the rows or fields that changed in that period. As I stated above, whenever I run the flow again and again more rows are inserted (indeed, the same rows are inserted). Is there a way to arrange my model in a fashion that this requirement may be implemented? I mean. Insert once and update incrementally as needed? If so, how? Is there any documentation about it? Regards!

170

4 years ago

franco69Contributor

asked in Data

AUTOCAD WRITER Unused Objects

Hi there Guys, i had a topic about few days ago..... i have an autocad file where i want to write not all layers from the template (only the layers which comes with an feature type in the reader).....tha i solved with help from you with the parameter: PURGE UNUSED OBJECTSSo now i have following problem with one layer:layer 0 has a block included which i have to explode and also use block feature type for components i have set on YESthis block has include some layers i dont want to write (because of purge unused.....) but they be written. (maybe because they are originally included in a layer which has a feature type?)how can i explode the block, take the original layer definitions with me and filter the not used layers (parameter?)I've tried also the REAL DWG WriterThanx and GreetzFranco

4 years ago

disco2000Contributor

asked in Data

Colour conversion from PDF to tiff

Hi,I have a PDF-map (image without marginalia) generated from ArcGIS Pro that I want to convert to a GeoTIFF (8-bit palette) and use as raster data. The colours look fine in the PDF but somehow come out alot darker when i convert them to GeoTIFF. This happens if I export as an RGB as well... Is there a colour handling property that I have overlooked or is this a common issue between ESRI and FME in their PDF reader/writer? Source PDFOutput GeoTIFF

940

4 years ago

dhavanv2Contributor

asked in Data

NetCDF writer - would like to output one file with 100+ bands and multiple variables, but as compact as possible

Hello, I am new with the NetCDF format so apologies if there is something obvious I am missing.I have a bunch of GeoTIFFs which I am reading in and extracting some variable information from. Then I use a RasterBandCombiner to go from about 150 raster features to 1 with each of it now a band. When I write this out as a NetCDF file, it is about 85 MB. All the Geotiffs combined were about 220 MB.A colleague of mine has tried this using Python (gdal, netcdf4), and is able to translate all the data into the netcdf file that is only 3 MB.It would be cool to have this workflow set up in FME for subsequent use. So I was wondering if there are any parameters or transformers in FME that might help with this? Is it possible to only write out the non-spatial data? I tried using a GeometryRemover, but the writer gave me a fatal error which said it requires raster geometry. Is this why the file output from FME is so much larger in size - the geometry?I also tried bringing in the 3MB file in to FME a

Support for GML 3.3 LRS?

GML version 3.3 adds support for Linear Referencing Systems (LRS).Apparently FME only supports GML up to version 3.2.1:https://knowledge.safe.com/articles/888/reading-xmlgml.htmlI've read elsewhere that "GML 3.3 is itself a GML 3.2.1 application schema".Does that mean FME can read and/write LRS data in GML 3.3? If so, how?Thanks.

I have a SDE Geodatabase with over 1000 featureclasses. I am wanting to select 30 of these and copy over to a Geodatabase. I am selecting these featureclasses by there name. What transformer do I need ? is it Tester ?

4 years ago

yitin

asked in Data

WFS write to PostGIS result failed to open in ArcGIS. Error: The field is not nullable

Hi,I'm a bit new to FME so pardon me if the question is basic: I use FME Workbench 2018.1. I have a PostGIS table with 1035 polygon features which needs to be updated (Table A). The update source is a WFS layer with 27 polygon features, among which 26 are updated features and 1 new feature to be added to Table A.The workflow is as follows:1. Since the WFS has a different schema to Table A, I use AttributeManager to make the WFS attribute table consistent with Table A. 2. I use Tester to filter out the 1009 features that don't need to be updated.3. I write the filtered 1009 features and the 27 updated/new features from WFS to Table A, hoping to get an updated layer with 1036 features. I have the following feature type settings:-Table A has a column 'gdb_geomattr_data' which seems to store geometry, so I set the Spatial Column of the writer as 'gdb_geomattr_data'-Table definition set to 'Automatic'-Create Generic Spatial Columns: No-Bulk Insert: No-Table n

FME to Socrata job not working with Socrata Covid Map

EDIT: After doing steps below I now am able to update the LOCATION field but no new records are being being added when I run the FME job. Any ideas?I'm attempting to automate reading an Oracle database and writing to a Socrata dataset using FME workbench 2018. Everything was working except that every time I ran the job I had to manually set the lat/long columns in Socrata to get the map working.I googled the issue and found this documentation: https://support.socrata.com/hc/en-us/articles/207040527-Working-with-Point-Data-Using-the-FME-Socrata-WriterThen I did the following:I used attribute creator to create a new field called LOCATION that concatenates the latitude and longitude columns so that the values look like “POINT (-87.61694 41.70737)”. This location field is already setup in Socrata as a point geocoded column.I have attached a screenshot of my workbench and the new Location field below.When I run the workbench it gives me an error, and although the LOCATION field is popul

Problem with conversion of CityGML to json

I just would like to convert CityGML files to json files. When checking the converted json file in the FME Data Inspector, I get the errors shown below. Maybe I missed some parameters during reading/writing. Thank you for your help!These errors are shown when checking the json file:CITYJSON reader: An error occurred. FME will attempt to provide more information on the error, but this may cause the translation to be terminatedRead Metadata Feature Command caught exception: stk::ex::FatalError[0] CITYJSON reader: [json.exception.parse_error.101] parse error at line 1, column 2: syntax error while parsing value - unexpected end of input; expected '[', '{', or a literal[1] auto __cdecl STFReader::openDataSet::<lambda_45ec89e6ecf0932b901acf3bedceb848>::operator ()(void) const(C:\\code\\fme\\foundation\\framework\\engine\\reader.cpp:80) My initial CityGML file:

Auto-name Excel files

Hello,I have the following situationI made a drop_down list (User Parameters) - Choise, from which to choose a specific locality.depending on the name of the locality, I generate an excel file.I want that when I create the excel file, this file should bear the name of the locality.Ex: BACAU => BACAU.xlsx AGAS => AGAS.xslx

100

4 years ago

arnovananrooijContributor

asked in Data

PostGIS FeatureWriter adds extra rows

Hi,I am sending 2600 features to a PostGIS FeatureWriter. When I look in the resulting table (which was completely empty) I see 2628 rows. How is this possible?I noticed it because first I had a primary key defined and I was sure there were no duplicates in the 2600 features, but I got an primart key violation error After removing the primary key I see several duplicates.Any ideas on how to solve this? Or did I find a bug?

Point cloud to Las fails with checksum mismatch error.

Running a simple script of point cloud to Las in FME 2020 desktop. My input is five .e57 files. The script fails after writing the first Las file with the error message: E57 reader: E57 reader: An error occurred while reading. The error was 'checksum mismatch, file is corrupted (E57_ERROR_BAD_CHECKSUM)' with context 'fileName=C:\\1-Working\\GDB\\E57\\1-01-C-01_200219-0.e57 computedChecksum=10167370 storedChecksum=2581601340 page=2465744 length=4044457984' (error code = 16)FFS writer: Failed to get block of 10000 points starting at index 167600000Stored 1 feature(s) to FME feature store file `C:\\Users\\Employee\\AppData\\Local\\Temp\\wb-cache-e572las-IwHIxO\\Main_point_cloud -1 2 -1 0 5593c33571a1202057291a33f5c5400df0f2e01f.ffsupdating'Saving spatial index into file 'C:\\Users\\Employee\\AppData\\Local\\Temp\\wb-cache-e572las-IwHIxO\\Main_point_cloud -1 2 -1 0 5593c33571a1202057291a33f5c5400df0f2e01f.fsi'Finished saving spatial index into file 'C:\

PowerPointStyler Stacked Column Chart

How do I set up my data to generate a stacked bar chart? For each CURRENT_OPERATOR_ALIAS category, I have 7 bins (RATE_CLASS) that I'd like to chart, where one bar is a CURRENT_OPERATOR_ALIAS with corresponding counts (COUNT(API) attribute), but I can't seem to plot 2 categories in the PowerPointStyler.

Hi, I want to clip WMS layer using polygon shp and write it out as a pdf. Any ideas?

Hello, I'm reading data from an ORA database wicht contains - a surname: Çad?rc? - a name: ËCadõrcõ In the inspector I see these names correct with encoded utf-8when I write this information to an (different) ORA-database with Varchar2(800) the table contains: - a surname: Çadrc (wrong) - a name: ËCadõrcõWhen I put the writer items to AUTOMATIC it will give also the same outputWhen I change the TYPE in the writer from Varchar2 to NCHAR the table will contain: - a surname: Çad?rc (wrong) - a name: ËCadõrcõ what am I doing wrong and how to correct this problem?

1327

4 years ago

benvkContributor

asked in Data

AGOL - Populate house table with oldest resident from second table

Hey, I have two tables in the ArcGIS Online space, one contains a list of houses (property_table), and the other related table contains a list of residents that reside within (resident_table).What I am trying to do is populate the property table with the AGE of the OLDEST RESIDENT inside each house. I have no idea how to do this and am seeking assistance to build a plan of attack. I have been able to add the two readers to my workspace successfully, I can do a feature joiner which proves the residents do reside within the said houses. But I don't think its actually necessary for the join for the calculation to take place.So can someone please tell me how I would find the OLDEST resident, and then insert that date of birth, and name, into the two fields in the property table?I have been staring at this for too long and my brain is not able to straighten out the issue in front of me, it should be rather straightforward :( Property Table:Resident Table:

4 years ago

fdwContributor

asked in Data

PDF writer: Document properties

Hi, We are daily updating a number of PDF's with FME for publication on our website but these files need to have the Document Properties set. Our webcare team won't allow for the pdf's to be published. So far I'm not able to achieve this.There seems to be an option with the format attribute 'pdf_document_info_metadata' but how do I format the properties in my writer?When reading a pdf which has these properties filled in, pdf_document_info_metadata is a feature type. I have tried adding this feature type to my pdf writer with the exact same schema. But no result; empty properties still.Does anyone know if writing the properties is supported and how to achieve it?

768

4 years ago

fmelizardContributor

asked in Data

Can I add a new layer to an existing Feature service on ArcGIS Online or ArcGIS Portal?

How to publish Covid19 data to socrata dataset without deleting columns

I created a workbench that reads oracle database view and writes to socrata dataset. I was trying different settings in the writer and now my socrata dataset is missing 4 columns. Is there anyway to create those columns while writing the data? I'm using FME 2018.

4 years ago

Page 74 / 273

Badge winners

kokilamhas earned the badge Integrate Spatial Data
ryanwillshearhas earned the badge FME Form Basic Training
barkushas earned the badge Welcome Back
anteboy65has earned the badge Welcome Back
tamadurhas earned the badge Welcome Back

Show all badges

30,366: Posts

Terms & Conditions Cookie settings

Sign up

Already have an account? Login

An FME Account is required to contribute

Login to the community

An FME Account is required to contribute

Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.

Enter your e-mail address

Back to overview

Scanning file for viruses.

Sorry, we're still checking this file's contents to make sure it's safe to download. Please try again in a few minutes.

This file cannot be downloaded

Sorry, our virus scanner detected that this file isn't safe to download.

A product of

Company Products Giving Back Careers Contact Privacy Feedback

Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.