- - Community Resources
Ideas
Product Updates
Events
Blog

Home
Forums
FME Form
Data

Data

Using readers and writers to extract and load data.

6,811 Posts
25,088 Replies

When you subscribe we will email you when there is a new topic in this category

6811 Posts

How to publish Covid19 data to socrata dataset without deleting columns

I created a workbench that reads oracle database view and writes to socrata dataset. I was trying different settings in the writer and now my socrata dataset is missing 4 columns. Is there anyway to create those columns while writing the data? I'm using FME 2018.

4 years ago

mr_fmeEnthusiast

asked in Data

Fill cell in Excel file based in previous information

Hi all,I have an excel file, where I need to fill in fields based on previous values. This should happen for 3 fields: total_suspects, total_positive, total_recovered, deaths. But some cells are not filled, they are empty. I need to fill in the previous amount, according to the previous date. Example: total_suspects for day 2020-04-06 3:00 is 19, but for day 2020-04-07, it is empty, so it should be 19. This same situation should continue in all empty fields in the sequence of days. This should happen for the 3 columns informed at the beginning. The problem is that this must happen grouped by name. However, each name set has the same date and different values.Look below, my situation now: Expectation: My file below:I have one sample workspace, with one sorter trasformer and one attribute creator with configuration below: My workspace run very well, but for one location (name) only. My file has many lines, and many different locations (names), and in my output file nothing happens. can

4 years ago

nedwatermanContributor

asked in Data

Extract XML from URL

Hi,I'm using a HTTPCaller to connect to an XML api service. Although I can connect and receive XML I can't create attributes from the file. I'm new to XML and don't know how to do this. I tried using an XML reader and successfully extracted the attributes, but then FME 2019.2 keeps crashing after reading the file, or reads the file and then doesn't show the reader in the workbench, so I'm hoping this will be a more robust method. I've tried using an XMLFragmenter to get the attributes, but they aren't showing - I assuming I'm missing a step somewhere. testxml.txt I've attached a test file.Thanks N

Write/Update Metadata using XMLUpdater

Hi there! I am new into the FME world and I am kinda hoping somebody can point me into the right direction. Currently I am trying to build an FME routine that will allow me to read metadata from a excel sheet (preferred format from the engineer department) and write it into a geodatabase feature class.This is how my workflow currently looks: Everything seems to work fine when it comes to updating the metadata fields (Title, Summary, Description, etc.) but when it comes to writing new values (writing in a blank field) the routine runs successfully but no values are being added to the feature class metadata. This is how my XML Updater looks: In this particular instance the feature class has a title so the FME routine is updating to the new title located in the Excel spreadsheet. That being said the Summary and the Credits metadata fields are empty in the original feature class and remain empty after the process is completed. I already tried all different Update

How to use FME as a tool to translate Geodata into minecraft?

Hi there, I am a new user to FME and therefore not so familiar with the interface and functionalities of FME desktop. I downloaded the software as I read in some articles that you could translate Cadastral property data and building height data into Minecraft with the help of FME. I am currently working on a project in Minecraft where the aims are to build the whole earth with human structure into the game. Even though the project has 100,000 users already participating, it is simply impossible to recreate everything by hand. If it is possible to load entire cities into Minecraft, it would help a lot. I would like to know which data I need, how I should utilize it in FME, and how the conversion process works to recreate the shells of buildings in Minecraft. Thanks in advance!

Best way to write to specific CAD layers using Generic Writer

I am in the process of converting one of our large workspaces that looked at the saved extents of an ArcGIS Map (.MXD) file and using a CAD seed file, wrote out an AutoCAD .DWG file. The old/original workspace wrote data out to specific layers in that CAD file. It also applied labeling, blocks w/rotations, etc... The end result was a decent looking CAD .dwg that our designers could use as a starting point for a new design project.My goal is to transition this to an FME Server App and make it a multi-purpose process that would use the Generic Writer and would allow users to select an extent within the FME Server App map and export to the same type of a CAD file, but also if needed, export out a KML, or other formats.What's the best way to accomplish this? (Writing features out to specific layers withing the CAD .dwg)

Geocoder attributes no longer available

After updating to FME 2019.1, I've found that the GoogleGeocoder transformer no longer works. I've been attempting to use the Geocoder transformer, but it seems that a lot of the attributes that were returned with the GoogleGeocoder are no longer returned with the Geocoder. This is causing issues with existing workbenches that rely on those attributes.Has anyone else encountered this? Is there a workaround for the Geocoder transformer to have it return all of the original fields, or has anyone been able to fix the GoogleGeocoder to work with FME 2019.1? It was suggested to use regrex to parse the individual fields, however since addresses aren't standardized it wouldn't be feasible to do so.

pass java parameters without env variable

Hello!I found a way to configure a JDBC connection using env variable but unfortunately, it breaks another software configuration I have on the same Windows machine. My goal is to set the following configuration: JAVA_TOOL_OPTIONS=-Djava.security.auth.login.config=...I felt like it was possible to directly configure the attached JRE (like in jre\\lib\\security\\java.security file) but it doesn't seam to work. Since I am running the fmeworkbench.exe, how can I pass the configuration to the program? Is there a config file which is read (seams to be the case for the server)?Many thanks for helping and wish you a great day!Christophe

4 years ago

aquamarine

asked in Data

How do I set the Feature Type as a published parameter in the FeatureReader with a Microstation DGN file

HiThis is a problem which has bugged me for years. I cannot work out how to set the feature type as a published parameter in a FeatureReader transformer that is reading in a Microstation DGN file. I have a number of different combinations of the following settings and none seem to work. All I get is one record coming out of the Initiator Port and the DGN file data does not come out of any of the ports. Can anyone help me with this?I am on FME Desktop 2019.1.Thanks!

4 years ago

adbailey

asked in Data

I am having a problem reading an ArcGIS Server feature service using a FeatureReader

This service used to read just fine, but now throws the following error: "Failed to obtain any schemas from reader 'ARCGISFEATURES' from 1 datasets. This may be due to invalid datasets or format accessibility issues due to licensing, dependencies, or module loading. See logfile for more informationFailed to read schema features from dataset 'https://apps.fs.usda.gov/arcx/rest/services/EDW/EDW_FireOccurrence_01/MapServer' using the 'ARCGISFEATURES' readerThe dataset 'https://apps.fs.usda.gov/arcx/rest/services/EDW/EDW_FireOccurrence_01/MapServer' was closed successfully" The strange thing is, this service reads just fine with a regular Reader, but I need to use a FeatureReader so that I can read in a bounding box through the initiator. It looks like the error occurs when setting the "Feature Types to Read" parameter.The service is public, so hopefully you can take a look and see what's happening. I am experiencing the issue in FME

How can I merge identical sheets in excel to 1 table with diffent colomns?

Hello,I have a excel with al lot of sheets per region (500 different names). All have the same form.On each sheet in different rows, I have measurements for each day. (Timestamp : 365 d)The name is in the sheet (yellow) and also in the sheetname (tab).Now I want a table where for each day, I have a new colomn per region. Example for 3 regions is attached, also how the result should look like.Thanks for any tip!

4 years ago

divyeshrana251

asked in Data

How to convert Cityengine shape file into citygml using FME, Also i want to convert into LOD2 model with roof texture, terrain and wall surfaces. Please do needful.

Trouble reading Excel files into my workbench

Hello, I am struggling with reading Excel files into my workbench. I built my workbench based on an Excel file with this format. When adding the reader I set the 'field names row' at 3, so the names in row 3 are used as column headers in FME. I want to use my workbench for other files as well. These files have the same format. Only the information from row 4 will be different. So I created two user parameters witin my workbench:- Input File- Output Directory If I open my workbench and run the translation I can fill in the user parameters: When I choose the original file which I used to create the workbench, all goes well. But when I choose another file, I only get missing values:I even made a copy of the original file, so it is exactly the same document with only a different file name. But even then I get missing values. Can anyone explain to me what I need to change in my settings to overcome this?Has the red indication in the User Parameters window something to do with it?

4 years ago

franco69Contributor

asked in Data

AUTOCAD Format Attributes or Transformer?

Hi Guys I don't know if i should open a new task but i do so now. I have 2 short answers Reader: dwgmy output dwg should be rotated (another north direction) and scaled to 1:1000are there format attributes for that or have i handle this with Transformer (Scaler,....) thx and GreetzFranco

4 years ago

louielou1234Participant

asked in Data

Max Features to Read in name only?

Okay newbie here (sorry in advance)--so I plopped an MSSQL reader into my workspace and set the max count for each feature paramater to read to 10. And indeed, the cache count stops at 10. However, in the log I can see that it is still going to town on 4 million spatial records in the SQL DB and the bucket keeps filling up even after it reaches the max 10 count. In other words, setting the max features to 10, 100, or 1,000,000 takes exactly the same amount of time (and resources) for the workspace to complete its processing. What am I doing wrong here? THANKS!

4 years ago

bogus

asked in Data

Find shapefiles in a folder, merge together and write output

Hi there,I am fairly new to FME so please go easy!I'm looking to do the following task, broken down into the following steps:Search through a folder to locate shapefiles (all line primitives)Set some criteria to filter which shapefiles I want to keepMerge & Dissolve the shapefilesWrite the output to a geodatabaseFor Step 1 I believe I need to use the 'Directory and File Pathnames' reader, with a *.shp path filter.For Step 4 I understand I need an Esri Geodatabase writer. For the rest, I am a bit lost...How do I then inspect what FME has found in Step 1?What are some useful transformers I could use at each step?

How can I calculate upstream watershed area?

Dear all, I am currently working on a project where I assess watershed areas upstream of several ditches. The area is a polder area (no significant internal differences in elevation) with flow direction somehow incorporated into the ESRI shapefile (see arrows in below figure), no clue how. In the end, all ditches lead to the red star. I have aready calculated the watershed area for each individual ditch. The next step would be to somehow add a column in my shapefile where all upstream watershed area's are added together to find the 'total watershed area' that provides the water flowing through the ditches.Some things to note: - Some ditches split into 2 other ditches. These, we assume, both get 50% of the water that flows from the upstream 'parent' ditch - Dark blue lines and lighter blue lines are major- and minor channels/ditches, and can be regarded as similar in function - Arrows show the direction of flow in all ditches - Since the area is a flat polder, elevat

1000

4 years ago

nedwatermanContributor

asked in Data

Extract json from json string

I'm trying to extract attributes from the attached json string using a jsonextractor. I thought the jsonquery field would be json["datetime"] but nothing is being returned. Any help is gratefully received. ThanksN

dwg coordinate system not being recognized 2019.2

when I view the drawing in map3d the cs is listed. When i view the dwg with arccatalog i can see the same CS. however when i add it to workbench it is not being recognized.I am trying to set up a dynamic workbench to re project coordinates dependent upon the cs of the reader being used. Is there a way to get the autocad cs and set that to a published parameter?

Issue with FeatureWriter (CSV)

Hi all, I'm working with FME Desktop 2019.2 and I have encountered a problem with the FeatureWriter. I need to save features in a CSV file and I want to use FeatureWriter settings to manage escape char and to qualify field values only if needed. Below the settings used.The problem is that FeatureWriter doesn't qualify a string value like the following:- Lorem ipsum dolor "sit amet"While it works correctly with a string like the following:- "Lorem ipsum" dolor sit ametThe CSV file created by FeatureWriter using the two string above is:"Test_2","Test_1""""Lorem ipsum"" dolor sit amet",Lorem ipsum dolor "sit amet"

Empty value converting JSON to Arcgis Online Feature Service

Hi All. I am working on fme workbench to convert json data from api into Arcgis online feature service. Actually i am quite new on this software.reader/ data link: https://covid19-public.digitalservice.id/api/v1/wilayah/jabar?level=kelurahanHowever, when i run, and export it as ArcgisOnline feature service the data on the arcgis online is empty its actually consist more than 5000features.feature service link: https://services6.arcgis.com/eV2M5wbf5ix69NmC/arcgis/rest/services/datajson22/FeatureServerand the one that makes me more confuse is when i export the data into file geodatabase format, i could see the all data. here the image: overall workflow on fme workbench: attached:fme workbench could anyone share what is the main problem from this or how can i solve this.Thankyou in advance

172

4 years ago

djmcdermottContributor

asked in Data

Add row(s) above header in Excel

Hi,I would like to add a new row containing some descriptive information from a seperate stream above header row in an excel spreadsheet.ThanksDavid McDermott

CSV file does not show up in the folder

Hi there, I am writing the output of FME translation in csv file using csv writer. However, I noticed that file does not show up in the selected folder. I tried to change the configuration of the CSV writer. But no luck. Couldn't anyone tell me where am I making the mistake? The attached file shows the parameter of CSV writer.

550

4 years ago

aaronContributor

asked in Data

ArcGIS Server Feature Service Reader Timeout

I'm using an ArcGIS Server Feature Service Reader to read in flood hazard polygons but it keeps timing out, returning zero features. There are no errors or warnings in the log and the workspace keeps running after the reader times out. There are about 3 million features in the entire dataset (for the United States). I only need two states (New Mexico and Texas) but even clipping to a search envelope on the reader makes no difference. It's a publicly available service, no login required. What can I do to successfully read in the features from this service?

Is there a way to create osm Boundarys out of oracle Data?

First of all sry for my bad english! ;) I like to recreate the osm scheme out of our own scheme with FME Desktop We'v got lines transformed with the keys:boundary=administrativeadmin_level=3-8 and combined lines with the keys:osm_relation_type=boundarytype=boundaryboundary=administrativename=*admin_level=3-8 My Problem is the FME osm-writer creates his own geometries without the line-tags but i'd like to get my own created lines as a member of the Relations.

4 years ago

Page 75 / 273

Badge winners

kraaz9395has earned the badge Expert (Grey)
kraaz9395has earned the badge Teacher (Grey)
kraaz9395has earned the badge Student (Grey)
dmccullagh.npcahas earned the badge Integrate Data with the FME Platform Training
sclarke_97has earned the badge FME Form Basic Training

Show all badges

30,370: Posts

Terms & Conditions Cookie settings

Sign up

Already have an account? Login

An FME Account is required to contribute

Login to the community

An FME Account is required to contribute

Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.

Enter your e-mail address

Back to overview

Scanning file for viruses.

Sorry, we're still checking this file's contents to make sure it's safe to download. Please try again in a few minutes.

This file cannot be downloaded

Sorry, our virus scanner detected that this file isn't safe to download.

A product of

Company Products Giving Back Careers Contact Privacy Feedback

Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.