Skip to main content

Data

Using readers and writers to extract and load data.

  • 6,931 Posts
  • 25,504 Replies
6931 Posts
philippeb
Enthusiast
philippebEnthusiast
asked in Data
GTFS-realtime reader

Hi community, I have links (with token) here that give me the opportunity to have GTFS data into realtime. So I could have bus realtime position or alerts by exemple : https://developers.google.com/transit/gtfs-realtime/ Would you know a method to read this data into FME? My goal would be to read this data and update the last position into ArcGIS via FME. If you would know a way to use the GTFS-realtime into ArcGIS, I would be interested too. Honestly I don't know what to to with those links... thank you

621
david_r
Evangelist
S
sani
asked in Data
To copy contents of a file into another.

Hello, How to copy the contents of a file (File_m.csv) into another file (File_n.txt). Such that it does not overwrite but only appends to File_n.txt. File_m.csv File_n.txt output expected: File_n.txt A B C D E F 3 5 6 2 7 7 A B C D E F M N O P Q R 8 7 5 0 1 3 M N O P Q R 3 5 6 2 7 7 8 7 5 0 1 3 Regards.

190
S
jamal
jamal
asked in Data
Converting ESRI shape file to coverage,

Converting ESRI shape file to coverage, I applied the tool “feature class to coverage” available in the ArcGIS to convert shape file to coverage but I got the error below: How can the shape file be converted in FME to Coverage? The result is expected to take the following format: Thank you Best Jamal

230
jamal
R
rob
asked in Data
Collapse (text) records where contiguous

Hi, Can anyone help me devise a method using FME to collapse a series of text records with no associate geometry, assuming that 2 values are equal and 2 values are contiguous. I have included some example data below, data is arrange such that the Animal and Colour values must match the preceding row and the preceding End value must match the Start value. I have a very nasty script for excel which may be possibly be adapted, but I would really like to do this with FME. I have tried to do some of this within a loop but got a bit lost. Could it be down using transformers or a Python script which can be included in a bench? Ideally any method needs to be adaptable as I would like to use it for other data sets with differing field names and more or less matches required. Animal, Text Colour, Text Start, Int End, Int Input Data Format Animal,Colour,Start,End Dog,Blue,0,1 Dog,Blue,1,5 Dog,Blue,5,10 Dog,Red,31,35 Dog,Red,35,56 Cat,Purple,101,102 Cat,Purple,103,106 Cat,Purple,106,111 Required O

130
takashi
Contributor
nrich
Contributor
nrichContributor
asked in Data
PDF Writer crashing when writing pdf_text to a fanout dataset

HI, I'm using the pdf2D writer in FME Workbecnh 2011 to create a bunch of maps. I'd like to use the fanout feature on my pdf writer, however when i add a pdf_type of pdf_text to the writer it crashes fme.exe when the writer starts to open I think. If i remove this pdf_type, by stroking the text then i can write the pdf with fanout, however if i drop the fanout i can write the pdf with text! I'm not sure it's a bug in the writer as such as I can create a much simpler workbench and write text. The bits i've dropped im my simple workbench that work are adding multiple pages, altering the page size to A4 portrait and landscape. I'll keep plodding though the workbench until i break my simple workbench and hopefully incover the problem, however i thought i'd ask here in case anyone has suffered a similar issue when writing pdf's? (in 2011 - as my server instance is still 2011) Nick

100
gio
Contributor
bubblebeb
Contributor
bubblebebContributor
asked in Data
spatial relator alternative

I have a layer of around 21m features and I want to group them geographically using a region layer which has around 10 regions. I have used the spatial relator which I think is working but it has been running now for 17 hours and has only processed 687,000. Is there a better way of doing this with a process that does not take so much time to run given that I have a large number of features to process.

1340
fmelizard
Contributor
J
joost
asked in Data
Text to Text Node

Hi, I'm trying to convert a DGN file with text to a DGN file with text nodes. What do I have to do? Any help would be appreciated.

140
J
jon
Contributor
jonContributor
asked in Data
Create DEM of highest point return

I've never worked with point data in FME before so bear with me. I have a request to create a DEM or surface model of the highest vegetation returns. The .LAS file is classified into Ground, Low Vegetation, Medium, Vegetation, and High Vegetation, etc. My assumption is only use the High Vegetation values and get the max value and create a DEM. Is this the best approach and if so, what transformers should I use? I've played around in FME but aren't really getting the results I'm hoping for. Should I use the ExpressionEvaluator to get the max value of Z values? If so, what's the syntax for this? Any thoughts would be helpful and I can provide more clarification if needed.

100
jon
Contributor
V
vito
asked in Data
Autocad Color

i am trying to transform Oracle to Autocad. I need to set some line to a user specifed color. the color is in RBG and not a autocad color index number. How would I go about doing that? Can I substitude autocad_color with fme_color?

110
V
D
djabx
asked in Data
How to merge all feature

Hi everyone, I've got two flow of data: _FA with attributes (a, b, c, d) and _FB with attribute (e) I've got X (>1) FA and only 1 FB. And I would like a way to have X features like this: _FC (a, b, c, d, e) How may I do that ? I first tougth of FeatureMerger but I can't. Any solution ?

70
D
jamal
jamal
asked in Data
Publish to FME Server: Workspace Name issue,

Publish to FME Server: Workspace Name issue, I couldn’t figure out why “Publish to FME Server” doesn’t take the name of my file as “Workspace Name” but instead it takes other name My file name is: UrbanMasterPlan_CAD_To_GIS_03.fmw The name read by the FME (Workspace Name) is: T1_Published From where the “T1_Published” is read considering that the name of my file is “UrbanMasterPlan_CAD_To_GIS_03.fmw” as shown in the screenshot below What might be the issue here? Thank you Best Jamal

110
jamal
R
rick_williams
asked in Data
Excel Writer: Dynamic Row and Column using Template File

I have an Excel report template in which I need to update specific row/column fields and they are not in consecutive order. In other words, I want to update Row1,Column3 and then Row10,Column8, etc.... Is there a way to dynamically set the Start Column and Start Row parameters within the Writer Format Parameters? I tried, with no success, creating Row and Column attributes and then setting the parameters within the writer as @Value(Row) and @Value(Column). Or is there a better way to do this when using a template for a Writer?

240
R
T
toby
asked in Data
Read/write to same GDB

Hi, I'm having trouble reading from, and writing to, the same ESRI gdb. I'm just doing a simple transformation and have tried on a few different gdb's but get the same problem - it just hangs part way through the process. I'm assuming it is possible to read and write to the same gdb. I've made sure all ESRI products are closed to prevent any locking, but to no avail. Any help appreciated. Cheers Toby

850
david_r
Evangelist
H
harry
asked in Data
Inappropriate handling of non-numeric values in shapefile reader / writer

Hi, We've stumbled across what seems to be a bug (or perhaps a design decision that we haven't understood!) in the shapefile reader and writer. The DBF format defines the type of a field in its header, but there is nothing physically preventing a field that is numeric from having text data stored in it (because internally, numbers are stored as text in the DBF format). Hence, it is up to any implementation of the DBF format to make sure that the correct types of values are written to a field, and to behave sensibly if the wrong types are read from the field. FME doesn't seem to do this. We translated some data to shapefiles. The data have a column entitled "Number" which contains road numbers. We set up the shapefile writer to store this data in a numeric field, without properly inspecting all the data. However it turns out that the data actually occasionally contain text values, such as "A3030" (reasonably enough because that's how British roads are numbered). I would argue that the e

80
david_r
Evangelist
A
amit
asked in Data
Extracting photos from Microsoft SQL Database to network drive

Hi everyone, I have photos store in a Microsoft SQL Spatial Database which I am trying to extract out on our job drive (i.e. a normal network drive). I've had a search of previous responses and it seems that the AttributeFileWriter is the relevant transformer, however I can't get it to work! The photo is stored in an Attribute field called P_Picture. I added the relevant reader and connected the AttributeFileWriter with the following parameters: Source: P_Picture Target Filename: P_Picture If Target Exists: Overwrite File Target File Character Encoding: System Default (fme-system) When I run it, I get an error: "AttributeFileWriter: This FME edition does not recognise the 'o' function. Please ensure that the current platform supports this function, the function name is spelled correctly, and that you have installed all the required plugins" I'm am running: Win 7, 64-bit, FME 32-bit, SP4 ESRI Edition (should be able to read db format in) Any pointers would be greatly appreciated! Thanks

460
fmelizard
Contributor
P
pheller
asked in Data
Print format errors after plotter change

We recently switched from HP plotters to EPSON plotters, and have had multiple issues with these over the last couple of months. Strangely, our floating licenses are able to print to these plotters without issue, our single use license user is having some problems. When he prints to these new Epson plotters, his print format have numerous errors. Some bubbles are larger, some smaller, some colors are off, some pieces are missing. The text appears to be untouched, but any bubble surrounding the text is skewwed, missing, or someplace else on the page. It is almost as if his license has a special driver or print filter that is doing this. Suggestions?

80
P
G
gistest
asked in Data
Colorramp to stretch raster image

Hi all, In GDAL it's easy to stretch a raster image with a colorramp that can look for example like this: 0 255 255 255 5 225 225 225 10 205 205 205 15 190 190 190 20 178 178 178 25 166 166 166 30 154 154 154 35 142 142 142 40 130 130 130 90 0 0 0 This ramp stretches for example every pixel with a value between 0 and 5 to an rgb value between 255 and 225. What is the smartest way to do the same thing in FME Workbench? I thought about using the RasterExpressionEvaluator but it seems to me like the expression needs to be quite complicated. Is there a better way of doing it?

600
david_r
Evangelist
esko
esko
asked in Data
I need to change table in Oracle reader, but how (FME dt 2014)

Is there any way to easily switch Oracle Spatial reader to read from another similar (ie. replica) table - than to rewrite to whole reader part ? This would ease my job a lot as I have several FME wb's with input data filtering (to reduce memory usage) from the same source data. Rgrds, Esko

131
K
sigtill
Contributor
sigtillContributor
asked in Data
Writing to FileGDB

When trying to WRITE with the Esri Geodatabase (File Geodatabase API) to an existing file I get the following error: Geodatabase Error (-2147024891): Access denied. FileGDB Writer: A feature could not be written. Some issues: 1. The workspace does not FAIL, but completes.Even though the writer gives this error. There is no option on this writer to IGNORE FAILED WRITES: NO 2. Why is access denied? I have created a completely new FileGDB from within FME and I am trying to update this.

341
K
S
si
asked in Data
excel found unreadable content

Hi, I am exporting a semi large dataset from FME workbench (2013 SP2) through a writer as xlsx format. Excel is saying "Excel found unreadable content in 'Export.xlsx'. Do you want to recover etc". I say yes, but when I do try to recover, excel only recovers a small amount of the data. I also get the following error in excel. Errors were detected in file 'Export.xlsx': Removed Part: /xl/sharedStrings.xml part with XML error. (Strings) Illegal xml character. Line 1, column 55442. Removed Records: Cell information from /xl/worksheets/sheet1.xml part Inspector with FME shows all the expected data. I can reload the file back into FME and all the expected data is there. Why is the data not exporting correctly? Any ideas?

1930
david_r
Evangelist
aguan
Contributor
aguanContributor
asked in Data
Can FME Desktop accesses ArcGIS server map service as a reader?

Can FME Desktop accesses ArcGIS server map service as a reader?

130
aguan
Contributor
P
pietrorossin
asked in Data
batch multiple txt files and append to postgis

Dear community members I'm facing a problem with fme desktop. I have many xyz files and they must be imported into a single postgis table. They are all equal, x,y,z column. I must query each file so the first/last row/column will be discarded due to artifacts. Then the result for each file must be appended to a table. Right now I used a csv reader with an InlineQuerier and a 3DPointReplacer in sequence and for a single file it works with a query set in the inline querier If I add more than a single xyz file to the reader only the first/last row/column of the whole set of points are removed.. So how can I make fme to process the query on each single file efore it is added to the output? Thanks Pietro

110
takashi
Contributor
venu
Contributor
venuContributor
asked in Data
Building dectection and mapping from las file in FME

Hi, I have Las file in that I need to map 3d buildings.Is there any procedure generate in FME. Please guide.

50
gio
Contributor
tbs_gis_team
Contributor
tbs_gis_teamContributor
asked in Data
Spatial filter not working against SDE reader.

I am trying to filter SDE database with spatial filter and it is not working. It works fine with a clipper, but I am not wanting to clip the data. I used this before with an FDO connection and it worked fine. Thanks, Chad

60
tbs_gis_team
Contributor
parashari
parashari
asked in Data
DBF Writer: DBF file exceeds maximum size of 2 GB

Hi, I am working on an XP machine with 2Gb RAM. I am trying to run some tools and everytime, the output shows the following details. What can be done regarding it. "DBF Writer: DBF file exceeds maximum size of 2 GB Unable to output data to file 'D:\\SixMonths\\FME\\New Folder\\SQL\\New Folder\\NewFeatureType.dbf' ... Last line repeated 2 times ... DBF Writer: DBF file exceeds maximum size of 2 GB."

251
david_r
Evangelist

Most helpful members this week

Badge winners

  • FME Form Advanced Training
    vmerguhas earned the badge FME Form Advanced Training
  • Integrate Data with the FME Platform Training
    rodavcashas earned the badge Integrate Data with the FME Platform Training
  • FME Flow Authoring Training
    bettsgeospatialhas earned the badge FME Flow Authoring Training
  • Welcome Back
    jimmy.chaohas earned the badge Welcome Back
  • FME Flow Authoring Training
    paigehhas earned the badge FME Flow Authoring Training
Show all badges
30,846
Posts

Popular tags

Terms & ConditionsCookie settings

Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.