Using readers and writers to extract and load data.
Recently active
Hi, I'm having trouble reading from, and writing to, the same ESRI gdb. I'm just doing a simple transformation and have tried on a few different gdb's but get the same problem - it just hangs part way through the process. I'm assuming it is possible to read and write to the same gdb. I've made sure all ESRI products are closed to prevent any locking, but to no avail. Any help appreciated. Cheers Toby
Hi, We've stumbled across what seems to be a bug (or perhaps a design decision that we haven't understood!) in the shapefile reader and writer. The DBF format defines the type of a field in its header, but there is nothing physically preventing a field that is numeric from having text data stored in it (because internally, numbers are stored as text in the DBF format). Hence, it is up to any implementation of the DBF format to make sure that the correct types of values are written to a field, and to behave sensibly if the wrong types are read from the field. FME doesn't seem to do this. We translated some data to shapefiles. The data have a column entitled "Number" which contains road numbers. We set up the shapefile writer to store this data in a numeric field, without properly inspecting all the data. However it turns out that the data actually occasionally contain text values, such as "A3030" (reasonably enough because that's how British roads
Hi everyone, I have photos store in a Microsoft SQL Spatial Database which I am trying to extract out on our job drive (i.e. a normal network drive). I've had a search of previous responses and it seems that the AttributeFileWriter is the relevant transformer, however I can't get it to work! The photo is stored in an Attribute field called P_Picture. I added the relevant reader and connected the AttributeFileWriter with the following parameters: Source: P_Picture Target Filename: P_Picture If Target Exists: Overwrite File Target File Character Encoding: System Default (fme-system) When I run it, I get an error: "AttributeFileWriter: This FME edition does not recognise the 'o' function. Please ensure that the current platform supports this function, the function name is spelled correctly, and that you have installed all the required plugins" I'm am running: Win 7, 64-bit, FME 32-bit, SP4 ESRI Edition (should be able to read db format in) Any pointe
We recently switched from HP plotters to EPSON plotters, and have had multiple issues with these over the last couple of months. Strangely, our floating licenses are able to print to these plotters without issue, our single use license user is having some problems. When he prints to these new Epson plotters, his print format have numerous errors. Some bubbles are larger, some smaller, some colors are off, some pieces are missing. The text appears to be untouched, but any bubble surrounding the text is skewwed, missing, or someplace else on the page. It is almost as if his license has a special driver or print filter that is doing this. Suggestions?
Hi all, In GDAL it's easy to stretch a raster image with a colorramp that can look for example like this: 0 255 255 255 5 225 225 225 10 205 205 205 15 190 190 190 20 178 178 178 25 166 166 166 30 154 154 154 35 142 142 142 40 130 130 130 90 0 0 0 This ramp stretches for example every pixel with a value between 0 and 5 to an rgb value between 255 and 225. What is the smartest way to do the same thing in FME Workbench? I thought about using the RasterExpressionEvaluator but it seems to me like the expression needs to be quite complicated. Is there a better way of doing it?
Is there any way to easily switch Oracle Spatial reader to read from another similar (ie. replica) table - than to rewrite to whole reader part ? This would ease my job a lot as I have several FME wb's with input data filtering (to reduce memory usage) from the same source data. Rgrds, Esko
When trying to WRITE with the Esri Geodatabase (File Geodatabase API) to an existing file I get the following error: Geodatabase Error (-2147024891): Access denied. FileGDB Writer: A feature could not be written. Some issues: 1. The workspace does not FAIL, but completes.Even though the writer gives this error. There is no option on this writer to IGNORE FAILED WRITES: NO 2. Why is access denied? I have created a completely new FileGDB from within FME and I am trying to update this.
Hi, I am exporting a semi large dataset from FME workbench (2013 SP2) through a writer as xlsx format. Excel is saying "Excel found unreadable content in 'Export.xlsx'. Do you want to recover etc". I say yes, but when I do try to recover, excel only recovers a small amount of the data. I also get the following error in excel. Errors were detected in file 'Export.xlsx': Removed Part: /xl/sharedStrings.xml part with XML error. (Strings) Illegal xml character. Line 1, column 55442. Removed Records: Cell information from /xl/worksheets/sheet1.xml part Inspector with FME shows all the expected data. I can reload the file back into FME and all the expected data is there. Why is the data not exporting correctly? Any ideas?
Can FME Desktop accesses ArcGIS server map service as a reader?
Dear community members I'm facing a problem with fme desktop. I have many xyz files and they must be imported into a single postgis table. They are all equal, x,y,z column. I must query each file so the first/last row/column will be discarded due to artifacts. Then the result for each file must be appended to a table. Right now I used a csv reader with an InlineQuerier and a 3DPointReplacer in sequence and for a single file it works with a query set in the inline querier If I add more than a single xyz file to the reader only the first/last row/column of the whole set of points are removed.. So how can I make fme to process the query on each single file efore it is added to the output? Thanks Pietro
Hi, I have Las file in that I need to map 3d buildings.Is there any procedure generate in FME. Please guide.
I am trying to filter SDE database with spatial filter and it is not working. It works fine with a clipper, but I am not wanting to clip the data. I used this before with an FDO connection and it worked fine. Thanks, Chad
Hi, I am working on an XP machine with 2Gb RAM. I am trying to run some tools and everytime, the output shows the following details. What can be done regarding it. "DBF Writer: DBF file exceeds maximum size of 2 GB Unable to output data to file 'D:\\SixMonths\\FME\\New Folder\\SQL\\New Folder\\NewFeatureType.dbf' ... Last line repeated 2 times ... DBF Writer: DBF file exceeds maximum size of 2 GB."
Hi, I have the foollowing error message when uploading a workspace on FME CLoud: Uploading as 'test.fmw'... Publish failed - FME Server encountered an error and reported the following message. <error><name>filename</name><message>The 'filename' matrix parameter is required</message></error> Request to http://geopol.inser-team.fmecloud.com/fmerest/v2/repositories/Samples/items returned status code 400. Error 400 means wrong filname extention. What's wrong in uploading a .fmw? I am experiencing this whatever workspace I upload. (FME 2014 Build 14230) I can upload on this FME Cloud instance with FME 2013 without problem. Thanks for your help. JP.
Hi Community, I got 3D-features, which are in their local coordinate system at the right position. But when I translate them to KML to watch in Google Earth they hang several meters above ground. I know there is a difference between Swiss height and WGS84 height. Is it possible to bring the 3D-features to the ground? Thanks in advance, GPT-Executer (Maria)
I have a Oracle database that store ArcSDE data. I read mapinfo data with FME (MAPINFO reader) And I write the data with FME (SDE30 writer) I use : FME(R) 2012 SP4 (20120926 - Build 12266 - WIN32) It's the same problem on server or desktop. My problem is about an attribut. It's a unique ID (ID_ORIGINE) that goes like this : Mapinfo value : 7271203048662889 FME (logger) value : Attribute(string) : `ID_ORIGINE' has value `7271203048662889' It's the same. But when I open my ArcSDE table with ArcMap or visualize the data from within Oracle with SQL Developper I have this : 7271203048662890 The end of the number is rounded (2890 instead of 2889). But I didn't ask for this. My data structure : Oracle : Number SDE : Double (precision 38 and scale 10) so it should go well cause If I insert it manually it's OK. But when FME write it down : the value is rounded. Is it FME that round the value while writing it down or Oracle or SDE that do the
Hi - I'm having some problems with translating the OS VectorMap Dataset - I know there's a reader for each of the OS Vector products - and I've cross posted on another thread (Issues Reading in GML) BUT what I need explaining to me are the different approaches that seem to have been adopted by each of the vector products, and the GML, and their Readers in FME. I've had no problem at all in translating OS MasterMap, I've created quite a nice workspace to translate and style the data into MapInfo; nor have I any problems with the VectorMap Local. But why does VectorMap District require to have an xsd called by a URI, and then store feature types in an SQLLite db (?) and so on... If anyone can enlighten me on the different approaches that seem to have been undertaken, then I would be very grateful !
I'm currently trying to read in some GML (as a first step to doing some more complex transformations on it), and I'm having some issues with reading it in, different depending on which method I've attempted to use. The file i'm trying to read in is an OSVectorMap District gml First method: Using the normal GML reader, I'm attempthing to read the file in, and pull the .xsd file from the Ordnance survey's website and I'm the error below. I know that I don't need proxy settings as I've used this method to read in OSVectorMap Local files in the past HTTP transfer error: 'Failure when receiving data from the peer' Please ensure that your network connection is properly set up No proxy settings have been entered. If you require a proxy to access external URLs, please ensure the appropriate information has been entered Second Method: The second method I’ve tried was to download the OSVectorMap District .xsd file, and alter the xml to reference it
I'm writing to a FGDB and keep getting a Warning message in the log file that says: WARN |Delete was not performed on feature for the table/feature class 'feature class name goes here' The thing is that there are 11 line features to be deleted, 9 of them delete correctly and the remain 3 stay in the geodatabase for some reason. There isn't really any other information in the log file as to why these 3 features aren't deleteing. I added an inspector to view the data going into the FGDB writer, and all 11 records are set to fme_db_operation = DELETE. I'm at a lose as to why 3 of them don't actually delete. Any ideas or other things I can check?
Hello all, I've got the following table: Building_ID, Color, Size, Roof 1,Red,1-10m,Flat 1,Blue,30-40m,Tiles 1,Green,20-30m,Flat 1,Red,20-30m,Tiles Now, I also have this related table which I want to populate: SubBuilding_ID,Item_ID,Subitem_Work,Subitem_Amount,Subitem_Units,Subitem_CPU I want to populate this using the following external file: Work_ID,Color,Size,Work,Amount,Units,CPU 001,null,1-10m,Paint,10,Liters,5 002,null,10-20m,Paint,10,Liters,5 003,null,20-30m,Paint,30,Liters,5 004,null,30-40m,Paint,40,Liters,5 011,Red,null,Flat,Flat Roof,100,m2,10 012,Red,null,Tiles,Flat Roof,1000,pieces,2 011,Green,null,Flat,Flat Roof,100,m2,8 012,Green,null,Tiles,Flat Roof,1000,pieces,1.50 011,Blue,null,Flat,Flat Roof,100,m2,6 012,Blue,null,Tiles,Flat Roof,1000,pieces,1 Now, I can get multiple Work types per Building, so I've got a 1:n relationship. The Excel layout is still up for debate, so it can be changed. I just can't find out how I would build rows using the external Exc
I have a PNGRasterWriter towards the end of my workflow. My project produces about 200 - 5,000 PNG files. In FME desktop I want to send a message (HTTPFetcher would be fine) each time a new file is finished being written. The GET HTTP packet would have to items of info: File Name File Size. The part that I am having trouble with is getting this informaiton AFTER the write is done. How can one do that? if this is really messy then I guess I will have to go earlier in the pipe (I do have a attribute called FileName) and then send that out without the size info. Any suggestions?
Is a spatial index required when reading a table/view using the Oracle Spatial Object reader? Are there any transformers that require the existance of a spatial index?
Hi, I'm using a workbench to launch a SQL query on a Oracle Database with many JOIN. I use SQLExecutor to do it. When I query a specific record, the result is very fast. But when I need to extract all the data from my query (more than 5000000 records) My computor goes slow and FME failed to execut it. I receive this : Insufficient memory available -- error code was 2 I need to extract all the point from my table ROL_UNITE_P and join 5 tables to add data. Should I use more than one SQL Executor? Should I use something else? Thanks! Francis This is my SQL query : SELECT rol.ID_PROVINC, rol.CODE_MUN, rol.RL0102A, rol.MAT18, adr.RL0101A, adr.RL0101B, adr.RL0101C, adr.RL0101D, adr.RL0101E, adr.RL0101F, adr.RL0101G, adr.RL0101H, adr.RL0101I, lot.RL0103A, prp.NOM_PROP, prp.PRN_PROP, prp.NO_COPROP, prp.ADR_POST, prp.MUN, prp.ADR_POST_COMPL_C, prp.ADR_POST_COMPL_P, prp.COD_POST, prp.DAT_INSC_ROLE, cd_gen.DESC_ AS CODE_GEN, cd_lien.DESC_ AS CODE_LIEN FROM rgora1a_res.ROL_UNI
Hi, i might be missing something fairly obvious, but i want to read standard ESRI GRID rasters into FME, work with them and write a single ESRI GRID raster. (Not DEM, GeoTIFF etc etc, just standard GRID). Do i have to convert the GRID to ASCII in Arc before FME can read it? and is this the case for writing too? thanks for any help
Hello guys, I have a big CSV file ( 1.3 Go) as an output of a modeling software. This CSV has 2 fields for Lat/Long and 145 more fields, each for a climate variable. I need to convert each variable in a raster, so 145 rasters, with a 500meters resolution (same as input resolution actually) I tried with the 3D point replacer wich seems to work, but I have problems to set the NumericRasterizer and the ouput is just full of NoDataValues. R is running out of memory while FME seems to do the job...good point ! Thanks !