Using readers and writers to extract and load data.
Recently active
I have setup a test filter that separates out a number of shapefiles based on part of the filename. I now want to use the output port that these features will run through as the new table name. Is there an easy way to do this, or do I need to create a writer for each port so that this has the correct table name.
Hi,I got a workbench published to FME server as an Jobsubmitter service now i want to call or execute this work space from an oracle trigger which triggers after an update to that table.Below is my function would be called from the oracle table trigger but i can't see the workspace execution has started or completed.I got proper UTL_HTTP access to the user and no errors while executing the function.Can anyone shed some light what is the proper way to achieve this?**********************************************************************************************************CREATE OR REPLACE FUNCTION DBUSER.FME_CALL RETURN BOOLEAN AS message varchar2(20); req utl_http.req; BEGIN message := '99999'; -- Call the workspace req := utl_http.begin_request('http://FMEServer/fmejobsubmitter/training/xml2dwg.fmw','POST'); utl_http.set_header(req, 'User-Agent', 'Mozilla/4.0');utl_http.set_header(req,'Content-Type','TEXT/HTML&#
I'm new to FME so would like some help understanding what is happening in the backend. I am running FME 2017 Desktop.I have the below workspace which is extracting the min and max band figures of an ECW and logging whether all of the bands have a min of 0. At present I am testing with only 3 ECW files.I have been watching the process run, and I saw the the RasterBandMinMaxExtractor took around half an hour which I expected. However, once the first feature reached the appropriate logger, there was then nearly a 45 minutes wait before the next feature was run through the process.This is what appears in the Translation Log following the final Logger line.2017-04-26 14:23:04|2301.7| 0.0|INFORM|================================================2017-04-26 15:07:26|3096.3|794.6|INFORM|Creating reader for format: ER Mapper ECWI checked the temp directory and there were no temp files being created during the time. So I'm a bit stumped on what's going on.TIA
Hi i am new to FME,we are trying to set up 2 tier architecture.Is that mandatory to have the file system installed on a separate server (please refer the attached image for the instruction i found online) or it can be on the same web/Application server itself? I assume that this file server is a shared directory and can be anywhere on the network.Thanks,Mari...
I'm exploring the new Geopackage tiles writer in FME2017. I would like to to create a GeoPackage with the "Use incoming feature" tiling option. In this case the projection should be UTM35. This works fine, but the option to generate additional zoom levels is disabled. This means that I have to use gdaladdo.exe on the GeoPackage file afterwards to generate additional zoom levels (overviews). Is there any way to create a GeoPackage with custom tiling and additional zoom levels?
Is it possible to manipulate text to conform to a standard if each line varies slightly?As you can see in the converted format I deleted the first 6-8 characters at the beginning of each line, removed some spaces and replaced with semi-collins some with commas, and added some ID at the end.This is currently done manually with a beefed up Notepad but I would like to feed the txt into a transformer to complete this.existing format 4 SDWP SAXEL FB N 47 58 46.1498 W 066 15 38.3245 5 FAWP TAVLU FB N 47 58 20.4669 W 066 13 02.6285 6 IWP GONEX FB N 47 56 54.2098 W 066 04 22.6074 7 IAWPL MATAX FB N 47 51 26.2613 W 066 03 30.7522 converted format SAXEL,N47:58:46.1498,W066:15:38.3245,16-05361 TAVLU,N47:58:20.4669,W066:13:02.6285,16-05361 GONEX,N47:56:54.2098,W066:04:22.6074,16-05361 MATAX,N47:51:26.2613,W066:03:30.7522,16-05361
FME 2017.0 reading CSV data including dates in the format 2009-04-23 with no quotes around them.I'd like to write this straight to an SQL Server 2012 table so I've connected the Reader straight to the Writer. However it won't write the date values from the CSV into the SQL Server table as a date field.So I thought I'd take advantage of the new DateTime functions to save having to stick a DateFormatter in the middle. So I've been trying things like @DateTimeFormat(@Value(col4),%Y%m%d) and @DateTimeFormat(@Value(col4),%Y"-"%m"-"%d) in the Value column under User Attributes in the Writer properties. But I can't seem to find anything that works. I've tried @DateTimeParse, @DateTimeCast, @DateTimeFormat but the Translation Log keeps throwing up warnings e.g. 'Failed to evaluate expression '@DateTimeParse(2008-02-29,%Y%m%d,repair)'. Result is set to null.'Is this actually possible or do I need to go back to using the DateFo
I need to convert 1 Channel, 1 Palette GEOTIFFs to GEOTIFFS with only 1bit (true b/w). So far I managed to split an incoming GEOTIFF-feature into several new GEOTIFF-features (one new feature for each used color in the palette). Afterwords I change them to black and white by using the expression evaluator. Despite the fact that only two colors are used it is technically a 3 channel 8-bit/channel Geotiff right now. By using a RasterSelector and the RasterPaletteGenerator I convert this to a 1 channel 1 Palette Geotiff, where the palette only has two used entries.However I found no way to save it as a true (1bit) bitmap with the GEOTIFF-Writer.Any ideas out there?Uwe
Hi, I have numerous DGN files and I want to set up a workflow that checks whether they are 2D or 3D files. I know I can use the DimensionExtractor to check individual features but I want to do a check on the actual file type itself. Is there an IGDS attribute I can expose to do this?I have both FME Desktop 2016 and FME Desktop 2017.Thanks
I'm using five Excel readers that are pulling the spreadsheet data from a REST endpoint (URL). The purpose of the model is to combine all five spreadsheets into one output. They have the same schema. The issue I'm running into, is that the spreadsheets will all have a variable number of rows from week-to-week and when I specify the Cell Range in the Reader Parameters as (*:) the writer is still including all of the blank rows below the data for each spreadsheet, so the final output has a chunk of data, blank space, chunk of data, blank space, etc. Is there any way around this?
Hi,I just encountered an unexpected problem with the MSSQL_SPATIAL writer.I originally set a manual command ("exec some-stored-procedure-with-arguments") to define a spatial index and more, which err'ed because there was no primary index on the relevant table. That's fine and understandable.As I really didn't needed a spatial index on the specific table anyway, I just switched the dropdown "Spatial Index Type" to "None", and reran the translation.But the translation still failed in the same place, while trying to execute the now deselected and unwanted command.It seems like the command was still present, and FME tried to execute it as such, even though the setup stated it shouldn't. This I believe to be buggy behaviour, the overall choice should not be overridden like that.Cheers Lars I.
The best practice to print 20,000 rows of 6-columns each in pdf landscape A3 page size while to repeat the 6-columns 3 times in each page ,I couldn’t figure out how to print 20,000 rows of 6-columns each in pdf landscape A3 page size such that the 6-columns can be repeated 3 times in each A3 page as shown in the screenshots below
Dear FME-Users, I’d like to create JSON-files which describes some metadata of NetCDF input data. It works well when I use the “RasterToPolygonCoercer” before, but this transformer takes a lot of time. Without conversion to vector data the JSON-file doesn’t describe the metadata of every single pixel, even by using “RasterCellCoecer”. Please take a look in the attachment, there are 2 screenshots of 2 JSON-files. Necessary is that every cell value of the input data is listed in the JSON-file. Maybe it works with the “JSONTemplater”, but I don’t have any experience with this transformer. If you have some ideas, it makes me happy. :) Thank you very much and best regards. Konrad
Hi, I'm trying to create an error file with AutoCAD DWG/DXF Reader/Writer and i've got an issue : Specified block 'CENTRO_LOCAL' is not defined in AutoCAD file. Skipping featureMULTI_WRITER: multi_writer_id `3': Writing feature 0 AutoCAD Writer: Performing database audit...AutoCAD Writer: An error occurred for dataset 'D:\\Routines2014\\LOGS\\DWG_ENGLOS_01122016.dwg', in function AutoCAD2004Writer::setExtents. Exception thrown: 'Invalid input'A fatal error has occurred. Check the logfile above for details... Last line repeated 2 times ... Anybody's got an idea ?
Is it possible to add extra z-value to a point cloud (las) with 3D affiner? I need to add extra 0,507m to my already existing z-values. I'm changing from a local to an national height system but i don't know the parameters from the local one.regards, Anton
Dear FME-users, I’ve created a GeoTIFF-file based on NetCDF-data. The problem is the spatial reference. A compare to OSM-data in the attachment shows you, that the GeoTIFF-file is not placed right. The second picture is from Copernicus and right placed. Do you have an idea? Thank your very much. Best regards. Konrad
Is this possible? Or do I have to recursively use FTPCaller for "each level" of the ftp/sftp and build the strings manually? This is on both FME Desktop and FME Server. Even better: Add support on FME Server to "Add resource" and Type of Resource Connection would be ftp/sftp
Not long ago, I updated the FME from 2017.0.0.1 to 2017.0.0.2, But today I found that it can't use Esri Geodatabase (File Geodb ArcObjects) format. If I choose Esri Geodatabase (File Geodb ArcObjects) format, FME prompts the error and automatically shut down. And I open workspace that include reader or writer with Esri Geodatabase (File Geodb ArcObjects) format, I try to change reader or writer's parameters, also can appear the same problem. FeatureReader and FeatureWriter have the same problem. I remember in the previous version?FME 2017.0.0.1 64bit?, there is no problem .Also installed the FME 2016.1.3.2 64bit on my PC?Use FME 2016.1.3.2 64bit can read and write with Esri Geodatabase (File Geodb ArcObjects) Without any problems.I also refer to the following URL?I confirm that I have installed [ArcGIS for Desktop and 64 bit Background Geoprocessing].Notes on FME and Esri Versions and CompatibilityDon't know if anyone had experienced the same problem with me. I want to know
Hi! I have been looking into the FBX reader/writer functionality in the 2017 version. All works as intended regarding the 3Dmodelling which is great! But i was hoping to add/read attribute data to/from this format.We use the Unreal engine to visualize and have the FBX format as our input to it.I read on the reader/writer page for FBX that it is not available in this versionSo, attribute data on the FBX format, is this something that is being looked into at the moment or is it beyond the horizon?We are mostly looking forward to the potential of writing the attribute data onto a FBX model.Thanks/David
I have a workspace in which I have selected a CSV file writer to contain the results. The user parameters selected to be written appear in the CSV file. On the other hand, the format attribute (fme_text_string) I have selected does not appear in the CSV. All the results appear in the data inspector, and for a workaround I am forced to select the data in the table and save it to a file from there.What am I doing wrong?
I spent most of today struggling trying to get a Socrata SODA API dataset to an ESRI file geodatabase. I am able to read the data using the text reader and this url as the source:https://chhs.data.ca.gov/resource/c2p3-ahn4.json?$where=county_name%20=%20%27ALAMEDA%27&$limit=5 This is valid JSON according to JSON lint, but no geometry is recognized because it isn't in a GEOJSON format. I have tried using the JSONFragmenter/JSONExtractor/JSONUpdater transformation tools but just can't get the hang of it. I am using the Data Interoperability extension for ArcGIS. Any help is appreciated.
The title says it all. Directly you can not upload these workbenches. But maybe using the FMEServerJobsubmitter. The password is then passed using published-parameter. You could upload a dummy workbench with the same name and exchange these directly on the FME server against the password protected Workbench. Or are there other tricks?
Hi,we´re having a AGOL account were we would like to put other stuff beside naps, layers, applications etc. It would be nice if it would be possible to write / upload a PDF or DWG file (item) to a specific content folder / group, so that our users easily could access this information in one place. Using the AGOL writer it seems that it´s only possible to select folders with feature services in?Christian
Hi, How can i extract feature service from arcgis online. I've feature service that i want to pull them down to local file geodatabase using fme. Kindly help
I'm a GIS guy and new to MongoDB. I'm trying to use FME to read and analyze data in a Mongo database. The collection I'm trying to read has 10 million plus records so I'd like to be able to filter that down to a subset for testing. If this was an Oracle or ArcGIS reader I'd place a SQL whereclause on the reader to limit what FME starts with. On the MongoDB reader I don't see a whereclause option. I see a Collection Qualifer and a JSON query parameter on the Feature Type window, so is this where I would apply a MongoDB filter? I tried entering so MongoDB'ish find() code in this but it doesn't work, so maybe I'm entering it wrong.To simplify an example, here's what I'd like to do on my reader in SQL:select * from MyTable where MyStringField = 'MyClassification' And here's the Mongo translation of the same thing:db.MyTable.find({ "MyStringField": "MyClassification" });Now I'm left figuring out how to