Using readers and writers to extract and load data.
Recently active
We use the same datasets on a number of projects. These are sourced from the following site http://environment.data.gov.uk/ds/catalogue/index.jsp#/catalogue, currently via download and manual management. We would like to build a process where users can set a tool running to download and manage the data. My thought is along the lines of:1. Go to the website2. Tick the WFS box for the layers you want on your project3. As you do this the WFS URL changes at the top of the page4. Copy this and paste it into a pre-made workbench5. Define the MBR of the project within the reader so only the relevant data is included – this part could be done by a clipper and a specific polygon in the workbench but then we would have to access the entire WFS. The thought is specifying an MBR in the reader moves the extent query to their server greatly reducing the data we have to download and the speed to run the final tool6. Run the workbench. It would create a File Geodatabase or any other format based
I am considering using FME Cloud for a series of quite small jobs. Each would take only a few minutes to run, and they would be infrequent enough that I would want to pause the instance between jobs. I understand 1 hour is the minimum billing amount, but would each time the instance is started be billed for one hour's running time, or would the times be added up?For example, say you run one five-minute job every day for a week. Would that be billed for seven hours (seven start-ups) or 1 hour (35 min rounded up)?
Any particular reason why I cannot find search envelope parameters for the FeatureReader when pulling in a WFS stream? Any assistance would be helpful :)Thanks.
I have got a shapefile (left picture) where the donuts have been included as part of the polygon. I need to try to get the outcome of the right picture, which is the correct polygon of the site and which will allow to calculate the correct area of the site.Is there an easy way of adding a transformer that fixes that?
Folks I have an An autocad_surface that is listed as a fme_aggregate. Want to convert it to a surface via something like numericrasterizer. However its only doing the line edges of the triangles - and not the flat surfaces. Any thoughts? Should i run through deaggregator first ?
i want point cloud generation code and algorithm
I have a GeoDatabase in which roofs and façades are defined by different appearance styles (diffuse color). I used the AppearanceExtractor and AttributeFilter to filter the data. I'd like to produce a CityGML file with the appropriate semantics, but if I connect the AttributeFilter output to the Writer the produced file does not contain any geometry but just the other GDB attributes. Any hints? Thanks
Hi,When I check my data in FME Data Inspector I have all my rows, but when I write the data to an Excel spreadsheet I have a lot of empty rows and some rows populated.Does any one face the same situation ? Thanks.Jacob
Hi,I have 100,000+ points I would like to group into a fixed number of members as spatially optimal as possible.The data comes delivered to me in either text or csv containing a variety of attributes including postcode. I have geo-referenced each row to the nominal centre of each postcode using OS Code Point. This gives me the easting and northing for each row. I have tried to convert this to an xyz to then follow the suggestions in this post https://knowledge.safe.com/questions/24064/cluster-points-based-on-location-k-means-method.html. However, I suspect due to my limited working knowledge of FME, I'm struggling to create the correct workflow.I have tried another approach in FME creating voronoi polygons from the points and then using PointOnAreaOverlayer to add the overlap/count of points in the polygons. I thought I could then look to dissolve the polygons based on nearest neighbour locations until the desired count is reached.Can anyone provide any advice, tips or instructions
I am using "Import Feature Type" to add an additional shapefile feature type to my canvas for my Shapefile Reader. When I go to click the Run button nothing comes out of that new reader, the feature types that were added before this still work. Why doesn't my new shapefile data come through?
Hi!I have a problem with a ECW Raster. When I try to write to ECW I get the Log information "ECW writer: The datum in coordinate system 'RAW' is unsupported. The resulting file will be bave no datum." So i assume FME somehow reads this 'RAW' information from the ECW header. How can I read this information into my workflow so that i could edit it somehow in the process.I've tried to set the correct coordinate system and so on but it didn't worked out.
Dear community,When I try to scan a directory where the folder contains brackets the files in this folder are not being read.I can imagine why brackets cause problems but sadly this data is often external input. Does a workaround exist? (Other than scanning folders for brackets and edit the folders.)folder:output: path_windows D:\\tmp\\New folder (25)\\input\\test D:\\tmp\\New folder (25)\\input\\[test] D:\\tmp\\New folder (25)\\input\\test\\data.txt D:\\tmp\\New folder (25)\\input\\test\\[data].txtmissing output: D:\\tmp\\New folder (25)\\input\\[test]\\data.txt D:\\tmp\\New folder (25)\\input\\[test]\\[data].txt
I am using a PostGIS reader for a table with geometry type "Point". Once the workspace generated, the feature doesn't have the geometry attribute anymore, does anyone knows why?
Hello,i have an issue with the convertion of shape polygon to autocad Mpolygon (with map-data)The mpolygons are ok displayed by autocad map, but when they are opened with other CAD (briscad) programms some of them dissappear. It seems to have something to do with the loop direction of the mpolygon. Autocad is able to detect the incorrect polygons but not able to repair them. (audit the drawing, "incorrect loop direction")When i import the mpolygons directly in autocad, this problem does not accur. So it has something to do with FME.I have been looking for an attribute which could let me change this, but I have not found one. Is there a way in FME to get to change to the loop direction?thanks in advance
I'm needing to write a reasonably complex GML v 3.2 file that will be created from a series of flat tables. I'm not sure how to construct the data in FME so that the writer will create the "nested table" structure required by the GML.An example of the output structure would be as follows:<gml:featureMember><geometryProperty><gml:Point> etc...</geometryProperty><propertyType>Private</propertyType><addressList><address><number>12</number><street>Main</street><zip>39945</zip></address><address><number>12b</number><street>Main</street><zip>39945</zip></address><address><number>13</number><street>Main</street><zip>39945</zip></address></addressList></gml:featureMember>Is there a tutorial on using the writer for complex output like this or can someone advise what the data should
Hello, When you use the SDERASTER reader or the SDERASTERCATALOG writer or the SDERASTERMAP writer it allows you to initially use the OS Authentication option but when it gets added to the canvas the username and password fields are empty and red. It won't let you run the workbench until they are filled in. How do you get it to use the OS Authentication option correctly? Thanks, Peter.
Does anyone know if in FME it is possible to create a PDF with different layers (shapefiles) where you can switch off and on their visibility?
I'm trying to read a SpatiaLite table with a Primary Key Unique ID (PKUID) field, made in QGIS. All the fields of the SpatiaLite table can be read, except for this PKUID field. Even an AttributeExposer does not do the trick. Is there a reason for this? For the moment, I created a workaround: I read the table, together with the spatial data, in a SpatiaLite reader. Then I read the same table in an SQLite reader, which can detect the PKUID field, but does not contain the spatial data. A FeatureMerger does the rest.
I have successfully implemented a WorkspaceRunners that run a workspaces that read in from CSV files, IFC files and Shapefiles. However trying to implement a similar WorkspaceRunner to read a batch of Sketchup files fails. If the WorkspaceRunner sends the Source filename that the Workspace being called was created with it works fine. However sending another Source filename, the workspace being called fails to read that file. On a different note, I cannot see how to use WorkspaceRunner to read a batch of Feature Datasets in a geodatabase.Thanks
I am creating a Spatialite database with FME 2016.1 64 bit. I have both polygons and lines. Polygons shows up perfectly, but the lines are all over the place. I have added a picture showing the lines that should have depicted a road network in a municipality. Have anyone had the same experience or any idea why the lines drawn like this? I have been trying to adjust how ArcMap reads the data with the "query dialogue" but no success. My ArcGIS version is 10.3.1
I have 13000+ CSV files, all containing around 13000 rows. These CSV files are arranged in 146 different folders. All CSV files have the same schema with ; as separator. These CSV files contain about 175 million rows of data combined. The folders are named as seen in the picture below (for example ...\\CSV_folders\\5785xxx)CSV files are named like these:I already tried to copy all CSV files to same directory and run them to one GDB with FME. This went well... but resulted too heavy ESRI GDB that takes forever to search through. (175 million rows, around 20GB)Now I would need to make things a bit differently. I would like to work with FME to create one ESRI Geodatabase. It should contain tables named like these folders containing CSV files. A table in the GDB should have all the CSV files loaded in the table, that are inside the original folder. This would split my 175 million rows to 146 tables, making it a bit faster to search trough with Arcmap.So my problem is, how to make FME read
Does anyone have an example of using the JSONUpdater? I am trying to update multiple Json files (up to 50) without opening them up and finding my value to change but am looking for FME to help fill this gap. Thank you in advance!
Hello,I'm exporting OBJ files from a CityGML dataset. In the mtl file, FME write a Tr attribute. Can somebody please tell me what is this parameter (I don't see it in the wavefront format definition)?
I have a workspace that contains an ArcGIS Portal writer with mixed-case field names. It runs without a problem when first creating the feature service on ArcGIS Portal. However, when I attempt to use that same workspace to write updates to that feature service, I see warnings similar to the following: ArcGIS Online Feature Service Writer: Attribute 'myFieldName' on feature type 'Feature_Type' does not match any field on the corresponding layer/table. The attribute will be ignoredIf I check the feature service itself, I can see that all the mixed-case field names have actually been set to lowercase (so, myFieldName is actually set to myfieldname).