Using readers and writers to extract and load data.
Recently active
Dear all, I am currently working on a project where I assess watershed areas upstream of several ditches. The area is a polder area (no significant internal differences in elevation) with flow direction somehow incorporated into the ESRI shapefile (see arrows in below figure), no clue how. In the end, all ditches lead to the red star. I have aready calculated the watershed area for each individual ditch. The next step would be to somehow add a column in my shapefile where all upstream watershed area's are added together to find the 'total watershed area' that provides the water flowing through the ditches.Some things to note: - Some ditches split into 2 other ditches. These, we assume, both get 50% of the water that flows from the upstream 'parent' ditch - Dark blue lines and lighter blue lines are major- and minor channels/ditches, and can be regarded as similar in function - Arrows show the direction of flow in all ditches - Since the area is a flat polder, elevat
I'm trying to extract attributes from the attached json string using a jsonextractor. I thought the jsonquery field would be json["datetime"] but nothing is being returned. Any help is gratefully received. ThanksN
when I view the drawing in map3d the cs is listed. When i view the dwg with arccatalog i can see the same CS. however when i add it to workbench it is not being recognized.I am trying to set up a dynamic workbench to re project coordinates dependent upon the cs of the reader being used. Is there a way to get the autocad cs and set that to a published parameter?
Hi all, I'm working with FME Desktop 2019.2 and I have encountered a problem with the FeatureWriter. I need to save features in a CSV file and I want to use FeatureWriter settings to manage escape char and to qualify field values only if needed. Below the settings used.The problem is that FeatureWriter doesn't qualify a string value like the following:- Lorem ipsum dolor "sit amet"While it works correctly with a string like the following:- "Lorem ipsum" dolor sit ametThe CSV file created by FeatureWriter using the two string above is:"Test_2","Test_1""""Lorem ipsum"" dolor sit amet",Lorem ipsum dolor "sit amet"
Hi All. I am working on fme workbench to convert json data from api into Arcgis online feature service. Actually i am quite new on this software.reader/ data link: https://covid19-public.digitalservice.id/api/v1/wilayah/jabar?level=kelurahanHowever, when i run, and export it as ArcgisOnline feature service the data on the arcgis online is empty its actually consist more than 5000features.feature service link: https://services6.arcgis.com/eV2M5wbf5ix69NmC/arcgis/rest/services/datajson22/FeatureServerand the one that makes me more confuse is when i export the data into file geodatabase format, i could see the all data. here the image: overall workflow on fme workbench: attached:fme workbench could anyone share what is the main problem from this or how can i solve this.Thankyou in advance
Hi,I would like to add a new row containing some descriptive information from a seperate stream above header row in an excel spreadsheet.ThanksDavid McDermott
Hi there, I am writing the output of FME translation in csv file using csv writer. However, I noticed that file does not show up in the selected folder. I tried to change the configuration of the CSV writer. But no luck. Couldn't anyone tell me where am I making the mistake? The attached file shows the parameter of CSV writer.
I'm using an ArcGIS Server Feature Service Reader to read in flood hazard polygons but it keeps timing out, returning zero features. There are no errors or warnings in the log and the workspace keeps running after the reader times out. There are about 3 million features in the entire dataset (for the United States). I only need two states (New Mexico and Texas) but even clipping to a search envelope on the reader makes no difference. It's a publicly available service, no login required. What can I do to successfully read in the features from this service?
First of all sry for my bad english! ;) I like to recreate the osm scheme out of our own scheme with FME Desktop We'v got lines transformed with the keys:boundary=administrativeadmin_level=3-8 and combined lines with the keys:osm_relation_type=boundarytype=boundaryboundary=administrativename=*admin_level=3-8 My Problem is the FME osm-writer creates his own geometries without the line-tags but i'd like to get my own created lines as a member of the Relations.
Hey everybody,recently I'm getting into converting 2d autocad plans into a simplified 3d model. So far I'm able to create a model. Now i have checked the tutorial of @daveatsafe in converting a dwg to ifc with rooms but I'm still having a question.How do I write the autocad layer property to an ifc file that I use in BIM dwg. Because at this moment whatever I'm trying I'm getting all the entities on layer 0.Thanks in advance,Andreas
Hi there!Do you know why the File Copy writer doesn't log an error when the source file doesn't exist?It only logs an warning:File Copy Writer: Ignoring the destination filename '<file>' because the source dataset '<path\to\file>' was not a file.It was problematic in my case because I monitore only the errors in the log files... :/Thanks in advance for your feedback,Rémi
Hi everyone, I'm trying to create polygons with arcs (airspace) from aixm 4.5 data. I found this extremely useful workspace from @takashi i here: https://knowledge.safe.com/questions/56273/how-to-create-a-good-geometry-with-coordinates-xy.htmlbut I'm having troubles with the pythoncaller to create arcs. While it works pretty well on some data subset (ead_identifier contains _EI) for others I get a "math domain error"(ead_identifier contains _EG) and I can't figure out why.Any help would be appreciated
Hi everyone! I am new to the FME World and I am testing the following scenario:Take a spreadsheet with lots of fields as input of several tables in a Postgres database. This unload should occur once a day.Has anyone expertise on this?Using FME Workbench 2019.1, my approach is to place the xlsx file as a reader. Then, use an attribute manager to remove certain fields and leave only those that are required for a specific table abc. After, I add an Postgres writer to to receive such information.Is this a good approach? How/where do I drive the "insert" to a specific table? How it the scheduler implemented (if possible)?Thanks a lot!
Hello,FME 2020.0.1.0 build 20218 Win64I'm trying to create MS Word documents with the MSWORD writer. Everything is good for now, except Paragraph Spacing.Usually we prefer to set "Compact" or "No Paragraph Space" in any documents, but the MSWORD writer seems to always set "Open".-- from a web page in the Microsoft siteI know that the Paragraph Spacing setting in the base document would be propagated to the destination document if you set the Base Document parameter in the writer, but in the current project I don't want to use a base document.Is there a way to control the Paragraph Spacing without using Base Document?
Hi,I have attachments (pdf, jpg, doc, txt) which are stored in Access DB as BLOBs.I tried to use template from link below to extract files. Files have been exported but I cannot open any of them. Except of *txt or *doc where I have only a binary information, like "4D5343..."https://hub.safe.com/publishers/safe-lab/templates/read-blobs-from-a-database-and-write-out-files In AttributeFileWriter I used 'Binary (fme-binary)' encoding for JPGs and 'UTF-8' for txt, doc and pdf.For JPGs I tried also use a RasterReplacer (JPEG format) to decode a binary attribute, but all records were rejected.I'm using FME 2019.1 What am I missing? Thanks in advance!
Hello, new to FME here,I have to convert an RVT file to a point cloud file (x, y, z, r, g, b), but I haven't been able to extract color. I know this should be really simple, but I haven't been able to find much of anything on the RevitNativeReader. I saw a couple posts from 2019 that said the format was still in development, but I figure color has to be available somehow. I've been able to chuck the RVT features into a PointCloudCombiner and extract coordinates, but how would I get the color of each point? Do I need to manually map materials to colors? My output file says everything is black, so something must be going wrong.
Hi all,I have been having an issue in an evaluation copy of FME Server 2019.1, where I have been attempting to connect to an Active Directory instance but have been getting 'An error occurred while communicating with directory server (8)'.I've included some logging below (with domains and URIs removed>. I've not been able to find any resources which address this issue - is there any kind of configuration in FME Server that I've missed?Mon-20-Apr-2020 05:36:42.672 PM INFORM RequestHandler-Thread 401933 : Successful login by user admin Mon-20-Apr-2020 05:36:42.714 PM INFORM pool-5-thread-1 408078 : (Active Directory) Connecting using directory configuration "<domain>"... Mon-20-Apr-2020 05:36:42.720 PM INFORM pool-5-thread-1 408001 : (Active Directory) Connecting to an available server in ( <domain.name.local>:389 )... Mon-20-Apr-2020 05:36:42.727 PM INFORM pool-5-thread-1 408002 : (Active Directory) Successfully connec
I am trying to upload to our Open Data site powered by Socrata. I did enough research that I saw I would need to change the data to something Socrata would accept. But if I tell the writer it is a 'calendar_date' field, the schema is uploaded, but no records are created.
Hello, I have a dwg file in Reader with existing layers. I would like to know if you know how to match the Dwg file in Writer again. I use ShemaMapper this to have worked, however in output Writer I always have the same layers. Please help me if you know a solution.
Hello,I'm trying to merge a GeoTIFF orthopotography file and a CityGML file with into a single SketchUp file.With FME Workbench, I have visually the result I want in SketchUp: Do you have an idea of how to do this?Thank you in advance!
I'm in a serious dead end - whatever I do, I can't export files with FME that would show in Blender. I have tried to export both in OBJ and FBX, but they don't appear. I can import the file, but nothing shows. My original file is in WFS format and looks just like it is suppose to in the FME data inspector. When I create a writer it makes the files and they appear in the folders they are suppose to. However when I try to open them with Blender, Unity or with Autocad I can't see any buildings that I exported. I can just see the contollers, looks like empty game/3d object. Can someone share their workflow when importing to blender 2.8? Is there some tickbox or selection that I should use? Oh the same file OBJ file I have managed to open with 3D Builder, but the 3D builder asks me to locate every singe texture surface (over 20 in one element) separately. Does this give any hints what I'm doing wrong with OBJ conversion? I only used the 3d builder for testing and I nee
Workspace has just ESRI Geodatabase reader and SHAPE file writer (FME 2019 version) as I'm trying to write to a shapefile.In Shapefile writer, few attribute definition names are being shortened(truncated) but retains the same data type(as reader). Tried to rename those shortened attributes(in writer), but still it reverts to shortened version. Thereby causing difference in schema definition between reader and writer.This is the case for all different types of Attribute Definitions - Automatic, Manual, Copy from Reader..I think I've tried possible workarounds, but still same issue. Any thoughts please? Thanks!
Can anyone explain the purpose of the extra xsi_nil attributes that accompany a Web Feature Service (WFS) data source?For example, when I create a WFS reader in FME for the topp:states demo layer of a stock GeoServer, the resulting table schema in my FME workspace includes extra attributes - one for each native attribute - named <attribute>_xsi_nil. Here's a screenshot highlighting the fields in question:The data type for all of these xsi_nil fields is xml_boolean. This happens for all WFS layers I consume from a GeoServer.What is the purpose of these extra fields?And is there a way to configure the reader - or the GeoServer layer - to not cause these extra fields to be created?Thanks
Hi,I'm working on creating a workspace with ESRI Geodb reader(or Feature reader) and writer for creating zipped shapefiles (with dynamic naming convention to zipped shapefile like yyyymmddhh_export for every run on schedule). The idea here is to have 'yyyymmddhh' dynamically updated for filenames whenever it is created on a schedule.Finally upload those files to SFTP server on a schedule.I've tried this in FME 2019.2 with using FeatureWriter and ZipArchive transformers, but not quite there yet. Finally to upload to SFTP, not sure if FTPCaller is better option or Automation with 'Upload to SFTP' action is viable options. Please suggest if any ideas. Thanks!
Hi FME Community I have encountered an issue that I cannot find the solution for. I have a Reader -> Writer setup. Reading from a MSSQL db and writing to a Postgres. When reading the table and making a preview, everything looks as it should be. Then I run the transport and looking at the preview of the writer after a successfull termination of the task. And this is what I see Even though I've made a Scheme called 'dbo'. and the DB-connection setup is successfully connected. The table isn't to be found in the PG-db. I've encountered this a couple of times, especially when writing to databases, however I cannot find any consistency in when it will or won't populate the tables. I've tried to make a transformer inbetween, sometimes it works, sometimes it won't. Sincerely