Using readers and writers to extract and load data.
Recently active
Hi, Im working on worspace that already has a subscription connected to a Topic Message. So when a new file is added to a specific folder, a Topic Message with the file path is sent as JSON. Depending on the name of the filepath I use a TestFilter to determine which test I want to do. This works! However, after choosing the test, I want to retrieve the file and further test the data. As the filepath is knowned and stored, I tried using a FeatureReader to open it, but it doesn´t work... Does anyone have a suggestion on how this can be fixed?
Hi!I want to interpolate the terrain model (black line) down in the terrrain. Below shown in orange is 4 points with z value - 10, -5, -1 and -7. To keep the terrain formations similar I want to shift the whole surface/triangle down at certain points. So the new terrain model is not the RED lines, but the yellow lines. Any easy way to shift certain triangle- edges down and have all the connected triangles also "move down"? Not sure if there is a better way to explain this :)
I made a workspace to automatically download spatial information from an API using the HTTPCaller. The API replies with a .zip file, with .shp files within these .zip files. In this workspace I want to unzip the API’s response and import the shapefiles into my PostGIS database. I use the unzipper to unzip the shapefiles and the tester to filter the .shp files. I use the AttributeKeeper to filter all the unused attributes. Then I want to use the “windows_path” attribute as a data source for the FeatureReader (in this case its 23 different .shp files). My goal is to write each shapefile into my database dynamically using the fme_feature_type as Table Name. It has to be dynamical since I want to use this workspace to download different datasets from the API.My workspace doesn't work no matter what I try, and i don't know what I'm doing wrong.When I open the FeatureReader, it looks like this: When I run this it shows no attributes, since I still have to expose them all.I tried
If someone of you gives me a help, I will be eternally grateful. I'm just taking my first step and I'm alrady confused.
Hello everyoneI have a directory of PDF files and would like to select some of them for merging.Is there a way of merging a list of PDF files to a multi-page PDF file?Thanks.
Hello,I have a question regarding the PCD Writer. I am trying to write a .xyz point cloud (consisting of ~ 2 million points) to the PCD format. The point cloud represents a segment of a tunnel-like geometry. The output cloud has regular strokes of missing points. From certain angles the point cloud appears to be complete, but when orbitting around one can see these strokes of missing data. Imagine slicing bread and removing every other slice.Any ideas on what could cause this issue? Thanks!
Hi, I have a MapInfo tab file which contains around 390,000 features some of the features are not displayed in the Map window. After some investigation it appears that tab file contains some features that are ‘not objects’. I’ll call this ‘spurious data’. My current process to remove the spurious data is to use SQL Select and apply the condition 'not obj' (see screen shot below). This identifies around 14,000 features which can then be removed. I naively thought, as the spurious data apparently has ‘no object’ and cannot be displayed in MapInfo , FME would automatically identify these features in a similar way and somehow inform me, at which point I'd use the relevant transformer to remove them. Unfortunately in this case FME reads all the data as valid data and when I attach an inspector it displays all the features !?!Could someone please advise me on what transformer and / or parameter I need to use to stop features which are not objects to be removed?Screen shot below s
i can finish ifc to standard citygml ,but the conversion between ifc to citygml ade has some propblem ,such as " The 'lod4Geometry' citygml_lod_name trait is invalid for 'Stair' features, the geometry will be ignored, valid values are {lod4MultiSurface}." when i set the citygml_lod_name value is lod4MultiSurface,there has more error,how should i set the CityGMLGeometrySetter transformer?
HiI'm having problem connectiong to Cambridge dataset in smallworld.my FME version is 2009 and I'm using smallworld core 4.0 , also i have tested it on smallworld core 4.2 with the same error.The full log is attached.Best regards,Aramideh
I cannot get hyperlinks to work when using a dynamic excel writer using schema. This is how I hook the hyperlink up to the file_path field.Basically just creating a new attribute as suggested in the manual: file_path.hyperlinkI get no errors but there are no hyperlinks in the written excel-file.Does anyone have a workaround for this?
I would like to label all the cells in a DGN file.
Is there a way of creating a Workspace where I can have a selection list (user parameters) of Feature Type (GIS.WaterMain, GIS.CombinedMain) to read the Schema with attributes (is this dynamic of generic)?I've been able to create a Dynamic Reading but Schema is fixed to attributes in the Reader, Generic is only shows on field.
@shazad.tareen@gmail.com
Hi,I have a dgn with faces which I want to convert to fme surfaces to be able to put texture on them. FME reads these faces form dgn as polygons. I tried to convert these polygons to faces with the FaceReplacer but a lot of the polygons failed because they are not planar. If the FaceReplacer works I plan to use the GeometryCoercer to convert the faces to surfaces. Does anyone have an idea how to solve this problem?Thank you
FME 2019.0 build 19238 Desktop on Windows 10 and Server on Linux (CentOS 7)I'm trying to migrate a large dataset (tens millions records) from a Google BigQuery table into a PostgreSQL table, but the translation failed and this error was logged. Tried some times, the translation stops always with the same error.Translation FAILED with 1 error(s) and 0 warning(s) (11375000 feature(s) output)FME Session Duration: 59 minutes 9.9 seconds. (CPU: 1945.4s user, 53.5s system)END - ProcessID: 11848, peak process memory usage: 1027252 kB, current process memory usage: 843964 kBjava.lang.Exception: Google BigQuery: Error retrieving query results from job '****************:********.********' executed on table job_*_***************. Provider error: 401 Unauthorized{ "code" : 401, "errors" : [ { "domain" : "global", "location" : "Authorization", "locationType" : "header", "message" : "Invalid Credentials", "reason" : "authError" } ], "message" : "Request had invalid authentication
I am running FME Desktop 2018.1 in a Windows 10. My source data is in a SQLServer database (2016 R2) and the destination feature class is in an ESRI Enterprise Geodatabase (ver 10.5.1) also using SQLServer (2016 R2) as the backend database.The destination feature class is in a Enterprise spatial data warehouse, business decisions are make on the contents in the destination feature class.I have a FME Workspace that reads a table with latitude and longitude and writes to a feature class in a ESRI Enterprise Geodatabase.The writer is set to TABLE Exists: 'yes', TRUNCATE Table First: 'yes' It appears that FME:1) Truncates data in the destination feature class, then 2) Reads the source table, then 3) Writes to the destination feature class. Is this correct or have I misinterpreted the FME log files? If this is correct, then there can be considerable time when the destination feature class is empty or being updated. If FME does truncate the destination data first, then can I
I have created an Excel template to be used as backup for billing expenses. The invoice has a header with specific places for billing date range(start, end), project name, project number, and project manager. The final output has a separate sheet for each project number. I have added AttributeManager to get rid of all of the other attributes and only have the 5 I need to insert into my template - the itemization is added later. I have everything working except getting the aggregated attribute value (i.e. project name) added to the named cell. How do I specify where to write each attribute?Version 2018.1 Desktop
I have a list of code numbers and descriptions. i wish top converted them into an xml file that has parent child relationship. Can any suggest tutorial or content that is appropriate? Thanks Bhere is a screen shot of my listand here is what i need
I have a Workspace running in FME Desktop 2017.1.1.1 on a server. The Workspace has a SQL Server Spatial Reader in it. The SQL Server database is on a different server which is in the same domain. The Reader is using a named SQL Server connection which is publicly visible. The connection uses SQL Server credentials to connect to the database (not Windows authentication). The connection is not encrypted.The Workspace runs fine when I'm logged in. It works as a scheduled task when running as me. But when I try to run it as a scheduled task using a different user (we have a Domain service account for running scheduled tasks), it fails. I've tried the registry fix suggested here:https://knowledge.safe.com/articles/32356/scheduling-software-doesnt-recognize-fme-connectio.htmlHowever I still get "Invalid authorization specification" in the log file. Interestingly, since applying the above fix, I also get errors about encryption (see below). I don't know if this
Hi, I want to overlay labels from a FGDB point file on a raster. I'm using the Labeller transformer but my points never pass through. I've uploaded a workbench. Thanks for any help.B
Hello,Main Query: I need to connect to an InterBase DB and it's not listed in the "Reader Gallery".I have installed the ODBC driver(s) for it. Can someone please advise how to go about creating this database connection possibly using the FDS file import? What about making use of an ODBC connection? I can already connect to this database using the IBProvider driver on a different server using a different application. In this particular case though, I must use FME to connect to this table to build a test Corporate Notification Service.Secondary Queries:I also noticed that Ingress was not listed as a "Reader Format" by default. Do you have any plan for implementing this? Has anyone managed to use FirmStep APIs in FME? Thanks,Sameer
In some of my workbenches in FME Desktop I can have over 400k of rows to update to a ArcGIS hosted portal feature service. However the writer will fail somewhere between 400k to 450k, roughly after an hour of updating. I think this is the token from portal expiring, but can't find a way to change the expire duration in FME Desktop when it gets a token from portal. Is there a way around this?
I need to figure out a way of testing if a writer which is an SDE layer is verisioned or not. If it is versioned I do not want to truncate and fill if from the reader but if it is not versioned then I would want to truncate and fill.
I'm sure I'm overlooking something embarrassingly simple but after adding 2019.0 to my computer (currently also have 2018.1 installed to a different path) I cannot complete addition of a .csv reader. I'm able to preview the input files and they show the correct column names and content, but the 'ok' button is greyed out. . preventing me from ultimately adding the reader. Automatic and Manual attribute definition give the same result. I don't have this issue with the Excel reader. Thanks in advance - pulling my hair out!
I would like to color the cells of my first column ID_WS based on the text value contained in a cell UPDATE_STATUS which basically contains the fme_db_operation value.I tried to apply the styling with ExcellFormater but it doesn't seem to be working in the output. What am I doing wrong here ? Thank you !