Using readers and writers to extract and load data.
Recently active
Hi, I am using S3ObjectLister and S3Downloader to connect to a bucket, download data and process it. I have two bucket setup in two different regions (ap-southeast-2) and ca-central-1. When I use settings for first region it works fine but when I change settings for region 2 i.e ca-central-1 (which is not available in transfomer's region drop down) S3ObjectLister rejects to connect and gives me follow errorsfme_rejection_code: ERROR_CLIENTfme_rejection_message: fme_rejection_message (encoded: UTF-8): Not able to determine region for ca-central-1.Please upgrade to a newer version of the SDKI am not sure how to access bucket setup for ca-central-1. I have tested bucket setting and it is responding well and tried using different regions with with bucket setup for ca-central-1 I get different error code 301.
I'm reading an XML with ticket details that has xsi:nil=true in the tag and it adds the field name with .nilAlso,there is a tag with xlmns andIm not sure if I have to use a List Exploder to further get its attributes.I'm a novice FME user and have no prior experience with xml eitherI'm looking for direction on how to resolve the field names to be the tag text and populate its values in ESRI Gdb formatAny help will be much appreciated
Hi! I have a workspace that converts LAS into Recap using a pointcloudlasclassifier before writing out to Recap. This workspace runs 100% flawless on desktop. When publishing to FME Server however, the workspace always fails the first run after logging into a new session in the browser. When running the workspace the second time (and following runs) without editing any parameter it runs successfully. Any ideas what can cause this issue? It seems that it is the rcs-files (which are written to a subfolder) that is causing the workspace to fail.
Hi,I use FeatureWriters in FME Workbench 2017 for inserting datasets in Postgres.But when there are errors in my input data, like a string instead a boolean value, the row is not written in Postgres but FME doesn't detect the error, the row is counted like a correct insert and I have the message "Translation was successfull" at the end. Of course, I could use AttributeValidators to detect such errors but I have a lot of fields and I wouldd like to avoid to inspect each field. Thanks in advance for any help,Christian
Hi,I’m having problems with SDE writer with a “Versioned Edit Session” transaction type in my workspace. The workspace in question is initiated by Workflow Manager. A child version is created in our flow and the name of this version is passed along as parameter value. I’m using a writer, with three feature types, with the following settings: What do I expect:The data is written to the child version. If writing to the tables succeeded, then the child version is reconciled and posted to the parent version. After posting, the child version is deleted. When the writing fails, the child version is not posted to the parent and is automatically deleted. What do I have at the moment:The data is written to the child version. If writing to the tables succeeded, then the child version is reconciled and posted to the parent version. After posting, the child version is not deleted. When the writing fails, the child version is not posted to the parent and is not automatically deleted. The followin
Is there a way process OSM data so that I organise the most important tags into separate attributes while keeping all other key/value pairs e.g. by dumping them into another attribute called "othertags"?For example; an entry in my Highway table might look like this:highway trunk lanes 2 oneway yes width 12 othertags "surface"=>"asphalt", "name"=>"Ermine Street", "maxspeed"=>"70", "lit"=>"no", "ref"=>"A1", "name:en"=>"Ermine Street", "maxwieght"=>"..... Basically I don't want to lose any info, but I don't want to expose all tags to potentially several tens of mostly useless attributes.
Hello,I'm migrating some shapefiles using FME to a PostgreSQL/PostGIS database. Is there any way to monitor all of the SQL commands that have been sent to it during the translation ? Thank you.
Hi all.A problem that makes me crazy :-(To collect data form Oracle, I use an SqlCreator with a parametric query.When I run the same query in SqlDeveloper, I takes the correct number of records. Usually a single record.When I do the same in FME, the record is duplicated.The strange thing is that if the correct result is one record, it is duplicated. So two records.If the correct result is three records, the result is four. One duplicated and the other fine.The production landscape where the problem occurs has two FME servers with distributed engines.In the second landscape (testing), the problem doesn't happen.Very strange...Thanks for your supportRoberto
Hello everybody, I have to solve urgently problem with generating of SVG files. I still cant find way, how to define title of SVG file and also, how to insert my attribute after <g id> tag, see pic below? Please, do You have any tip, how to figure out this problem?Thank You so much!Lubo
I have ifctoCityGml template (no idea how it is working) and its working fine and convert my ifc to citygml but the only problem i am facing is that it is not exporting roof while it successfully exported WallSurface, FloorSurface and window. And here is its transformation snap:As you can see there is no output from FeatureMerger_2 Transformer. I guess Merged, is not producing any output.
I have an excel file containing 100,000 items placed in 700 boxes. The file has two columns, one column is the item and the other is the corresponding box. I would like to get them rearranged as a file with 700 columns and each column (box) has all it's corresponding items listed underneath. Attached is an example: Excel's Pivot table and FME's AttributePivoter are asking me to group the items by columns AND rows. But I don't have rows to group the items with. I just wanna group the items by columns.
I like to have a simple shapefile been written to CityGML.A simple task, I assumed, but it doesn't work out the way I want... at least, not in the end.The color and the transparency don't come with the CityGML file. What am I doing wrong or what am I missing here?Attached the Workspace and the data.
im trying to know if a feature is empty or not, iv tried a counter box butit does'nt do the job (but the counter receive no input).
Hello everyone, I'm getting a Python error when trying to add a ArcGIS Online reader. I noticed this today, but can't say for sure when exactly it happened. I'm getting this error in the console log:Creating reader for format: Esri ArcGIS Online (AGOL) Feature ServiceTrying to find a PYTHON plugin for reader named `ARCGISONLINEFEATURES'Loaded Python module `arcgisonlinefeatures' from file `C:\Program Files\FME\plugins/python27\arcgisonlinefeatures\__init__.pyc'Python Exception <ImportError>: No module named arcgisonlineCommonCould not create Python Reader `ARCGISONLINEFEATURES'A fatal error has occurred. Check the logfile above for detailsFailed to open the dataset '<Unused>'DATASET INFO:  DATASET: <Unused>  NUMBER OF PARAMETERS: 3    RETRIEVE_ALL_FS_NAMES = Yes    NC = envita<space>gis<space>Esri<space>ArcGIS<space>Online    _MERGE_SCHEMAS = YESSo it's complaining about the arcgisonlineCommon module, but I don't
Hi,I'm writing to a MapInfo extended tab, and noticed there is a delete operation on the tables. I have tested this out, and have got it to work, however the tables aren't compressing after and the deleted records are staying in the table albeit flagged for deletion. A pack of the table solves this, but cant it happen on closure of the writer?Cheers,James
Hello everyone reading this. I am new to gml conversion, so I seek help here.I am trying to read gml files provided with xsd schemas, validate them, insert those data into PostGIS database, do some database processing, then exporting data from database into gml files using same xsd schema (meaning output file should be more or less copy of input gml file, with less or more features)Importing gml files into PostGIS database is going fine, however I have several problems with exporting data, as the files differ in several places and I am unable to figure out how to duplicate the structure.Unfortunately I cannot provide xsd file and have to use modified gml, sorry :( I have several problems:gml file looks like this (change extension, website did not allow me to upload gml):1) FeatureCollection attributes.snippet of xsd:<xs:element name="FeatureCollection" type="eczr:FeatureCollectionType" substitutionGroup="gml:AbstractGML"/> <xs:complexType name="FeatureCollectionType">
The reader in my workspace is an ESRI shapefile. I am clipping this to a buffer, changing coordinate system (SnakeGrid) and writing to a MicroStation .dgn. I am copying all the attributes from the .shp and writing as tags. Can I do this for several .shp's at once and it only carry the necessary attributes for each .shp? When I input '*.shp' to read all .shp in a folder, it outputs then individually but with the attributes of all the input shapefiles.
i had 2 question ,1, how to create an empty polygon (i v tried with creator function but with no result)2, how to verify if a polygon had an entity or is empty ?
Is there any way to write merged cells in excel:
Hi I've a very funny error. I've never had it before. I created a new workspace and translate some data into an MAPINFO-Output. But I get this message in the log: ERROR |Destination Feature Type Routing Correlator(RoutingFactory): Failed to open FME feature store file `' for write -- ensure that the folder exists INFORM|=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- INFORM|Feature output statistics for `FFS' writer using keyword `W_2': STATS |=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- STATS | Features Written STATS |=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- STATS |============================================================================== STATS |Total Features Written 0 STATS |=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- STATS |Stored 0 feature(s) to FME feature store file `' ERROR |Failed to open FME feature store file
Hello. I am trying to use the Esri ArcGIS Portal Feature Service Reader/Writer in FME Desktop 2018.1 and am getting a Python KeyError exception when trying to authenticate the login. I am using "Generate Token" as the authentication type, and trying to connect to an instance of Portal for ArcGIS (10.6.1).Here's the relevant excerpt from the log:|INFORM|ArcGIS Portal Feature Service Writer: Logging into '[host url]' as user '[username]'|ERROR |Python Exception <KeyError>: u'token'|ERROR |A fatal error has occurred. Check the logfile above for details|ERROR |... Last line repeated 2 times ...The strange thing is that this workspace ran successfully a few days ago.Any input would be appreciated. Thanks!
. The data contains englands ceremonies as polygons. After converting the shp according to this demo https://knowledge.safe.com/articles/775/inspire-gml-basic-writing-example.html only a few polygons get the geometry attributes.fme_geometry (string) fme_aggregatefme_type (string) fme_areaxml_type (string) xml_areaMost of the polygons get these kind of geometry attributesfme_geometry (string) fme_undefinedfme_type (string) fme_no_geomxml_type (string) xml_no_geom--------------------------------------------This is my project. Kind regardsNico
I have a GoogleDriveConnector connected to my account so that it lists all of the pdfs from a folder. As soon as there are more than 100 files in the folder, GoogleDriveConnector just keeps running forever (it got to 32000 outputs before I stopped it)With 91 files it works correctly :After uploading 10 more files the transformer gets to 741 outputs and would keep increasing if not stopped. Is there a way to fix this issue?I've tested on different files/accounts and nothing seems to help.
I have windows, walls, doors etc object in IFC when I view it in FXKViewer. But after converting the IFC into CityGML all the objects are appearing with genobj:GenericCityObject name. What i am doing wrong? I have just generate CItyGML from IFC.
Hi, I was wondering of it is possible to use an if statement in the Pointcloudexpressionevaluator to calculate values based on values of other components. I created a pointcloud with different z components (for example z_2018_01 and z_2018_02), these z values represent measures in different months in 2018 . There are also components that represent the year and month (YM_2018_01 filled with value 20801 and YM_2018_02 filled with value 201802) There are also points that have only one measure value (or in period 1 or in period 2), see table below. From these z components I want to calculate a z_actual (most recent z value). Is there a way to use the Pointcloudexpressionevaluator? An expression like : “If JM_2018_02 > JM_2018_01 then z_2018_02 else z_2018_01” will be useful but I don’t know of this is possible, and if, how to build such an expression in the Pointcloudexpressionevaluator. Can anyone help me? I prefer the pointcloud transformers because it gives a very good performance, (