Transformers

Using transformers to modify data.

  • 12,275 Posts
  • 49,276 Replies

12275 Posts

jacques_gitter
Contributor
jacques_gitterContributor
asked in Transformers

Obtain path to database connection file using pythoncaller

Hi, I’m working on a script which copies data from an Esri arcsde geodatabase to a postgis database. I need to know if the table in the arcsde geodatabase is versioned before I run the rest of the script, and I have a pythoncaller which runs something like this (simplified here):layername = "schema.layerName"connectionFile = "path_to_connection_file"arcpy.env.workspace = connectionFiledesc = arcpy.Describe(layername)isVer = desc.isversionedfeature.setAttribute("versioned",isVer) self.pyoutput(feature)Now, I don’t want to type in the path to the connection file manually, but rather obtain it by some function which can collect the data from the list of database connections I entered in the settings. Is this possible? I’ve looked into the FME Web Services API reference for such a function without any luck. Best regards,Jacques 👷🏼 

1
jacques_gitter
Contributor
17 days ago
mariaka
Contributor
mariakaContributor
asked in Transformers

XMLFragmenter/XMLXQuery*: only get root elements

Hi there,I don’t see the forest for the trees 🤔Input dataXML excerpt (source https://ifc43-docs.standards.buildingsmart.org/IFC/RELEASE/IFC4x3/HTML/annex-a-xsd.html):<?xml version='1.0' encoding='ASCII'?><xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ifc="https://standards.buildingsmart.org/IFC/RELEASE/IFC4/3/DEV/9d19c824" targetNamespace="https://standards.buildingsmart.org/IFC/RELEASE/IFC4/3/DEV/9d19c824" elementFormDefault="qualified" attributeFormDefault="unqualified"> <xs:element name="uos" type="ifc:uos" abstract="true"/> <xs:simpleType name="Seq-anyURI"> <xs:list itemType="xs:anyURI"/> </xs:simpleType> <xs:complexType name="uos" abstract="true"> <xs:sequence> <xs:element name="header" minOccurs="0"> <xs:complexType> <xs:sequence> <xs:element name="name" type="xs:string" minOccurs="0"/> <xs:element name="time_stamp" type="xs:dat

2
mariaka
Contributor
17 days ago
robertgilley
Contributor
robertgilleyContributor
asked in Transformers

I have been asked to replace a python program that logs into a website and download a text file with an FME solution. I was given the working python application and, using the HTTPCaller Transformer, along with the Creator Transformer.

Although the Python can connect with the information provide, the FME tries to connect but is not able to. This is simply downloading a text file from the site once the connection is made and storing the text file locally.The goal is to modify the process that currently uses three software packages into one FME solution. Details on the HTTPCaller in the attachment

14
robertgilley
Contributor
17 days ago
joeri
Contributor
joeriContributor
asked in Transformers

Remove duplicates from database table (keeping all unique record)

I have a database table in which a number of the records are duplicated. I'd like to remove the duplicates but like to keep 1 record (so similar to the duplicateFilter unique port).I have no unique identifier in the table so if I match a record (based on a combination of attributes) I'll remove all of them from the table, I need to add 1 of them again afterwards.  Currently I use the following approach:duplicateFilter to find the duplicatesremove the records from the table using fme_db_operation = DELETE (this re re-insert the unduplicated record Any suggestions to improve this methodology?

6
fme_can_do_it
Supporter
17 days ago
totree30
Contributor
totree30Contributor
asked in Transformers

IFC to CITYGML, how to aggregate all the featuers?

Hello! I would like to convert a IFC building into a CityGML building, I created all the components of buildings(wall, floor, roof, etc.). But I don’t know how to merge all these components into one CityGML building. I would like to create a building with appropriate CityGML hierarchy. Could you help me?  

2
kailinatsafe
Safer
17 days ago
william.walker
Contributor
william.walkerContributor
asked in Transformers

FME multipatch - Shading and lighting in ArcGIS Pro

Hi,I have a multipatch containing buildings in a file based geodatabase (.gdb). I want to be able to replace the geometries and attributes of certain buildings by updating them with buildings from another multipatch based on a unique ID. This is successful but when I load the multipatch into ArcGIS Pro the updated buildings appear grey instead of white. I don't think it is a problem with the appearence stored within the multipatch (I have tried GeometryColorSetter) but rather that ArcGIS Pro doesn't recognize the difference between walls and roofs in the objects written by FME.  When lighting is not enabled all objects appear white  When I enable lighting the object in the middle (written by FME) has grey roof instead of grey walls (shadow) and a white roof as the other buildings have.   If I edit a multipatch created by FME in ArcGIS Pro the roof becomes white again. Has anyone encountered this or have an idea for a soultion?

4
joycearabian
Participant
17 days ago
fbrws
Contributor
fbrwsContributor
asked in Transformers

Complicated JSON key-values as import for a database table

I want to transform this JSON for import in a database what would be the best practice when using the standard transformers?{"keys":["day","domain","subject_identifier","kpi_type_code","kpi_count","kpi_value_min","kpi_value_max","kpi_value_avg","kpi_value_last","kpi_alert_condition_lower_limit_last","kpi_alert_condition_upper_limit_last","kpi_value_extremum_ok","kpi_value_avg_ok"],"values":[["20240407","production","7ab20028","RA_MAP_FREQUENCY",1256,"24,0000","24,0000","24,0000","24,0000","0,9000","50,0000",1,1],["20240407","production","7ab20028","RA_EXCEPTIONAL_CONDITION_PERCENTAGE",1440,"5,9418","5,9418","5,9418","5,9418",null,"25,0000",1,1],]}

3
danilo_fme
Evangelist
18 days ago
tanguymorgane
Contributor
tanguymorganeContributor
asked in Transformers

I have the FME 2021 Workbench version. I have an API protected by an SSL certificate (.crt, .csr, .key, and .p12). How do I read my .p12 certificate and connect it to my API https://**** in FME? Is it possible in HTTPCaller?

I have the FME 2021 Workbench version. I have an API protected by an SSL certificate (.crt, .csr, .key, and .p12). How do I read my .p12 certificate and connect it to my API https://**** in FME? Is it possible in HTTPCaller?In my directory C:\Program Files\OpenSSL-Win64\bin\partner_key, I have my four files, the .crt, the .csr, the .key, and the .p12 (with a password for the .p12 file). On the other hand, I have my API https://api-pp-iop.******.com/******/******. I want to extract all the data from this API. It won't work if I can't associate the .p12 certificate with it.

4
danminneyatsaf
Safer
18 days ago
gjaguilar
Contributor
gjaguilarContributor
asked in Transformers

There a way to debug the execution of the script in FME?

In a process in my company we use a Python script (PythonCaller) to call a download API, we need to modify it, but we get some errors that I don't get when running the script from Python on the Windows command line.Is there a way to debug the execution of the script in FME?

4
gjaguilar
Contributor
19 days ago
B
bdel
asked in Transformers

How to select a unique file in the dialog box of a parameter of a Spatial ETL Tool?

Hi everyone, I have finished a FME Workbench and I wish to share it throughout my organization via an ArcGIS Pro Tool (Spatial ETL Tool). To create an ESRI Spatial ETL Tool, I followed this tutorial (Create an Esri spatial ETL tool—ArcGIS Pro | Documentation) and this specific section: “Reference an existing FMW file to an ETL toolbox”.Through the user tool interface (as in the Geoprocessing pane), I would like for a user to be able to:-         Define the output file name and the path where the file location.==> Those two parameters work.-         Select the unique input file in a Geotiff format (extension *.tiff)==> Doesn’t workTo do that in the FME Workbench (ArcGIS Data Interop. Workbench) --> Navigator (left panel) --> User parameters --> Right click --> Manage User parameter --> Insert new user parameter of type “File/URL” (trough the green cross) --> Click on the parameter to opens his settings à In File/URL Configuration :Items to Select : FilesFile conf

4
weihong
Contributor
19 days ago
ecx
Contributor
ecxContributor
asked in Transformers

How to process items grouped by a field, for multiple transformers.

I have 100 items of data, which are grouped together in various sub groups using a field 'groupNo',I want to process each set of groupNos individually, I need to select a group, and do the following processes for each individual group: Take one item with status 'original' which has data and duplicate that data to the 'new' items in that group. How would I do this using the groupNo identifier? An example input output data set: Input:OID, Status, Data, Group1, Original, XYZ, 12, New, <null>, 13, Original, ABC, 24, New, <null>, 25, New, <null>, 2 Output:OID, Status, Data, Group1, Original, XYZ, 12, New, XYZ, 13, Original, ABC, 24, New, ABC, 25, New, ABC, 2  

1
davidlee
Supporter
19 days ago
ecx
Contributor
ecxContributor
asked in Transformers

SDE reader adds item to Navigator panel but not main editing window

Hello, I’ve run into a strange issue. I am accessing a polygon feature class via a SDE connection. Whenever I add the reader it does not appear on the workbench, it does appear in the navigator panel, but not on the actual main workbench menu, where the workbench is visually developed.item appears in navigator panel, but no where else. I can run other readers from other .sde connections but there is something wrong with this one. I can access/read the data via the same SDE connection via ArcPro. 

3
ecx
Contributor
19 days ago
egge
Contributor
eggeContributor
asked in Transformers

Write hyperlinks to Word document with pyhon docx in PythonCaller

Hi, I would like to share a solution to an issue I had, just to hear what you think about it. The challenge is: to create hyperlinks in a Word document.FME allows you, with the use of MSWordStyler transformers, to create a Word document in a very versatile and easy manner. These transformers, however, do not allow you to insert hyperlinks into the doc.Please see the attached zip file hyperlinks-to-word-python-docx.zip for a workflow where the document is created using MSWordStylers, followed by some post-processing with a PythonCaller to insert the hyperlinks: POIs with a link to Google Maps, in this case.No, I did not write all of the Python script myself, hell, no! I used some existing bits and pieces and tweaked them a little bit to get the job done.The add_hyperlink function was found on github and the docx_find_replace_text one on stackoverflow and I only modified them slightly to work together to replace my search text, the POI name, with a clickable hyperlink. More than 150 line

5
egge
Contributor
22 days ago
egge
Contributor
eggeContributor
asked in Transformers

Add pictures to table in Word doc using Python docx in PythonCaller

Recently I have been playing/fooling/struggling around with the creation and modification of Word documents using a combination of MSWordStylers and the Python docx library, called from a PythonCaller. A very instructive journey :-)To make sure I don't forget immediately everything I've learned, I have decided to summarize my findings in a template. And because a lot of input has come from this community I will share the results here. Please see the zip file attached: images-to-table-python-docx.zipIt starts by reading Parcel information from a database (gpkg) and it creates a summary table in a word document, including a little image with the parcel plotted on an aerial image.Inspiration has been taken from these to discussions:How to pull a raster from an ESRI basemap to use within FME workbench, especially the answer given by @nielsgerrits​  MSWordStyler how to include picture in table? with the Python example by @ebygomm​ With some slight modifications I have integrated these two e

4
egge
Contributor
22 days ago
ana.villa
Participant
ana.villaParticipant
asked in Transformers

Error 400 writing to AGOL, character problems

Hello... I’m new here, and I have a basic question. I’m trying to update a hosted feature layer in AGOL, but I have a 400 error about HTML. I check my attribute table, and I have one attribute with notes (string), and the character that I have a problem with is <, but I cannot change it because I need the information. So, I cannot use the text encoder or change the character. Can anyone give me an idea of what I can do?  

2
frro
Contributor
22 days ago
soly
Contributor
solyContributor
asked in Transformers

how to move the rectangle to right side ?

Hello I would like   to move the rectangle in the direction of  right side ?fme 2021thanks in advance 

2
virtualcitymatt
Influencer
22 days ago
nico42
Participant
nico42Participant
asked in Transformers

problem of date with testfilter and attributcreator

Hello everybody.I'm stuck I think over a trifle.I have a dataset with dates like "1970".I want to create a column allowing me to have date classes in it (for example [1970-1980[ ).It seems to me that I need to go through a testfilter and a creatorattribute but what I do inside doesn't work.Could someone give me a hand on the operation so that I understand how it works?Thanking you very much in advance

3
nico42
Participant
22 days ago
M
markb
asked in Transformers

Is there a file size limit when using the AttributeFileReader?

I have it reading some JSON files and some go through and others dont with an error of "INVALID_FILE_INACCESSIBLE". The file sizes range from 500Mb to 8Gb in size.

4
DanAtSafe
Safer
22 days ago
boubcher
Contributor
boubcherContributor
asked in Transformers

Sampling with spatial relation condition

hello There we  number of hexagon feature and we want to sample them , but the sampled features should not have any spatial relation to other one , in other terms should not touch the other sampled hexagon Thanks  

2
boubcher
Contributor
22 days ago
lneidhardt
Contributor
lneidhardtContributor
asked in Transformers

API with separate arrays for columns/rows

Hi there,Relatively new to FME and very new to JSON and API’s. I’m working with the Smartsheet API and I’m stuck on how to the get the data set up properly, I’ve gone down many of the JSON Transformer rabbit holes and I can’t find a similar example to how this data is set up. When I use the JSON Fragmenter, I get columns and rows returned as separate arrays:  The columns are set up like this: And the rows/cells:I’ve gotten as far as exposing the attributes for each array separately, but I don’t know how to get cells back to their associated rows/columns. And I’m sure there’s a better way than I’m doing it :)

4
danilo_fme
Evangelist
23 days ago
evagardner
Contributor
evagardner
asked in Transformers

Swagger API using HTTPCaller

I’m using a HTTPcaller connect to a swagger API that ONLY receives raw JSON data in the Upload Body or uploading a file (one block of JSON data per file). I can’t pass through any parameters from a attributemanager. When I try to pass through a parameter in the Upload body of the HTTPcaller, I get this error message:Attribute(string: UTF-8)          : `_response_body' has value `{"errors":[{"code":400,"message":"A non-empty request body is required.","title":""},{"code":400,"message":"The dto field is required.","title":"dto"}]}'When I try to pass through {“active”: true, “firstName”: “@Value(firstName)”}, I get this error: Attribute(string: UTF-8)          : `_response_body' has value `{"errors":[{"code":400,"message":"The dto field is required.","title":"dto"},{"code":400,"message":"',' is an invalid start of a value. Path: $.active | LineNumber: 1 | BytePositionInLine: 12.","title":"$.active"}]}'I need to use the POST method to create an employee but I can only create one employee p

2
evagardner
Contributor
23 days ago
alexortziply
Contributor
alexortziplyContributor
asked in Transformers

Bufferer Generating OGC Invalid Geometry

I have a set of line data that I wish to buffer for a client.The requirements of their GIS system are such that the shapes must not be 2D reentrant, lack donut holes, and be OGC Simple + OGC 1.2 compliant.I buffered the lines to .0003 ground units (coordinate system is in degrees)Input feature qty is 9287.The validator only passes 8175 of them.Every failure reported from the listConcatenator is “Fails OGC Compliant,Fails OGC Simple”.Is there something that I’m doing wrong?

5
alexortziply
Contributor
23 days ago
agajo
Contributor
agajoContributor
asked in Transformers

RasterConvolver incorrect standard deviation calculations

Hi there :), I’ve been using RasterConvolver in FME Form 2023.0.1 to calculate the standard deviation with a round kernel, so some weights were equal to 0. The results were always incorrect, so to understand what’s happening I simplified the problem and used a raster with all values = 1 and one weight in kernel (3x3) = 0.From what I understand, the formulas used in RasterConvolver for calculating the weighted mean and weighted SD are a bit off. Here is what I deducted:The weighted mean is divided by a number of values (kernel cells) instead of by the sum of them. The weighted SD: in the sum equation the weight is multiplied by a cell value instead of by a square of the difference between a cell value and a weighted mean AND the sum equation is divided by a number of values instead of a sum of weights.This may seem pretty convoluted (pun intended :)), so I have attached a pdf with a visual explanation. I additionally attach a fmwt file with two examples of RasterConvolver misscalculatio

3
mark2atsafe
Safer
24 days ago
yagami9
Contributor
yagami9Contributor
asked in Transformers

reading dynamic AutoCAD blocks properly

Hi everyone ,I have dynamic doors in my dwg AutoCAD file :If I click on the small arrow the door changes direction , and the big arrow changes the size of the door I want to read on fme the default position and size of the door , just like a normal static door. But here is how the DWG Reader reads it :   Any Insight is much appreciated !-Yagami

2
yagami9
Contributor
24 days ago
akoure_terracon
Contributor
akoure_terraconContributor
asked in Transformers

JSON Reader Folder - Get File name

I’m using the JSON reader to read in a folder of JSON files. The reader is working how I want it to, however I would like to get the original file path/name as an attribute on each feature.  Is there an easy way to do this? I’m hoping that I am missing something silly.   

3
danilo_fme
Evangelist
24 days ago

Badge winners

  • Enchanted by Attributes
    nordpilhas earned the badge Enchanted by Attributes
  • I Treasure Lists
    nordpilhas earned the badge I Treasure Lists
  • Teacher (Bronze)
    tvahas earned the badge Teacher (Bronze)
  • Teacher (Grey)
    tvahas earned the badge Teacher (Grey)
  • Student (Grey)
    anteboy65has earned the badge Student (Grey)
Show all badges
28,731
Posts
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.