Using transformers to modify data.
Recently active
Hello, I am configuring FME to receive file uploaded from website through API - POST method. We want the api to directly pass the uploaded file from website to FME for processing. Note that the file is not saved into any DB.The issue I am facing is the the configuration of HTTPCaller Transformer to receive file datatype. Anyone can help suggest on this? Would be very helpful if I can get any advice to configure it the correct way since I’m stuck with this. The logs: The api test through postman has give success response to upload file.The file type I am testing to upload is excel file. Later will need to test with CSV & Shapefile.
Hi all, I am currently using a Dissolver with Attribute Accumulation > Accumulation Mode set to ‘Use Attributes From One Feature’.How does FME decide which feature's attributes are used? I assumed it would be the first one to enter the transformer, but I couldn't find anything on this in the documentation. Kind regards,Joep
The polygons below have corners that are 90 roughly degrees. I would like to split them at the corners so that I get four lines from each polygon for further processing.The two areas below are only examples, but the rest of the data have similar elongated, bent shapes. I tried finding the corners by intersecting against a bounding box but the result was not very useful because of the irregularity of the shapes. My guess is that I would get the best result by measuring where the vertex take a 80-100 degree turn. Any suggestions on how to achieve this? Or other clever ways to breake the polygons appart?
Why do I get the message: The following potential problems were detected in this workspace:Transformers are used that are not installed: 'AttributeExposer (version 2)', 'Clipper (version 16)' The install/machine that created this workbench, and the install/machine I’m trying to run it on are different, is it a version mismatch? How can I fix it?
Hi all,I have a set of JSON files which return an error upon trying to read them in - that it encounters an unexpected character "\\". If i open the JSON in a text editor and remove all the \\ characters it reads successfully in FME then.As i have a batch of 300 JSON files i cant do this manually. I have tried reading in each one as a text file and using a StringReplacer but it is incredibly slow.Has anyone got any suggestions or solutions?Kind regards,Jack
I need something like the AttributeManager but it needs to work only on a limited set of attributes. What I need is a transformer that sets values (conditionally) to a handful of attributes. I need the ability to copy paste this box to various places in a workspace or to other workspaces. I’ve been using AttributeManager but the problem is it gets overwhelmed with other attributes and the transformer changes based on the input stream. I need something that’s sort of shielded from what’s in the pipe. These limited set of attributes are common to various workspaces or workflows (eg, datemodified, lastuser, datecreated etc) and are always guaranteed to exist. So naturally I need them handled in a simple, separate transformer that can be reused. Is there any way to achieve this? Thanks
HiI have a spreadsheet (see table below) with a start and end point coordinates for different road sections and I'm trying to extract the road segments between the start and end points from another dataset containing my road geometry. Unfortunately the start and end points do not always lie on the road so I would need to snap it to the closest road. The road directions (road ways) are stored as separate features in the dataset with a common road id.Section IdStart LatStart LongEnd LatEnd LongA-33.938..., 151.151.-33.935...151.131...B-33.949...151.138..-33.931...151.154...C------------E.g. For Section B, I want to extract the two segments in green from the underlying blue road. The eastbound (EB) and westbound (WB) road features have a common road id of 1.I tried using two vertex creators, a point on line over layer, a topology builder and feature merger, but I haven't had too much success. Any ideas on how this could be achieved?
How to buffer a line with different distances on the left and right side?The question is about road centrelines, where buffer needs to have different distances from centre line on left and right side.
I had a workspace with 13 user parameters. When renaming them all, I accidentally erased one's identifier, and it duplicated it and the new one is bugged. When I try to save, the workspace crashes. After publishing it on the server, trying to run it shows not a single user parameter. I can not delete it, and it is linked to every single transformer of the workspace (copy pasting any transformer from the bugged workspace to a new one imports the bugged user parameter and therefore corrupts the workspace).
Hello,I have previously in Form 2023 added custom paths in the “Edit Data Download Properties → Include Data from Writers” when preparing to upload to Flow.The issue I now have in Form 2024.1 is that after I have published the workspace to Flow or if I reopen the workspace my custom paths are not saved within the workspace, or if it’s saved to Flow as it was in Form 2023. This makes the Republish option button not useful as I can see as I have to rewrite the custom paths. Or have I might missed something? To be clear I belive this only counts for custom paths and the benifit with custom paths is that I can customize how files are organised within the .zipped file from the Data download service.
Hi, this must be easy but can't seem to find it...I want to convert duration from seconds to hours, minutes, seconds. So 337759 seconds should be 93h49m19s.I thought the DateTimeConverter should help me, but can't seem to get it right:results in 21h49m19s...I could calculate it myself, but I guess I'm overlooking a setting in this transformer?
After deleting some files using the os.remove instruction within a PythonCaller the translation ends successfully but it gives these errorsWorker 215652 > Failed to apply the constraints feature to the readerWorker 215652 > Failed to apply the constraints feature to the readerWorker 215652 > Failed to apply the constraints feature to the readerWorker 215652 > Failed to apply the constraints feature to the readerWorker 215652 > Failed to read features from dataset 'C:/Users/LOCA1A~1/Temp/225/wb-cache-CLMS_Download_20240607_DESKTOP_2-igKtLc/LEGACY_DuplicateFilter_2 2 194 fo 0 UNIQUE 2 7b50d2c16b579db61a32ffc86b48776319f4311b.ffs' using the 'FFS' readerI have searched the web for the origin of the error but I can't find anything that makes senseI would greatly appreciate any help
Hello,I have several lines in my data base which are self overlapping. On the the picture you see the vertices and the order when drawn. Vertices 3 and 4 have been drawn before vertex 5 which makes the line self-overlapping or self-intersecting. How can I detect lines in my database which have that kind of problem ? I tried the geometry validator but I noticed that it detects only some of them. So it is not fully reliable in my case. Is there any other transformer ? I thought maybe of calculating the bearing as the change of direction could indicate that problem as well. Thanks for your help. (btw I don’t want any transformer to automatically correct them, I only want to detect them)
This is either going to be a very stupid question or a very silly bug.I want to check if MyUserParameter has a value, so I put a Creator and a Tester on my canvas checking whether MyUserParameter does have a value. It tells me it doesn’t, whilst it clearly does! Hmm, what’s going on?Am I being dumb? Why can I not test a User Parameter?I can get it by using a ParameterFetcher, so it’s not a big issue.We recently upgraded from v2023.1 to 2024.1, and I thought this used to work, but I could well be mistaken.EDIT: It does work if I check if MyUserParameter is of type string or when using RegEx. It sounds like the ‘has a value’ and ‘is null / missing / empty’ doesn’t work with user parameters.
I have several rasters with varying Z values and thousands of parcels. I’d like to extract the lowest z value from multiple rasters by parcel. Any help is appreciated!
Hi,The below python code works in ArcGIS Pro:But returns the below error in FME:What would be the ‘FeatureProcessor’ python symbol?Regards
I am writing a workflow that refreshes data within a hosted AGOL feature service.To enable continuity of service, I have 2 hosted feature services for the same layer. eg. Playgrounds_A and Playgrounds_B. There is a view defined called Playgrounds_View whose source is either Playgrounds_A or Playgrounds_B My workspace plan is to refresh the contents of the ‘non-active’ feature service, and then set the Playgrounds_View to point to it.I have managed this previously with Python for ArcGIS scripts using commands like item.add_relationship() and item.delete_relationship()How can I recreate this functionality in FME workbench ?
HiI would like to extract some info from a website that are inside a p tag inside a div class = paragraphs_item__body_paragraph_bundlethe website is https://www.padovanet.it/notizia/20221025/strade-chiuse See picture belowthx for support Francesco
I have a very basic dynamic workbench that is reading data from SQL Server. Before writing data to a postgis database it is changing an attriubte name with SubstringExtractor. But it is extremely slow doing so after almost 3 hours 150 000 objects has been read by the reader. Typically 100 000’s of objects is handled in matters of minutes. It is reading about 25 different feature classes I need to migrate daily.I cant wait hours for the workbench to be complete. 2024-09-27 15:11:02|6056.7| 0.8|INFORM|Reading source feature # 470002024-09-27 15:11:53|6106.9| 50.2|INFORM|Reading source feature # 47500It takes 50 seconds to read 500 features? What is wrong? Why is the process so slow? I cannot understand where the bottle neck is.
Hi! I’m new to FME. I’m trying to replace an existing feature class with data from a SQL query. I got it to work by using a Reader that specifies the existing table, SQLCreator then a Writer. When I add the writer, it’s using the “Copy from reader” option. Is there a way to use the FeatureWriter transformer and specify an existing feature class instead of using a Writer?
Hello,For the logged known issue in FME Form “FMEENGINE-82549” with “Excel Writer: Field Names row not written” the solution described is to Set the Row Height to 0 on the Feature Type, as discussed here. But what are the steps to do that? Is it to simply just use transformer “ExcelStyler” and use setttings for “apply style to”? Thanks!
Hello everyoneI need to run a test process that reads a file in netCDF format and clips it using a Clipper. I am using a shapefile for clipping, but the Clipper is giving me the following error:Clipper_<Rejected> (TeeFactory): Clipper_<Rejected>: Termination Message: 'Clipper output a <Rejected> feature. To continue translation when features are rejected, change 'Workspace Parameters' > Translation > 'Rejected Feature Handling' to 'Continue Translation'' Here is an example of the process I'm trying to test.I want to write two output files, one in NetCDF format and the other in Geotiff format.Afterwards, I need to open one of the result files and the original netCDF file in QGIS to check the values.The goal of this test is to verify why, in another process, the clipped result is altering some values related to the original global input file (netCDF). Thank you in advance for any assistance you can provide.
I have data stored in a SQL server database in a native geomtry type format. I would need to migrate this data to a postgis database and store the data using simple geometry (point, lines and polygons instead of the generic geometry datatype).I have seen options how to modify the destination schema. But it does not include how i can specify the geomety type. https://support.safe.com/hc/en-us/articles/25407537293069-Dynamic-Workflows-Destination-Schema-is-Derived-from-a-Schema-FeatureMy data is stored as geometry but it is known and expected that specific tables will polygon or lines. Is there a somewhat simple way to do this without creating the destination tables beforehand?
Reading the documentation and online search regarding this transformer - ArcGISOnlineConnector I have some questions. The Actions are pretty self-explanatory and it seems possible to both upload and download data from AGOL. However that does not seem to be the case. I can list every Group and item. However when I try to Download it just says: cannot download file of type 'Web Scene'cannot download file of type 'Scene Service'cannot download file of type 'Feature Service'cannot download file of type 'WMS'cannot download file of type 'Geocoding Service' I was hoping to download a Geopackage or something else. Is there anything that CAN be downloade? Any other usage areas for this Transformer?
Hello,We have an excel file with different first lines.For this reason, we have defined the featurereader in the excel file, which allows us to have the column names in A, B, C ... and not the column name with the values of the first line.This works very well, but as soon as we put a parameter in the dataset (instead of a hard-coded directory path), we lose the A, B, C field names and can't use them generically.do you think there's a solution for keeping column names in A, B, C?Best regards