Creating and organizing workspaces. How to use Workbench.
Recently active
So I have a big job to process and I’m trying to work out some theoretical numbers for the client (a rough estimate on the processing time range depending on various factors). I want to leverage the fmeworker limit of 16 parallel threads (I’m assuming a machine with essentially unlimited resources) because the fme.exe limit is 8.I have a batch job of many tiles with many rasters and features which all need to be processed at once. The easy thing to do is to just throw it to the workspace runner but the license only gives me 7-8 parallel threads (which is probably more than enough to be fair, most likely CPU will not be the bottle neck).Anyways for fun I did a test to see what the limit was on the number of parallel groups to send through a custom transformer when parallel processing was on. I pretty quickly got the following error: An error has occurred. Check the logfile above for detailsf_6 (TransformFact): Transformer 'ThreadLimitTester': Unable to create worker; try reducing the pa
Just started my new job and the work environments are very similar (ArcInfo / Advanced License; new office FME Professional edition - old office FME oracle edition) and all I can see is the Open FileGDB reader writer. Wondering if safe discontinued the ESRI File GDB reader and writers?ArcPro 3.3.2 - advancedArcMap 10.8.2 - infoFME 2023.1 - Professional
Is it possible to retrieve automation tasks/items using the FME Server REST API? Specifically, I’m trying to find all workspaces that are run by automations and I’d also need to determine the published parameters passed to each workspace.The Schedules and Notifications (Subscriptions) REST API methods allow all “items” to be retrieved, which provides access to the workspace details but this does not seem to be exposed in the REST API for Automations…I’m using FME Server 2019.2.2 Build 19817 - win64.
Hello Forum, I’m attempting a fanout on a geojson file based on specifying an attribute. This is a simple task which I’ve carried out dozens of times before with success on other datasets. However, for some unknown reason it’s failing to fanout on a particular dataset. I’ve read in the geojson with FeatureReader. This is connected to a to a FeatureWriter. In the FeatureWriter I have… Selected format: Geojson (Geographic JavaScript Object Notation)Browsed to a destination UNC path and ensured the .geojson extension is shown. Note that the geojson writer it defaults to .json extension, so you have to change it to .geojson manually.In Parameters changed ‘Reproject to WGS84’ to NoIn General>Feature Type Name selected ‘open text editor ‘ and then selected Primary_Function from the FME Feature Attributes list. Note that Primary_Function is the attribute I wish to fanout on. The attribute exists in the source dataset and does contain values. However, this then fails to write fanned out geo
I am trying to compare the total of features from my original data source to the number of features ultimately written to a dataset. I thought I’d create a variable from Statistics Calculator result to refer to in my Tester. Is there a way to reference the global variable I have set within the Tester phrase ?If not, is there an efficient way of comparing the stats between the beginning and the end ?
Hello FME community,I'm trying to create polygons from Polylines. With Aerabuilder I could create 95 percent of Polgons. As you can see in the picture, in some cases no polygons were created.Polygon creation failed (yellow frames)
HiI’m using oData web service url to do http request and extract data.The request fetches every 5000 requests, I want to loop until ‘oData.nextlink’ attribute in the data is nil\empty.Its frustratingly taking so long to resolve this issue !FME doesn’t allow feature holder\sampler in loop, and there is no proper help\round about method.And can’t see \ debug from custom transformer fmx file .. so, how to resolve ?Attached are my ETLs, (main ETL & custom) (2023.0 version) for reference, can anyone please help identify issue and advise for fix Please ?Thanks
Hi, I'm currently in the process of making available a new Windows VM for FME Form users within our organisation. Our IT guys anticipated also using this machine for ArcGIS powerusers; but that's not preferred by the application manager team.As such it has some lovely CUDA GPU capacity assigned. Which is a bit of a waste if we don't use it. Can FME use CUDA for certain work? Are there plans for using CUDA?If there is no clear case for retaining the GPU, i intend to give it back to the resource pool. There is no guarentee I can ask it back: it is a finite resource, first come first serve.
Hello,please help me! During translation I can read error message "ERROR |MULTI_WRITER: No MULTI_WRITER_TYPE{*} or MULTI_WRITER_TYPE{NULL} was specified" -> whats wrong please?
Hello Forum, I need to find the records from one dataset where the value in one of its attributes is the same as the value of an attribute in another dataset. For example, If I have two datasets (A and B) I want the output to show which records in A have shared value in B, based on a specified attribute. I want the output to be a selection of A. Something like this… I’ve tried Matcher, but that didn’t work for me. Thanks,Stuart
I remember back in 2021 when this happened, someone from Safe was able to point me to where the “recover” file was saved. Please help me again as i have created a very robust workspace but have not saved it and it would take me 3 hrs to recreate this workspace
I’m a newby to FME.I’m about to write workspaces to refresh a list of AGOL feature services. So far I’ve written a simple one that works on a small service.I’ve got a few questions.When I search for potential pitfalls in this forum, they are 5+ years ago and relate to AGOL time outs and also issues updating large datasets. . I’m on the latest release of Form & will use Flow (Hosted). Are these issues still relevant these days ?My largest dataset is our Cadastre being about 140K features. Am I safe to use the Writer ? I am doing truncate & inserts. One might argue that I could use ChangeDetector beforehand, however, I reckon the processing required to compare 140K features would be greater than a simple truncate & insert. (am I right?) If an error is encountered, I would like to quickly restore the destination feature service back to what it was. Any tips on the best approach to doing that ? My idea is to copy the destination service elsewhere in AGOL before commencin
I have an FME workbench that runs every 10 minutes that reads from an ESRI Feature class in a SQL database. Once in a blue moon I get the following warnings.Unable to retrieve feature from table/feature class 'xxx'. This feature, and any other feature from 'xxx' that cannot be read, will be skipped. The error number from ArcObjects is: '-2147154945'Unable to retrieve feature from table/feature class 'xxx'. Aborting reading from table 'xxx'. The error number from ArcObjects is: '-2147216010' Yet the next run 10 minutes later has no problems. When this occurs I want the workbench to stop rather than proceeding with the incomplete data set. Is there a way to force a termination when this happens?
HiWe need to allow users create a zipped shapefile with a single feature. The shapefile needs to be named as the reference for that feature (an attribute in the table)I can create a zipfile by clicking the zip icon in the dataset field of the writer. However, this just creates a zipfile which is named after the folder it is saved in.Is there a way to name the zipfile using an attribute? Or is there a way to automatically rename the zipfile? Thanks
I have a shapefile with sub areas and I want to create an FME workbench to assign the areas to 3 operators with the condition that the operators don't work on areas close to each other. which FME transformers can I use as I want the output to be 2 shapefiles one with a list of areas that can be assigned to the operators at a particular time and another one with a list of areas that cant be assigned at that time.
Hiyas,I’m having an issue where I try to import data from a excel spreadsheet into a Choice user parameter.I only ever get the first row to import, even though it displays all data.Choice → Choice Configuration → Dropdown → Import from datasetDataset = My excel fileFormat = Microsoft excelParameters shows the valuesNextImport Mode - Import from: Attribute namesNext Import then results in only the heading being imported; Am I missing something really obvious? Thanks,
Dear FME community.I hoop you will help me with the question I have. It seems simple question, but I could not solve it. I want to find out overlapping pipelines in the same dataset with different owners. I want to have list of the piplines who overlap with each other and who have different owners.I used line on overlay transformer but I could not get what the result I wanted. Any help is apricated and thank you in advance
At the FME Flow (we’re looking at 2022.2 specifically) level, many people will be familiar with the ability to set time outs for queuing and running Through the API you can also specify these by using the Job Directives: https://docs.safe.com/fme/html/FME-Flow/ReferenceManual/Job_Directives.htmThe scenario that we’re facing is there is an external process (not FME) that is setup to call a bunch of different workspaces depending on what is passed through. This is a process not managed/administrated by us. Depending on the workspace called, we want to specify the running expiry at the workbench level, of course this could be doing with logic in the external process using tm_ttl/tm_ttc, but that would mean needing to release updated code each time we want to change this.We’ve tried specifying tm_ttc as a workspace parameter (both published and private) but FME Flow doesn’t acknowledge it. Talking with @todd_davis we’re both sure you used to be able to do this (specific run time limits at
Hi all,Using FME(R) 2023.2.1.0 (20240104 - Build 23774 - WIN64) and tested in 2024 also.When I write a KML that includes multipolygons (aka nested donuts, aka inner rings) the records display correctly in FME Inspector (correct outline and fill) but when opened in Google Earth the fill is missing for records that have polygons within polygons (ie a donut hole has polygons within it). Simple polygons and simple donuts have correct fill.I’ve tried reversing the orientation with Orientor.Also tried playing around with the DonutHoleExtractor without luck.Any good ideas for a fix?
Hi All,I have a json templater which i’m using to create a json that has ‘firstname’ and ‘lastname’ - there’s 3 rows of data.I created the following in the root template but it outputs the 3 separately:{ "firstName" : fme:get-attribute("first_name"), "lastName" : fme:get-attribute("last_name")} I then tried creating a sub template -with the following settings: Here is the output I get:{ "firstName" : "first2", "lastName" : "Last1" }{ "firstName" : "first2", "lastName" : "Last2" }{ "firstName" : "first3", "lastName" : "Last3" } I can see that it’s missing an extra set of curly brackets, and when trying to run this result through a JSON validator it fails and gives this message: “Unexpected non-whitespace characters were found after the JSON text” I’ve looked at the JSON templater documentation and the exercise but I can’t seem to get this working.Any help would be greatly appreciated Many thanks, Sam
So this is going to be a dumb (yes, really their are dumb questions) but I can't for the life of me find a way to simply "print" or export my workspace to PDF for documentation purposes. Sure, there are screenshots but a screenshot of a massive workbench is illegible. When I go to "print" it will let me choose a paper size... but all I want is the ability to a) choose a custom size or b) "print" or export the entire workbench at (say 300 dpi) or something of that nature. Currently the "best" I can do when using Adobe to "print to PDF" is 47 ANSI E sized pages on export! Seriously, there has to be a better way???? Any ideas or suggestions? Note, I've tried programs like "snagit" to do a rolling screencapture but that doesn't work either. Matt
I have an “Extras” attribute that is XML-like with different number of attributes in different order within the attribute value.. When I use the , XMLXQueryExtractor* and configure a query, it only returns rows that has one tag only called <PAGE_TITLE>, like below. The query returns this query result in the list attribute. THe resr of the rows are rejected as invalid XML Queries.For the other failed queries, they have multiple tags, as shown below.What I’d like to do is aplit the tags into field::values regardless of number. This attribute value:<PAGE_LINKS>https://p3tips.com/community/</PAGE_LINKS><PAGE_PRECISEPUBTIMESTAMP>20240905151800</PAGE_PRECISEPUBTIMESTAMP><PAGE_TITLE>Lynchburg Police searching for suspect after armed robbery at Family Dollar</PAGE_TITLE> Woud look like below with all the attributes in the Extras value split into however many tagged attributes.PAGE_LINKS PAGE_PRECISEPUBTIMESTAMP PAGE_TITLE https://p3tips.com/com
Hi, I’m finding XMLFragmenter removes geometry, is this expected?Data Interoperability in Pro 3.3, so FME 2024.0.1.0 build 24202.Thanks.
Hello, anyone knows how to split a DWG into tiles? i can make it happen with tiler however i cannot split it or simply chhose the tile i want to export
Hi everyone,I have a workflow that exports data from Oracle to another formats, like Shp, GML... using dynamic readers and writers. This workflow recive some parameters, like table name, that is used to read de Oracle table and to write the file name. The question is about GML file and date values. When the Oracle table has datetime attribute, FME exports as text, by default. I've achived that FME exports GML with attribute as date type, but this attribute has null value, beacause the format of date is wrong. That is, the exported date value is "19630904000000" but it shoud be "1963-09-04".As I said, I use dynamic readers and writers, beacuse I don't know what table will be exported and I can’t see the table structure, neither, when data goes out by Generic port of Reader. So I can't use StringReplacer and DateTimeConverter, because I would need to know the name of attibute. Anyone has any idea how to modify the format of the date?