Creating and organizing workspaces. How to use Workbench.
Recently active
Running some truncate and loads from DB2 to ORACLE.Since the query is basically 'select *', is there any way at all to increase performance of FME selects from DB2?One table of 72k rows takes about 30 seconds. That isn't monstrous but as we get to larger tables the time just increases. The same truncate and load in Informatica takes 4 seconds.Looking for any 'out of the box' ideas here. i would like to use FME because dev time is much faster but i can't justify the 'job run time and resource' usage with multiple tables.I have read most of the suggestions that i have found with general searching. User where statements, order by, select less columns..etc.Also tried odbc connection and straight jdbc string connection but no difference. Thanks.
Hi, Is there a way to get a list of feature type names of all Readers and writers inside workspace? I don't want to read data. Just need to create some sort of report of what feature types do I read and write. And in my workbench there are lots of them, so doing it by hand is gonna be time consuming Regards,Želimir
Hi FME-community,I'm having problems with the performance of a workspace that straightforward translates a CSV to a Microsoft Acces DB. Somehow the performance is not stable. During the evening it was 30 times slower than during the day, while more memory was available and all other settings did not change. FME Server is deployed on a Azure VM (Windows Server 2016). In the usage notes I read: The performance of this format is dependent on the amount of memory allocated to the Java Virtual Machine (JVM).Is this an explanation and how can I solve this?I hope someone has an answer to this.-GeoJanMov
Hi, I have this problem https://knowledge.safe.com/articles/944/remove-the-geometry-type-appended-to-feature-type.html , and the Writer does not support the option of using the Geometry from the schema definition without adding the geometry type to the dynamic name (i.e. "_polygon"). This is a problem since I'm inserting into existing tables with an fixed geometry type. There seems to be no way around it?
The What’s New in FME 2019.2 blog contains some writing about FME AR and underground features.FME ARLoad an AR model in 1:1 scale in addition to the existing “table model mode”. Pick the scale you want from a list, then inspect your data against a real backdrop.See an improved display for underground features (those that are below 0).See improved texture appearance when there is an ambient material property.But I'm not sure how to make this happen? What I tried was to read a cable feature, create a "floor" level with a boundingboxreplacer and an extruder, offsetting the cable to 10 meters below the floor. Looks OK in the inspector but in the app it seems the model is positioned with the lowest part on the detected surface.
So this could be something really basic. I have an FGDB that is created from an ArcSDE Reader. The Feature Class Names are of the format:SDESchema.Feature Class NameHowever, the FGDB Open API Reader instead reads the Schema Feature with the fme_feature_type replacing the "." with a "_" like this:SDESchema_Feature Class NameThe problem comes then when try to use this in a Dynamic Writer, because now the Schema Feature has a fme_feature_type attribute that uses a "_" in place of the "." but the Features themselves are attributed with the original format with the "." qualifier, and the Writer complains that it didn't get a matching Schema definition for the Features it is trying to write, because of the mismatch created by the Reader.One workaround would be to use a StringReplacer on the fme_feature_type attribute to make them match, but I feel I'm missing something obvious?
I already use Readers (ESRI Legacy) to "merge" tables based on a relation condition. This can be done by adding the SDE30_IDs clause to the mapping file. (Since it's not supported in Workbench)Since Readers and FeatureReaders seems to use the same underlying libraries, would it be possible to do the same thing at the FeatureReader level. This would allow flexible and efficient way to retrieve data since a more complex conditional clause could be built along the transformers way.Thanks,Mathieu
I wanted to centralize assignment of Database Connections in my workspace as I have quite a few transformers, readers and writers that use them and want the flexibility of switching this from one central location while testing (dev vs. test vs. production) and am noticing there is no "Connection" or 'Database Connection" type of parameter. I see "Web Connection" but nothing else. Curiously when a SQL reader is added though, a parameter is indeed added to the workspace as you can see below, but it doesn't show what kind it is (2019.1):I will re-purpose one of these and re-link, then delete the redundant ones, but this does feel a bit of a workaround and was expecting to be able to create the user parameter from scratch. Unless of course I'm missing something? Thanks.
Hello FME community, The attached Workbench is based on @david_r Workbench "Retrieve a value from the Python startup script".I added a Shutdown Python Script to retrieve the initiated global variable in startup and write its value in the Log File. I got the following error on Translation log:PythonFactory failed to load python symbol `FeatureProcessor'Factory proxy not initializedI am using FME2018.1.1.1 WIN64 with 3.6 Python Compatibility.Does anyone have any solution or other potential solutions to pass global variable from startup to Shutdown and write its value in the Log File? Link for @david_r orginal Workbench:https://knowledge.safe.com/questions/58082/passing-parameters-from-a-python-script-tool-to-a.html Regards,Afia@mark2atsafe @egomm12 @david_r @jlutherthomas @arnold_bijlsma @jdh @majdoleen @redgeographics @danilo_fme
When I run a workspace from FME Desktop the Log Message Filter setting in FME Options/Translation is respected. But it's ignored when I run fme.exe from the command line and pass a Command file. Is there a parameter I can set on the command line or in the command file? I just want Errors and Warnings.
i would like to aggregate the dataso i used Aggregator Group by Main unique IDAttributes to Concatenate 3 Attributes name that i want to combine but i found each row has one Attribute main ID, and many id at the first and second (Attribute to Concatenate ) duplicated more than one but the third not , my question : how could i remove the duplicate IDs inside the attribute to Concatenate , i need only the unique IDs only if dupliated .i tried Duplicate filter but it does not work . after i used the aggreator attribute to oncentate submain 1,submain 2 ,submain 3 the ouput looks like that ,i would like to delete duplicate in each row at eaach attribute Main submain 1 submain 2 submain 3 5 2,5,4,4,3,3,4 6,6,3,2,2,5,1,5 9,4,7,8,2 FME 2018so what should i do ,thanks for help
Hi.I'm trying to build a simple workspace that extracts logged features from a log file.Since a logged feature in the log file is wrapped by a starter line and an end line, I'm using a variable to keep track of whether I'm inside a logged section or not.I'm reading the log file as a text file, and forking the read line feature 3-ways with connection order: (1) check for start line, (2) output if started, and (3) check for end line. In (1) and (3) I'm changing the variable to 1 or 0 resp., while in (2) I'm checking whether the retrieved variable value is 1 before doing an output.Unfortunately it seems that my check in (2) is always true ??It looks like the FeatureReader is sending all line features into each connection, one by one, so first (1) receives all features, then (2) receives all features, etc.I inserted a FeatureHolder immidiatedly after the FeatureReader, in case it was a problem with the reader, but it changed nothing.As far as I understand, a single
Hi!I'm using FME Workbench 2019.0 and overnight it has changed the appearance. Now the font is totally different and can´t get back my original settings. According to FME options this new font is the default and definitely it isn't that one. How can I change it? Any help here? Not a big deal but annoying..Thank you very much in advance!!
Since I am at a customer, not being able to connect the our license server via vpn, I installed a temporary fixed license. This is installed without issues using the license manager, however, my FME slows down terribly! And with that I mean everything: opening FME Desktop, opening FME Inspector, loading data, running a model, ... I am really not able to work like this. Anyone an idea what is going on here?
Problem Summary: Fme engine installed on another server can't read from the FME_SHAREDRESOURCE_DATA location since it is the local path of the fme server and not the UNC path. Problem long description: I have an FME Server automation that gets triggered by an email, and has a workbench that reads the contents of the email and writes it to a csv which is stored in a sub-folder in the $FME_SHAREDRESOURCE_DATA directory of the fme server. This then passes the full local path location of the csv in $FME_SHAREDRESOURCE_DATA to another workbench. The problem is this workbench is run on an engine this installed on another server which then attempts to find this file locally but it doesn't exist since the $FME_SHAREDRESOURCE_DATA full path location is only on the fme server and not the UNC path.Question: Is it possible to get the UNC path of the $FME_SHAREDRESOURCE_DATA so a an fme engine installed another machine can access this location when trying to read data in this it in a wo
I can see another question about this. However if my understanding is correct. The where clause is not working for me at any level.I am using the blm national dataset which has a large number of records for all states. I only need to utilize 1-3 states at this time in my workbench. I have attempted a where clause on the reader , an SQL before read (which seems like the most efficient) and a where clause on the table. However the sql before will not run at all.When running the where clause SECDIVID LIKE 'MN% it runs but in the test filter directly after the reader i begin getting unfiltered records. This should not be possible if it has already filtered by the where.I am not willing to sit through all of the records (it will be at least 4 hrs) to see what happens.does this work for anybody else?What am i doing wrong?I can see the following in the log. but why doesn't it only return these states. It works in Arcdesktop. GEODATABASE_FILE_1 Reader: Using Enhanced geometryUsing WHER
I am trying to pass a string as parameter the string has "\\n" as new line character. When I use fmeobjects.workspaceRunner.runWithParameters to pass the string the string is cutoff at the first line. I tried to escape the backslash but I got mixed results. The entire string was passed but the string contained the actual "\\n" in it. I need the new line character like this
This is the final step to some complicated data sorting - so as an example lets say I have the following data: indexequipmentNamesequenceequipCount1a102b103c204b215c316d307d418b429b5310e5011b6412e6113b7514f70 The goal is to get data ordered like this: abbccdd... etc. This works by sorting the sequence column ascending and the equipCount column (unique counter of equipment name) descending. The problem is that if there is a sequence pair that does not contain a zero (both pieces of equipment have appeared before) then its no longer beneficial to sort equipCount descending, instead it needs to be sorted ascending(see index 7 and 8 sequence pair 4, they should not reverse order 2 to 1; and index 11 and 12 sequence pair 6, they should reverse order to be 1 to 4). Again this is all to get the final order to be abbccddbbeebbf. My solution to this was to check if each equipCount value in a sequence pair was not zero, and then to make even numbers = 0, because an even unique count means it is
I installed FME 2020.0.2 (build 20232 -win64) on a Windows Server 2012 R2 Standard 64-bit (6.3, Build 9600) machine.It works fine but throws an error message on exit and I'm concerned it may hide some important unwanted functioning I haven't found out so far. Before installing FME Desktop on our production server we want to make sure everything works as expected, so I can't overlook this apparently minor thing. Any idea of what is it about?A repair install did not solve the problem. See error details below: Application Name: fmeworkbench.exe Application Version: 2020.7.39.20232 Application Timestamp: 5eab95e4 Fault Module Name: Qt5Core_fme.dll Fault Module Version: 5.12.3.0 Fault Module Timestamp: 5cffcf60 Exception Offset: 0000000000028658 Exception Code: c0000409 Exception Data:
I have a dataset where want to write separate files based on multiple attributes. From the example below I would like to write a file with all the Jones, another with all the Smith, a third with all the Farmer and a fourth with any Kim. Fanout does not handle multiple attributes and in this case the StringConcatenator trick wont help, since most of the rows would be contributing to separate files. Any ideas on how I can achieve this? A B C D Jones Farmer Kim Jones Smith Jones Smith Farmer
i would like to create the readers in workspace and output of the first workspace as input for the second workspace . i tried workspace runner but i did not get it to work and i am not sure where can i find some articles about transponder sender and reciver if that what should i useFme 2018 any help plz and thanks
Hi,I've been looking for some documentation on FME Data Express with no luck. Specifically I'm looking for what published parameters are supported by Data Express.Some of my published parameters in a workspace has the label [unsupported] added to them when I open the workspace in Data Express.It would be great help to know which types of published parameters are supported.
Hello,I want to create a unique numeric ID in a new attribute field from a two-letters string attribute, this way:feature - IDAA - 1AB - 2AC - 3BA - 1BB - 2CA - 1CB - 2 CC - 3 CD - 4etc... I guess I need to count how many times a letter is listed and then create IDS for every count and repeat but I don't know how to do this. Many thanks, Max
I want to set the value of an attribute, depending on the environment, where my workbench is running (desktop, server dev, server prd),. I could do this with an parameterFetcher and get e.g. FME_SERVER_HOST. But I don't want to do this for each feature, but just once at the beginning.What's the best way to do this?I tried it with a scripted parameter, but I get an error, for FME_SERVER_HOST is not know when I run the workbench on desktop.Then I tried it with a startup-script. Here I don't get an error, but how can I set a parameter here, that I can use in the workbench?Thanks, Vera
Hello,FME 2020.0The screenshots below describe the question.