Creating and organizing workspaces. How to use Workbench.
Recently active
Since upgrading to FME 2018 (32-bit version), I've been getting intermittent connection errors when using either my database connections or web connections.There are times when I run my workspace without any issues, but subsequent runs may, inconsistently, produce connection errors.If I go into my Database Connections or Web Connections through FME Options, I either get "Error retrieving connections", or it's "unable towhen the Testing the connection the test fails as it's unable to connect (i.e. "Unable to connect to Oracle database, because the database environment could not be initialized".
Is there a generic way to extract the feature count for each feature type in the writer? For example, I would like to capture this information either in the same workspace, or from the log file's Feature Written Summary section. I would like to capture this info for each updated feature type to a central location for review and reporting. Most of our FME processes are running on FME server.
This might rather be a simple task, but so far i have no clue how it works to merge and concatenate joined_list elements. I want to concatenate the two element values into one string and at the end write all the list elements out into one attribute.Result should be {KA:28,KA:12,KA:4}
The issue I am having is that I want to read in only 1 feature per and therefor end up outputting only 1 feature per type without breaking the already established connections. I have a feature reader with 300+ types of features setup/with connections across a massive work space. If I change the parameter in the FeatureReader to "1" for max features to read, it asks me to select a workspace. Each .dxf "workspace" has only a couple features, but combined with the remaining 100's of .dxfs I get all of features I need. So when I select one dxf "workspace" it only provides x amount of features. If I hit "cancel" at this point then all the features in the reader go away. The "PATH" is setup to parse through all of the .dxfs in the folder. Is there a way to change that parameter in the feature reader without breaking the already established connections?
Just going through the motions of upgrading an FME Cloud instance and I'm looking for a way to migrate the job history into a new instance. I know, however, that this isn't supported with the backup and that some db work is required (https://knowledge.safe.com/articles/816/fme-server-job-history.html) The problem is that I can't seem to access the fmeserver schema. My work around will be to use the REST API to get the data and create my own job history table but I would rather avoid that if possible. Am I missing something here?
I have hit a wall. I realize that job downloader in FME Server will zip the files but I am in a situation where I must use Job Submitter, but I need the final results to then be zipped for download. The entire problem originates from a need to use custom projections for the shapefiles. Here is the process:read in CADD files and other data from Excel filesProcess the information and create Shapefiles using FeatureWriterRead in the custom projection and write the prj file into the shapefiles using FileCopy writer.This saves all of the shapefiles as individual files. Since I might have 15 shapefiles, at 8 files per shapefile, I end up with over 100 filesto download. I would like to then zip up all of the shapefiles to make one easy download either for each shapefile or, preferable, for the whole batch.Please note - I am using job submitter because the prj file cannot be added to the file created in data download.DGN2GIS_Step2GB_CategorizedShapesProjection.fmwDGN2GIS_Step2GB_CategorizedSha
I'm having some trouble with the WorkspaceRunner transformer in FME 2018 windows 32 bit.Say Workspace B contains a FeatureReader transformer. In Workspace B, create a published parameter called FileList with a filename (Multiple) and set this as the dataset field in the FeatureReader to read txt files. Set the value of FileList to C:\\temp\\a.txt,C:\\temp\\b.txt. I have no errors running workspace B as is. Using FME 2018, I trigger workspace B using a WorkspaceRunner in Workspace A, I get an error which highlights the paths in the Filename (Multiple) parameter are not valid. It is using the same value as FileList (C:\\temp\\a.txt,C:\\temp\\b.txt) that is used in workspace b published parameter.It seams that the WorkspaceRunner doesn't send through the FileList value correctly in FME 2018. I have examined the log files and can see that in 2012, the Command line arguments are different. See below for differences (notice the qutation marks missing in 2018): FME 2012 log file:`--Fi
@takashi I am not getting any output from my OSMDownloader. 0 features are returned. I have tried switching my coordinates to make sure I am placing appropriate values in min/max and x/y. I tried reducing my extent significantly to a .05 difference in height and width. Unfortunately, I still get no features from the OSMDownloader. Any idea as to why? I am relatively new to FME.
Sometimes I really need to use a Group By in a transformer, but the parameter is not available. Is there anyway I can hack it?
I am processing a directory of XML files. I have set the source dataset to just .../*.xmlHowever, I would like to save the results according to the source Filename. How do I set the output filename to be the same as the input filename?
Hi everybody, it seems to be an easy job, but at the end it is taking a lot of time to implement. I have patterns in my oracle database to access and read multiples Excel files, which contains several sheet. When i access to this database, i get the filename, i cannot call directly the Reader for excel passing the name, sheet name and format. I did some experiment with subroutines (like external calls) but it seems more complicated than expected. How can i read Excel files whose name is in the database without calling another workflow ? which is the logic standing behind this ?
I want to split a line using distances along that line that are from a separate data table: startendcategorylength02.3Cat 22.32.315.1Cat 312.815.126.6Cat 211.526.627.3Cat 50.727.329.9Cat 22.629.931Cat 51.13133.3Cat 22.333.334.3Cat 5134.339.7Cat 35.4 I am probably just tired, but I can't seem to work this out.
Hello.Currently I'm using the FeatureWriter to update my file geodatabase. It's running at a rate of 18 features per minute, and is comparing 13,998 features against 1.2 million in the dataset being updated. I'm running a test of this process on a group of features which is a good representation of what a typical update will be in terms of feature quantity. Is there any way to speed this up?Thanks.
I'm trying to write a custom transformer with a python script that looks at the Public Parameter.There is a lot of trial and error going on but I think I'm doing it as it should now. This helped me getting started:https://knowledge.safe.com/questions/44873/getting-published-parameters-from-within-a-python.html The transformer works if the Public Parameter is set to a fixed value or if it is set to a feature attribute value. But when there are two instances of the Custom Transformer it looks at whatever starts first. So the second transformer uses the fixed value of the first transformer. custom_transformer_name = 'MultiInstanceBug'parameter_name = 'codeLength'instance_name = FME_MacroValues[custom_transformer_name + '_WORKSPACE_NAME']mv_codeLength = FME_MacroValues[instance_name + '_' + parameter_name]if mv_codeLength[0:6] == '@Value': codeLength = int(feature.performFunction(mv_codeLength))elif mv_codeLength[0:6] == '@Evalu': codeLength = int(feature.performFunction(mv_codeLength))e
Hi, i want to find objects by tracing a route.But sometimes there are more than one route because of branches and to not get into an endless loop i need to save the ID of the segments which is already used.I couldn't find a solution to save an attribute value temporary in a loop, so I created a list (in the form of all relevant IDs, comma separated) an for every iteration where a segment with the ID is used, the ID should be deleted in the list.And because this List counts for every feature, I need something like a (published) parameter, which is dynamic. How can I do this?thanks, keough
I want to write features to an Oracle database. Some featuretypes have geometry others are non-spatial. Tables are created by the writer, schema is dynamic and read from a FME schema txt file.If I use Oracle Spatial Object writer ALL features get a GEOM attribute and are treated as if they are spatial.So I tried to optimize the workflow and use a second writer - an Oracle non-spatial writer - for the non-spatial features.This improved performance considerably, as I have 50% non-spatial features.Problem:Oracle Spatial Object Writer ALWAYS creates tables with UPPERCASE names (and converts them if necessary) while Oracle non-spatial writer ALWAYS creates the tables as they are in the controlling attribute.In this way I cannot use the same schema definition file for both writers.Anything I missed out on ?
In workbench I have a published parameter that specifies a folder output value. I want the default value to be a resource that is already registered with the server $(MY_OUTPUT_FOLDER). How do I make the published parameter the server resource the default value? I've tried using the same path as the server resource but it doesn't pick it up. I've also tried using $(MY_OUTPUT_FOLDER) but the parameters won't let me set it because of the $ character. Ideas?Essentially what I'm trying to do is to allow the user to run the workspace on FME Server without having to specify any parameters by converting all public parameters to private parameters and publishing.
Hi all. Is it possible, like in traditional programming, to create functions or subroutines in FME, when you have a piece of common FME coding you want to reuse and execute several times from a workbench or from different workbenches i.e logging of certain events or error situations. If so, please let me know, how to do it.
Hello,I am trying to develop a startup python script, that will look at an incoming file and list the feature types that are within that file. Then compare against another list of mandatory feature types. Essentially I would like to make sure that required feature types exist in the file. I am fairly new to Python, and still learning, but any guidance would be helpful.import fmeobjectsimport fmeclass FMEFeature(object): value = fme.macroValues['SourceDataset_AUTOCAD_OD']fme.getFeatureType(value) #unsure what to do nextThank you!
Hi @sander!I'm using your ListAppender inside a custom transformer and it's throwing python error. This is the message:INCLUDE -- failed to evaluate Python script `def ParamFunc(): return FME_MacroValues[FME_MacroValues['ListAppender_WORKSPACE_NAME'] + '_' + 'APPEND_VALUES']value = ParamFunc()macroName = FME_MacroValues['ListAppender_WORKSPACE_NAME'] + '_APPEND_VALUES_WWJD'if value == None: return { macroName : u'' }else: import six try:  value = six.text_type(value) except UnicodeDecodeError:  value = six.text_type(value, 'utf-8') return { macroName : value }'Program TerminatingTranslation FAILED.Traceback (most recent call last): File "<string>", line 5, in MF_Include_1546897172100 File "<string>", line 4, in ParamFuncKeyError: u'$(test_WORKSPACE_NAME)_ListAppender_10_APPEND_VALUES'No issues when ListAppender is on the main canvas.
Cause table & attributes name are longer than the available size. And the over solution is not so simple to use.
Hello. I am trying to run a workbench on AWS EC2 instance.What I want right now is that the workbench is initiated by itself at a scheduled time.Going ahead, the later on thing is to turn the machine on, run the workbench, wait till the workbench finishes(till the output is upload on S3), and stop the machine.Thanks.
Lately I have been working a lot with Postman. One of the nice features of this tool is that you can define environments, e.g. development and production. For each environment you can define it's own set of parameter values. So when you switch between development and production, the values for the url, userid and password are automagically changed also.How can I achieve this same behaviour in FME? I would like to have a published parameter that lets me choose between development and production and subsequently sets the correct values for url, userid, password and possibly other parameters.
We are using collector for ArcGIS to edit assets with a GPS unit to portal hosted feature services to sub-foot accuracy. We also have corresponding production feature classes in a SQL server sde of networked water assets that is the system of record for attribution. I have a workspace that will update changes in attribution from our production sde database (SQL server) to attribution in a portal hosted feature services that has editor tracking enabled. I want to maintain the last editor and last edit date fields in collector that is done by the field staff. When a feature is update by the workspace, it is overwritten to the run date and time and with my name as the editor. I was wondering if there is a way to use python caller to disable and enable editor tracking at the beginning and end of the workspace. I am not very python savvy, so any help in this regard will be appreciated. If there is another way other than python caller I am interested in that also.
Hi, I was wondering if anyone has created a dynamic update detector. Ideally where you pass the original table, revised table, key attribute and attributes to match as parameters to a workspace or custom transformer. something along the lines of:At the moment I would need to have all the tables dynamically popualted along with the key attribute and attributes to match. This is because when I tried to do it with defined tables I run out of memory, so it would be more ideal to do it one at a time. This could then be super reusable in all code. Many Thanks for any help or pointers. Oliver