- - Community Resources
Ideas
Product Updates
Events
Blog

Home
Forums
FME Flow
Flow Hosted

Flow Hosted

Deploying FME Flow in the cloud

79 Posts
241 Replies

When you subscribe we will email you when there is a new topic in this category

79 Posts

Recently active

Most replies Most views Newest first

dejan11Contributor

asked in Flow Hosted

FME Server CPU (Dynamic Engines) ≈ FME Cloud?

As far as I understood from the documentation, the self-hosted FME Server (installed on my own server) can be also licensed by CPU (Dynamic Engines)?That means that I have a full control and security (especially in terms of data privacy, etc) of FME Server like self-hosted FME Server with standard engine(s), however in terms of license it acts similar to FME Cloud?So if both questions are correct then the difference between self-hosted FME Server licensed by standard engine(s) and self-hosted FME Server licensed by CPU (dynamic engines) in regards to license is that the client pays for an extra standard engine only once, but for CPU Usage as long it in hours takes - continuous payment?

asked in Flow Hosted

I am trying to access a FME Cloud instance but am getting a consistent error. I followed http://docs.safe.com/fme_cloud/FME_Cloud/Content/About_Instances/Instance_Lifecycles.htm but with no success. Any insight into how we can get this instance up ?

asked in Flow Hosted

Issue with FME Server jobs - XML module errors

Hi there, I am running a number of jobs (all using the same workspace) on FME server, and some of them are running for a period of time (each job processes a number of URLs, so it is getting through some of the list, and then failing), and then failing with an XML module error. There are two errors:XML module Error: XalanFileOutputStreamWriteException: Error writing file: and XML module Error: 'SAXParseException: invalid document structure I am running 14 jobs at a time (on an FME Cloud Professional Server, so enough CPUs), and I have noticed that three will fail at the same time, and others will continue to run, so I suspect there is something interrelated (perhaps they all try to write to some XML log file at the same time?) If anyone has come across similar issues, or has any ideas of what I can try to work out the cause of this issue, that would be very gratefully received -it is a fairly urgent problem! I am running FME Server 2022.1.2 Build 22627 - linux-x64 I have attached

paalpedersenContributor

asked in Flow Hosted

If I set up azure ad in Authentication Services, will I be able to read and write from an external azure fileshare which use azure ad authentication?

I wonder if I setup azure ad authentication service on my FME Cloud instance, would I be able to read and write drirectly from a azure fileshare? sourceFile //xxx.file.core.windows.net/yyy/source.extdestinationFile //xxx.file.core.windows.net/yyy/destination.ext Is this a common thing?

asked in Flow Hosted

Our fme cloud export has been going wrong for about 2 months.

Our fme cloud export has been going wrong for about 2 months.Every other week we import gml-files into a Postgress database. The export from our Postgress database gives errors in a small part (16 of 235) of our export files in the formats csv and shape.Only the BGT_WGL_*.shp/shx/prj/dbf and BGT_WGL_*.csv files fail. The fme scripts have not been changed in recent years.When we run the fme script locally (fme2019) on a PC the export of these files gives no errors.Has anything changed in the "FME cloud" environment lately? In some of the transformers? Last Friday (sept 16) we did a reboot of the FME-cloud environment but this did not help. Attached are some examples of a shape and csv file which look like an error file in html format.Also, we can't find any log files in the cloud of the fme creation process of the shape and csv files.These log files (which we do have locally) can help us on our way solving the problem.We also did not receive an error message that it was no

asked in Flow Hosted

How to add quotes around parameters values

I want to run a SQL before a writer to a postgis database.I am using parameters to dynamically run the SQL queries. Since SQL is case sensitive and my tables include case characters I need to put quotations around my table names. But putting quotes around my statement make it a string. I just want to add quotes the parameters value.select * from "$(schema)"."(fme_feature_type)"Desired resultselect * from "MySchema"."A_Table" Is there a way to escape the quotes somehow to just add quotes around the paramaneter value.

paalpedersenContributor

asked in Flow Hosted

Python Exception <ModuleNotFoundError>: No module named 'fmepy_emailer'

Hi, After upgrading FMECloud to version 2022 our emailer complains with the following error:Python Exception <ModuleNotFoundError>: No module named 'fmepy_emailer'PythonFactory failed to load python symbol `fmepy_emailer.Emailer'Factory proxy not initializedEmailer_CALLER (PythonFactory): PythonFactory failed to process featureAn error has occurred. Check the logfile above for detailsIt looks like the emailer depends on a internal package called fmepy_emailer that does not exist in the environment.

asked in Flow Hosted

FME Cloud - send email when process starts. send email when process fails or completes with log file attached.

Hi FMEers, I'm new to Cloud and have been looking at the Automations section. I feel like what I want to do is really simple, but I'm not sure if it's something I set up in Desktop, automations, notifications (classic) or in the Job Submitter notification area? I have my workspace in the Cloud. I've tested it, and it runs and outputs all my data (although it comes up with a failure message). This is why I want to have an email automatically sent to me when the process gets started via a server app by the end user and when it ends (or fails) along with the log file, so I can check if it has written out everything, but is outputting as a failure. I've attached the log file. Thanks in advance for your help.

asked in Flow Hosted

Hi Team, we have deployed fme server 2020.1.3.0 version into the linux machine. It is having python 2.7 version. Currently, one of the workspaces requires python 3.0 support. So, Is there any posibility that I can upgrade to the python 3.0 ?

Hi Team, we have deployed fme server 2020.1.3.0 version into the linux machine. It is having python 2.7 version. Currently, one of the workspaces requires python 3.0 support. So, Is there any posibility that I can upgrade to the python 3.0 ?

asked in Flow Hosted

Import error when using paramiko and scp python libraries on FME Server

Hello everyone, I'm using the paramiko and scp python libraries in a pythoncaller to transfer files from my local environment to a remote Linux server. When testing locally I received an import error which I could solve by removing all contents of C:\\Users\\<user>\\Documents\\FME\\Plugins\\Python and reinstalling paramiko and scp. However, when I transferred the contents to the Plugin folder on my FME Server instance, managed by FME Cloud, I received the same import error I previously received on FME Desktop even though the Plugin folder on FME Server was empty to start with. The import error I receive is formatted like this: Python Exception <ImportError>: cannot import name '_bycript' from partially initialized module 'bcrypt' (most likely due to a circular import) (/data/fmeserver/resources/engine/Plugins/Python/bcrypt). The following python code is the source of the import error:import paramikofrom scp import SCPClient I noticed that the importerro

tim.rastall1Contributor

asked in Flow Hosted

Is there a way to add Azure Blob Storage web connections in FME Cloud?

Hi folks, My FME workspace reads data from Azure Blob Storage. I'd like to run it in FME Cloud, but the account/password credentials do not get published to the server.I am able to replace the SourceDataset user parameter with a custom string containing embedded credentials ("fme://safe.azurestorage....") then run the workspace successfully, but this solution has some maintenance issues (not scalable, and would need to be applied to every workspace). I was hoping that I could create a Web Connection for Azure Blob Storage in FME Cloud, but it looks like I can only create an S3 bucket web resource: I also cannot add a new web connection: Can anyone help solve this issue? Thanks.

asked in Flow Hosted

When publishing a workspace in FME to track version history a 'Commit' link is enabled as part of the publishing procedure. When I attempt to publish a workpace to I am not presented with this link.

When publishing a workspace in FME to track version history a 'Commit' link is enabled as part of the publishing procedure. When I attempt to publish a workpace to I am not presented with this link.

asked in Flow Hosted

Upload Url to Fme Cloud for processing

Good Day Everyone I am trying to create a directory watcher for a NextCloud folder. Once changes are made, run a workspace and publish results back into the NextCloud folder. So I am attempting to first upload the folder to fme server so that I can create a Directory Watcher to look inside the FME Server Resources folder and go from there. I have managed to get the download url from NextCloud, but I am struggling to conceptualise how to upload the url link as a .zip file to the FME Server Resources / temp folders? I have tried using the "FMEServerResourceConnector", but I either get errors, or it uploads only the url text - and not as a zip file. Resources I looked at:upload .zip file to FME Serverfme server playground - upload file in session As this first post is already 5 years old, I was wondering if there easier way of doing this, without me having diving deep into Rest Apis. I have attached my nextcloud workspace.fmw as well. Thank you for your timeCheers Fred

asked in Flow Hosted

Userparameter datefields missing clickable date selector, in workspace app.

Hello, So this is the second time our FMECloud app looses its clickable date selector for user paramterer datefields. When the soultion ran in test I had tom remove the full application, then republish everything. But I won't do the same approch in production, is there a qick way to get my clickable date selectors back?

posted in Flow Hosted

So today we did an upgrade from FMECloud 2020 to FMECloud 2022, I was a bit nervous because I was afraid some of the apps would break.

So today we did an upgrade from FMECloud 2020 to FMECloud 2022, I was a bit nervous because I was afraid some of the apps would break. Of course there was a reason for my concern, because I had modified FME 2020 to fit our needs. We like to make nice reports with a structured layout, thats why we use 3rd party libraries. These does not come with an FMECloud instance, so you need to do some tricks to install libraries on it. But how do you install libraries on an instance which is made up from containers, wouldn't they dissapear suddenly? Well, there are some persistent storage on FMECloud which you actually use quite frequently. I'm talking about the FME_SHARED_RESOURCE storage. Yup, you can install libraries on this storage. There are 2 ways to achive this, fire up a linux machine with the same os as FME Cloud install a library to a folder, then upload that data to FME SHARED RESOURCES or use the underlying machine to do the task. I choose option 2, I want packages I can use with t

asked in Flow Hosted

run my cloud workspace from html page. Get error: Authentication failed: Account is disabled

I adapted code of Upload File In Session example to call my workspace in cloud. I get error message: "Authentication failed: Account is disabled"What am I missing?

asked in Flow Hosted

How can I retrieve an XML that was sent to an FME Server Workspace which was initiated by a webhook but never run (aborted)?

I have a workspace on FME Server which is initiated by a webhook. There is always an XML that comes with the webhook. The XML is passed into a parameter for the workspace. The workspace parses the XML and processes it into a database. Occasionally, the workspace fails or is aborted. I would like to try to run that workspace again or at the very least extract the XML payload. When the workspace fails (could be because the XML had an incorrect value or there were network issues), is there any way that I can retrieve the XML? Further, and this is probably more critical, if the workspace has been aborted (perhaps it was accidentally cleared from the queue), can I retrieve the XML? Or reprocess the message somehow? If I try to resubmit the failed job again , it doesn't seem to have the XML anymore. Or do I have to send the message again from it's source (Maximo)?

antoineContributor

asked in Flow Hosted

Is it possible to get the billing dashboard info through the API?

In the documentation, i can only get something like :{"balance":0.0,"available_credit":243.67,"estimated_costs":8.99}Is there a way to get as much details as on the dashboard page through the API? If not is there a way to generate a token to call directly the billing dashboard page?

asked in Flow Hosted

How do you edit transformer settings in FME Cloud?

I've uploaded an old FMW workspace for GeoTIF to ECW transformation that was created in FME 2016, but if unchanged it will output the ECW in WGS84, whereas the TIF is GDA2020. How can I update the ECW coordinate system for the output in FME Cloud?

asked in Flow Hosted

FME Cloud / Server performance

Hi there, UPDATE 14th Feb 2022: I have now added my workbenches, though not sure how much more insight they will provide - I am wondering if it is the fact that an 'umbrella' process is running and taking a long time and therefore using up a lot of processing? Or if it is simply downloading a lot of zip files and then loading them into the database in a linear fashion that makes it take such a long time.I have also now upgraded my FME Cloud instance to 'Professional' - m5.xlarge, 4 CPUs, 16 GB RAM and 100GB temporary disk space - this did show an initial improvement (see additional graph data 14th Feb), but it is still averaging 00:02:15 per file, so will take 11 days just to download and upload 7,00+ files - this doesn't seem right to me?!Any pointers very gratefully received. And thanks again. I have an FME Server running on a standard FME Cloud (8 GB RAM, 2 CPUs, 25 GB Temp disk space) I am trying to understand the best set up of my workbenches / FME Cloud to en

peteratqfesContributor

asked in Flow Hosted

Error using AttributeFileWriter and FeatureReader on FMECloud - Unfinished file?

I have a workbench which is writing out a zip file containing a shapefile using AttributeFileWriter then using a FeatureReader on the resulting zip file. It works fine in FME Desktop 2021 writing to a temp directory. When I run on FMECloud using the Temp resource folder I get an error when reading the zip file. If I download the zip file from the resource folder the file is valid. It's like the file is being held onto by the AttributeFileWriter (or it hasn't finished writing).I don't get the two warnings below in FME Desktop. It seems to imply the files haven't finished being written out. Is there any way to avoid this unfinished writing, or is there any way to take the binary shapefile zip data and read that directly without writing out to file?It might be possible to break out the workbench into multiple workbenches in the automation. This might happen on FME Server as well as FMECloud (I haven't tried it yet)

asked in Flow Hosted

featureWriter returns different output structure in FME Cloud

Hello. my script in Workbench provides nicely zipped result from FeatureWriter. one excel sheet and one shapefile, named using User Parameter ("someID") When I publish this to FME Cloud, the structure of resulting zipped is completely mangled, and the user attribute value is not provided to the final file names (SomeID is fallback default value)(picture shows the flat view, each zip is in one of those folders) Can this be fixed so the result output of Cloud workflow has the same structure as the Workbench result? if I set the publish parameters in following way, the final zip file contains only one of the result files, either SHP or XLS

asked in Flow Hosted

Elevation numbers exist, but output visually empty in ArcMap?

I am very new to the FME tool, I downloaded it to change my LAZ files to LAS. I have gone through a few tutorials, and my output is not empty. I have LAS point elevations, but visually it is empty. I have a big reg square, representing the outline of the LAZ tile I downloaded, located in the correct place on earth. There is nothing in the red square. Where do I go from here?

asked in Flow Hosted

Open a file on FME_SHARED_RESOURCE_DATA Directory with pythonCaller

Hello, I want to open a file with the PythonCaller stored on the [FME_SHARERESOURCE_DATA] directory. The python enviroment can access the directory and the script end with an exception:TypeError: expected str, bytes or os.PathLike object, not NoneType Is it possible to map the FME_SHARED_RESOURCE_DATA Directory in the python script? Kind regardsStefan

itayContributor

asked in Flow Hosted

Using POST and body to trigger FME Server workspace.

Hi There,I am pretty sure what the answer is, just covering all bases....Normally when running a workspace published to FME Server the workspace parameters are populated via the query parameters and therefore the workspace URL contains the parameters provided. Registrating the workspace to the streaming service and defining the streaming output content, allows to provide the requesting application with the transformation results. Another way to run workspaces on FME Server is via the REST API but this does not return the workspace output to the requesting application, since the streaming service is not used. Now my question is: Is it possible to trigger a published workspace by posting the parameters values in the body and NOT in the URL while using the streaming service to return the transformation results to the requesting application? so using a POST + body and getting the transformation results and NOT the job id (async) or job transformation status (synch) Does naybody know if thi

1
2
3
4

Page 2 / 4

Badge winners

lancehas earned the badge Teacher (Bronze)
lancehas earned the badge Teacher (Grey)
nordpilhas earned the badge Enchanted by Attributes
nordpilhas earned the badge I Treasure Lists
tvahas earned the badge Teacher (Bronze)

Show all badges

28,732: Posts

Powered by Gainsight

Terms & Conditions Cookie settings

Sign up

Already have an account? Login

An FME Account is required to contribute

Login

Login to the community

An FME Account is required to contribute

Login

Enter your username or e-mail address. We'll send you an e-mail with instructions to reset your password.

Username or e-mail

Back to overview

Scanning file for viruses.

Sorry, we're still checking this file's contents to make sure it's safe to download. Please try again in a few minutes.

OK

This file cannot be downloaded

Sorry, our virus scanner detected that this file isn't safe to download.

OK

A product of

Company Products Giving Back Careers Contact Feedback

Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.