Question

FME Cloud upload

jatoxa
Rank
Badge +2

Hi,

 

 

I have a diffcult time grasping the differences between FME Cloud and FME Server.

 

Mainly I want to use REST API to interact with FME Cloud via web forms but I can't seem to get anything to work 100%.

 

 

I trying to duplicate some of the upload examples on the demo page and it seems I must be missing something. https://playground.fmeserver.com/demos/

 

16 replies

redgeographics
Rank
Userlevel 5
Badge +25

Can you elaborate a bit more? Which of the demo's are you trying to run? Are you getting any specific errors?

jatoxa
Rank
Badge +2

Can you elaborate a bit more? Which of the demo's are you trying to run? Are you getting any specific errors?

Yes, sorry for being for so unspecific.

 

 

My main question is whether the same REST commands exist for both Server and Cloud?

 

 

I have tried to modify all of the upload demos "EasyTranslator", "GML Validator", "CAD validator" to run on Cloud with no luck. Then I tried installing an FME Server demo on my home computer and it seems to work when running locally.

 

 

 

 

 

redgeographics
Rank
Userlevel 5
Badge +25
Yes, sorry for being for so unspecific.

 

 

My main question is whether the same REST commands exist for both Server and Cloud?

 

 

I have tried to modify all of the upload demos "EasyTranslator", "GML Validator", "CAD validator" to run on Cloud with no luck. Then I tried installing an FME Server demo on my home computer and it seems to work when running locally.

 

 

 

 

 

The samples do need some editing before they will run on your FME Cloud. I'm afraid I don't have a full list of what needs to be done, but taking the EasyTranslator sample:

 

  • Change ./js/resources.js to use your FME Cloud instance. It actually refers to a JSON file in Safe's AWS bucket, you'll need to make your own copy and alter it to reflect your environment. For that you'll also need an FME Cloud token.
  • You will need to drop the HTML/Javascript etc on some webhost, you can't host them off of FME Cloud directly.
  • You'll probably need to add that host to the CORS section of your FME Cloud System Configuration settings.
There will most likely be more steps necessary, and then there might be version issues because most of those demo's were made on FME 2013/2014 and we're at 2018 now of course.

 

 

Which kinda makes me wonder... you say you managed to get it working on your FME Server, but can you doublecheck the jobs list and confirm the jobs actually ran on your server?
jatoxa
Rank
Badge +2

Can you elaborate a bit more? Which of the demo's are you trying to run? Are you getting any specific errors?

I have already followed all of those steps in order to make it work and here the main problem is that it says the Server is not defined.

 

Yes, the jobs actually run on the server at home although I get some other issues with the upload script itself.

 

 

Taking a much simpler example https://playground.fmeserver.com/javascript/server-resources/upload-file/

 

How would I go about making it work on Cloud?

 

 

 

 

redgeographics
Rank
Userlevel 5
Badge +25
I have already followed all of those steps in order to make it work and here the main problem is that it says the Server is not defined. 

 

Yes, the jobs actually run on the server at home although I get some other issues with the upload script itself.

 

 

Taking a much simpler example https://playground.fmeserver.com/javascript/server-resources/upload-file/ 

 

How would I go about making it work on Cloud?

 

 

 

 

This:

 

window.onload = function() {
	FMEServer.init({
		server : "https://demos-safe-software.fmecloud.com",
		token : "568c604bc1f235bbe137c514e7c61a8436043070"
		});
	};
That should be the url to your server/cloud instance and the token you generated there.
jatoxa
Rank
Badge +2
This:

 

window.onload = function() {
	FMEServer.init({
		server : "https://demos-safe-software.fmecloud.com",
		token : "568c604bc1f235bbe137c514e7c61a8436043070"
		});
	};
That should be the url to your server/cloud instance and the token you generated there.
Yes, I've already tried it and configured CORS but still no luck.

 

 

jatoxa
Rank
Badge +2
This:

 

window.onload = function() {
	FMEServer.init({
		server : "https://demos-safe-software.fmecloud.com",
		token : "568c604bc1f235bbe137c514e7c61a8436043070"
		});
	};
That should be the url to your server/cloud instance and the token you generated there.
It seems the simple upload works since files are actually uploaded to the server. However the browser gives a list of errors 

 

"Uncaught TypeError: Cannot read property 'insertBefore' of null    at showResults ((index):46)    at XMLHttpRequest.req.onreadystatechange (FMEServer.js:183)"

 

 

G
Badge

Hi @j2

Which version of FME Server are you using on FME Cloud? There was a change regarding the data upload service in 2018.0 which affects the data upload Demos. I switched the examples on the Playground to a new version of the FME Server JS API just now. Please make sure that you use the FME Server JS API v1.2 with FME Server 2018.0. For older versions of FME Sever please use the version v1.1 .

Please let me know if this helped in your case.

jatoxa
Rank
Badge +2

Hi @j2

Which version of FME Server are you using on FME Cloud? There was a change regarding the data upload service in 2018.0 which affects the data upload Demos. I switched the examples on the Playground to a new version of the FME Server JS API just now. Please make sure that you use the FME Server JS API v1.2 with FME Server 2018.0. For older versions of FME Sever please use the version v1.1 .

Please let me know if this helped in your case.

Hi GerhardAtSafe

 

I'm using the newest version of Cloud and it seems I get a little further now.

 

 

I have tried recreating this workflow http://demos.fmeserver.com/demo-validation-direct/

 

except I just need to upload and run a workspace.

 

Seems to work except I get a "422 error" - "2018-05-17 08:49:23| 0.2| 0.0|FATAL |PDF Reader: Failed to open document '/data/fmeserver/resources/system/temp/upload/Validation/2018_PDF_NSM_datareader_001_CLOUD.fmw/6868125341/A-45-4-57-0041.pdf' because it does not exist or because the reader does not have permission to access the file".

 

 

In index.html and CadValidation.js I have changed to FME Server API 1.2, then updated host and token to my own instance on cloud and changed the SourceDataset in the submitURL variable.

 

 

 

takashi
Rank
Userlevel 2
Badge +17

Hi @j2

Which version of FME Server are you using on FME Cloud? There was a change regarding the data upload service in 2018.0 which affects the data upload Demos. I switched the examples on the Playground to a new version of the FME Server JS API just now. Please make sure that you use the  FME Server JS API v1.2 with FME Server 2018.0. For older versions of FME Sever please use the version  v1.1 .

Please let me know if this helped in your case.

Hi @j2, if the version of FME Server is 2018.0.0.1+ and the version of the FMEServer.js library is 1.2, probably you will have to add the opt_namespace parameter with the value of 'BuildForm.session' to the upload [corrected] request URL. However, I don't know how to modify the JavaScript code since I'm not familiar with jQuery unfortunately. @GerhardAtSafe, please check this.

 

----------The opt_namespace parameter has been newly introduced in FME 2018.0.0.1. See the FME Server 2018.0 "What's New".
--------------------------FME 2018.0 b18293 20180326--------------------------<br>FME Server DataUpload: Added new optional parameter, opt_namespace to use custom namespace (PR#82853)<br>
----------

 

I think the retirement of jsessionid, the introduction of opt_namespace and the upgrade of FMEServer.js library are important information for developers. I strongly hope relevant documentations and demos would be updated appropriately.
G
Badge
Yes, I've already tried it and configured CORS but still no luck.

 

 

To follow up on some of your questions:

 

The FME Server REST API is available on any on-premise FME Server & on FME Server running on a Cloud instance.

 

@redgeographics pointed out correctly that for all code examples on https://playground.fmeserver.com you only need to change the server URL and the FME Server token. You can read more about how to obtain an FME Server token here. As a start, you can also just enter your server URL & token directly in the code examples and refresh the output.
G
Badge
Hi @j2, if the version of FME Server is 2018.0.0.1+ and the version of the FMEServer.js library is 1.2, probably you will have to add the opt_namespace parameter with the value of 'BuildForm.session' to the upload [corrected] request URL. However, I don't know how to modify the JavaScript code since I'm not familiar with jQuery unfortunately. @GerhardAtSafe, please check this.

 

----------The opt_namespace parameter has been newly introduced in FME 2018.0.0.1. See the FME Server 2018.0 "What's New".
--------------------------FME 2018.0 b18293 20180326--------------------------<br>FME Server DataUpload: Added new optional parameter, opt_namespace to use custom namespace (PR#82853)<br>
----------

 

I think the retirement of jsessionid, the introduction of opt_namespace and the upgrade of FMEServer.js library are important information for developers. I strongly hope relevant documentations and demos would be updated appropriately.
The opt_namespace parameter is already part of FME Server JS API v1.2, hence the recommendation to use this version with FME Server 2018.0. When using the JS API the opt_namespace parameter is replacing the jsessionID to keep the functionality. There is also documentation for the new parameter available in the 2018.1 doc.

 

 

takashi
Rank
Userlevel 2
Badge +17
Hi @j2, if the version of FME Server is 2018.0.0.1+ and the version of the FMEServer.js library is 1.2, probably you will have to add the opt_namespace parameter with the value of 'BuildForm.session' to the upload [corrected] request URL. However, I don't know how to modify the JavaScript code since I'm not familiar with jQuery unfortunately. @GerhardAtSafe, please check this.

 

----------The opt_namespace parameter has been newly introduced in FME 2018.0.0.1. See the FME Server 2018.0 "What's New".
--------------------------FME 2018.0 b18293 20180326--------------------------<br>FME Server DataUpload: Added new optional parameter, opt_namespace to use custom namespace (PR#82853)<br>
----------

 

I think the retirement of jsessionid, the introduction of opt_namespace and the upgrade of FMEServer.js library are important information for developers. I strongly hope relevant documentations and demos would be updated appropriately.
Hi @GerhardAtSafe, in my understanding, with FME Server 2018.0.0.1+ and FMEServer.js version 1.2,
  • FMEServer.getSession method generates a unique namespace (10 digits) automatically, calls the REST upload API with setting the namespace to the opt_namespace parameter, then the API returns the namespace as session ID.
  • If you call the FMEServer.dataUpload method with the session ID, files will be uploaded to a folder named as the session ID, since the method calls the REST upload API with setting the session ID to the opt_namespace parameter.
  • However, if you want to upload files using the raw REST upload API into the session ID folder, you will have to specify the session ID via the opt_namespace parameter explicitly.
If my understanding above was correct, I think the JavaScript source of the demo "CAD Validation/Quality Assurance" (which @j2 is trying to replicate) should be modified to specify the opt_namespace explicitly when uploading a file if you would upgrade the version of  FMEServer.js to 1.2.

 

Since the demo is still using FMEServer.js version 1.1 currently, any problem has not appeared yet, but @j2 has changed the library version to 1.2 according to your suggestion. I therefore think that this error occurred when the datastreaming request was submitted.

 

----------

 

"422 error" - "2018-05-17 08:49:23| 0.2| 0.0|FATAL |PDF Reader: Failed to open document '/data/fmeserver/resources/system/temp/upload/Validation/2018_PDF_NSM_datareader_001_CLOUD.fmw/6868125341/A-45-4-57-0041.pdf' because it does not exist or because the reader does not have permission to access the file"

 

takashi
Rank
Userlevel 2
Badge +17
The opt_namespace parameter is already part of FME Server JS API v1.2, hence the recommendation to use this version with FME Server 2018.0. When using the JS API the opt_namespace parameter is replacing the jsessionID to keep the functionality. There is also documentation for the new parameter available in the 2018.1 doc.

 

 

@GerhardAtSafe, thanks for updating the help on Data Upload Service in FME 2018.1. I think it would be better if the FME Server version in which opt_namespace is applicable was clarified and the description on the jsesssionid could also be remained for the users who are using FME Server 2017 and earlier
G
Badge
Hi @GerhardAtSafe, in my understanding, with FME Server 2018.0.0.1+ and FMEServer.js version 1.2,
  • FMEServer.getSession method generates a unique namespace (10 digits) automatically, calls the REST upload API with setting the namespace to the opt_namespace parameter, then the API returns the namespace as session ID.
  • If you call the FMEServer.dataUpload method with the session ID, files will be uploaded to a folder named as the session ID, since the method calls the REST upload API with setting the session ID to the opt_namespace parameter.
  • However, if you want to upload files using the raw REST upload API into the session ID folder, you will have to specify the session ID via the opt_namespace parameter explicitly.
If my understanding above was correct, I think the JavaScript source of the demo "CAD Validation/Quality Assurance" (which @j2 is trying to replicate) should be modified to specify the opt_namespace explicitly when uploading a file if you would upgrade the version of FMEServer.js to 1.2.

 

Since the demo is still using FMEServer.js version 1.1 currently, any problem has not appeared yet, but @j2 has changed the library version to 1.2 according to your suggestion. I therefore think that this error occurred when the datastreaming request was submitted.

 

----------

 

"422 error" - "2018-05-17 08:49:23| 0.2| 0.0|FATAL |PDF Reader: Failed to open document '/data/fmeserver/resources/system/temp/upload/Validation/2018_PDF_NSM_datareader_001_CLOUD.fmw/6868125341/A-45-4-57-0041.pdf' because it does not exist or because the reader does not have permission to access the file"

 

Hi @takashi,

 

thanks for pointing that out! I wasn't aware that the CAD Validation Demo was using the data upload service outside of the JS API. In this case, you are correct & this might be causing the issue @j2 is seeing. I will add a task to update this Demo to improve the experience for the future. Thanks!
takashi
Rank
Userlevel 2
Badge +17
Hi @takashi,

 

thanks for pointing that out! I wasn't aware that the CAD Validation Demo was using the data upload service outside of the JS API. In this case, you are correct & this might be causing the issue @j2 is seeing. I will add a task to update this Demo to improve the experience for the future. Thanks!
I'm not familiar with jQuery scripting manner, so I cannot provide a solution to @j2's case. Could you please provide how the script should be modified for FMESever.js version 1.2?

 

Reply


Community Stats

28,669
Posts
109,357
Replies
36,531
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.