Skip to main content
Solved

FME Server 2018.0: Is it intentional that Session ID becomes User Name?

takashi
Evangelist

FME 2018.0 build 18284

In the object returned from the FME.getSession function, the value of "session" becomes user name who logged in the server. Is it intentional?

App:

<!DOCTYPE html>
<html>
<head>
<title>FME Server Session ID</title>
</head>
<body onload="onBodyLoad();">
<script type="text/javascript" src="http://api.fmeserver.com/js/v1.1/FMEServer.js"></script>
<script type="text/javascript">
function onBodyLoad() {
    var host = 'http://*************.com';
    var token = '****************************************';
    var repository = 'Tests'; // example
    var workspace = 'fme-parameters.fmw'; // example
    FMEServer.init({server: host, token: token});
    FMEServer.getSession(repository, workspace, function (obj) {
        alert(JSON.stringify(obj));
    });
}
</script>
</body>
</html>

Returned JSON (formatted): "session" = "superuser" (user name)

{
    "serviceResponse":{
        "statusInfo":{
            "status":"success"
        },
        "session":"superuser",
        "files":{
            "path":"",
            "folder":[
                {
                    "path":"$(FME_DATA_REPOSITORY)/Tests/fme-parameters.fmw/superuser","name":""
                }
            ]
        }
    }
}

Best answer by jlutherthomas

This issue has now been fixed in 2018.0.0.1 - Build 18292+

The data upload service now accepts a new parameter, "opt_namespace". If this parameter is specified, uploads will be placed in a folder named after the value of opt_namespace. If the parameter is unspecified, uploads will be placed in a folder named after the username of the uploading user. If a user has a created a third party application that uses a single token to connect to fme server, it should be recommended that they supply a random number or guid to the opt_namespace parameter. The FMEServer.js library has been updated to support this change on server: http://api.fmeserver.com/js/v1.2/FMEServer.js
View original
Did this help you find an answer to your question?

13 replies

takashi
Evangelist
  • takashiEvangelist
  • Author
  • March 21, 2018
In fact, I created a web application (JavaScript script) which accepts file upload and submits a job, based on a premise that the Session ID is unique for each session. The function looks like this.

 

function uploadAndSubmitJob(repository, workspace, fileInput) {
    FMEServer.getSession(repository, workspace, callbackGetSession);
    
    function callbackGetSession (obj) {
        var session = obj.serviceResponse.session;
        FMEServer.dataUpload(repository, workspace, fileInput, session, callbackDataUpload);
    }
    
    function callbackDataUpload (obj) {
        var session = obj.serviceResponse.session;
        var filename = fileInput.value.replace(/^.*[\\/]/, '');
        var parameters = {
            "publishedParameters" : [
                {
                    "name" : "SOURCE_DATASET",
                    "value" : "$(FME_DATA_REPOSITORY)/<repository>/<workspace>/<session>/<filename>"
                }
            ]
        };
        FMEServer.submitJob(repository, workspace, parameters, function (obj) { });
    }
}
With FME Server 2017.1.2.1, it worked fine since Session ID returned from the FME.getSession function was unique. This is a response of the FME.getSession from FME Server 2017.1.2.1.

 

Session ID = 5A73001A453ECFFF50F4AD2F50539545

 

{
    "serviceResponse":{
        "statusInfo":{
            "status":"success"
        },
        "session":"5A73001A453ECFFF50F4AD2F50539545",
        "files":{
            "path":"",
            "folder":[
                {
                    "path":"$(FME_DATA_REPOSITORY)/<repository>/<workspace>/5A73001A453ECFFF50F4AD2F50539545",
                    "name":""
                }
            ]
        }
    }
}
However, in FME 2018.0+, this could cause a wrong result or an error, since the Session ID is equal to User Name (is not unique for each session). I will have to fix the application if the new behavior (Session ID = User Name) was really correct.

 

It's a very critical issue for my current project. Please confirm whether the new behavior is correct as soon as possible.

 

J
Forum|alt.badge.img+2

Hi @takashi

I have a feeling this is intentional because of some changes coming later in 2018 but I will confirm this for you and update.

takashi
Evangelist
  • takashiEvangelist
  • Author
  • March 21, 2018
jlutherthomas wrote:

Hi @takashi

I have a feeling this is intentional because of some changes coming later in 2018 but I will confirm this for you and update. 

Thanks for your response.

 

The application should allow multiple clients to access the FME Server with the same user account and run a job with an uploaded file independently.

 

In FME 2017.1.2.1, Session ID was unique for each session and the uploaded file will be saved in a directory specific to the session, so I was able to use the session ID to distinguish individual uploads.

 

However, in FME 2018.0, since Session ID is always equal to the user name and all the uploaded files from the same user account will be saved into the same directory (folder name = user name), the application cannot distinguish individual uploads.

 

In my test with this simple application and FME 2018.0, the object returned from the FMEServer.dataUpload function contains an array which contains the information on all files saved in a folder with the session name (i.e. user name) at the time.

 

How can you select which file has been uploaded from the current session in this client?

 

Or, am I missing something?

 

<!DOCTYPE html>
<html>
<head><title>FME Server File Upload</title></head>
<body onload="onLoadBody();">
<div>
  <input id="selectFile" type="file" onchange="uploadFile(this);"></input>
</div>
<script type="text/javascript" src="http://api.fmeserver.com/js/v1.1/FMEServer.js"></script>
<script type="text/javascript">
var host = 'http://**********';
var token = '***************************************';
var repository = "Tests";
var workspace = "fme-parameters.fmw";
    
function onLoadBody() {
    FMEServer.init({server: host, token: token});
}

function uploadFile(fileInput) {
    FMEServer.getSession(repository, workspace, function (obj) {
        var session = obj.serviceResponse.session;
        //var session = 'RANDOM_VALUE';
        // Even if you assign a random value to session, folder name becomes user name.
        FMEServer.dataUpload(repository, workspace, fileInput, session, function (obj) {
            alert(JSON.stringify(obj));
        });
    });
}
</script>
</body></html><br>
Even if you uploaded just a single file, the response contains the information on all the files saved in the "session" (i.e. user name) folder at the time. You cannot control the folder (session) name from the application. There seems to be no way to distinguish which file has been uploaded from which session / client.

{
    "serviceResponse":{
        "statusInfo":{
            "status":"success"
        },
        "session":"superuser",
        "files":{
            "path":"",
            "file":[
                {
                    "size":983,"name":"datadownload_response_json_template.txt"
                },
                {
                    "size":1168,"name":"datetimeconverter-autodetect_log.ffs"
                },
                {
                    "size":45518,"name":"fme-parameters.fmw"
                },
                {
                    "size":59238,"name":"path-reader-unc.log"
                },
                {
                    "size":9479,"name":"sample2.html"
                },
                {
                    "size":237,"name":"subscriber.json"
                }
            ],
            "folder":[
                {
                    "name":""
                }
            ]
        }
    }
}
takashi
Evangelist
  • takashiEvangelist
  • Author
  • March 21, 2018
jlutherthomas wrote:

Hi @takashi

I have a feeling this is intentional because of some changes coming later in 2018 but I will confirm this for you and update.

I think it also causes a problem that a file uploaded into the "user name" folder would be overwritten, if another client with the same user account would upload a file having different content but the same name as the file existing in the folder. Such a conflict cannot be resolved unless the folder name would be unique for each session.

 

 

takashi
Evangelist
  • takashiEvangelist
  • Author
  • March 22, 2018
jlutherthomas wrote:

Hi @takashi

I have a feeling this is intentional because of some changes coming later in 2018 but I will confirm this for you and update.

Perhaps this one?

 

==============================

 

FME Server 2018.0 "What's New"

 

==============================

 

FME 2018.0 b18263 20180206

 

--------------------------

 

FME Server Web Services: Added support for Authorization via header. Namespace dataupload service uploads by username instead of jsessionID (PR#81344)

 

I don't know why this change was necessary, and also I'm afraid that this could cause serious side effects against existing applications using data upload service in conjunction with job submit service.

 

J
Forum|alt.badge.img+2
jlutherthomas wrote:

Hi @takashi

I have a feeling this is intentional because of some changes coming later in 2018 but I will confirm this for you and update.

 

@takashi These changes were made to implement fault tolerance in FME Server, but I've passed on your comments to development.

 

This might be worth submitting a support case so that we can record and raise it officially with all of your feedback
takashi
Evangelist
  • takashiEvangelist
  • Author
  • March 22, 2018
jlutherthomas wrote:

Hi @takashi

I have a feeling this is intentional because of some changes coming later in 2018 but I will confirm this for you and update.

Thanks for your consideration. In the interim, I downgrade the FME Server for the project to FME 2017.1.2.1.

 

 

takashi
Evangelist
  • takashiEvangelist
  • Author
  • March 22, 2018
jlutherthomas wrote:

Hi @takashi

I have a feeling this is intentional because of some changes coming later in 2018 but I will confirm this for you and update.

I submitted a support case #C135583.
takashi
Evangelist
  • takashiEvangelist
  • Author
  • March 25, 2018

I looked at the source of the "Easy Translator" demo to learn how I can identify a file uploaded in a session.

Easy Translator | FME Server Demos

In my understanding, the app seems to identify selected files with their names simply. If the directory path was unique, the app works fine independently for each session, even if multiple clients (sessions) access at the same time.

However, in FME 2018, session ID is equal to user name and therefore directory path is not unique. I checked the fact with this demo too: Upload File In Session | FME Server Playground

This could cause a conflict in some scenarios. For example, a case where two clients - "A" and "B" operate the "Easy Translator" demo in the following order.
  1. "A" uploads "sample.zip" which contains a Shapefile dataset, and sets "Esri Shape" to the Source Format parameter.
  2. "B" uploads "sample.zip" which contains a Mapinfo TAB dataset. The file name is the same as "A"'s, but the contents are different.
  3. "A" selects the "sample.zip", submits the job, then clicks "Download Results"... oops!

You can reproduce the symptom easily. Try the scenario with the attached sample files: samples.zip

I think Session ID should be unique and the directory path should contain the unique Session ID, in order to allow any application to process uploaded files independently for each session.

J
Forum|alt.badge.img+2

This issue has now been fixed in 2018.0.0.1 - Build 18292+

The data upload service now accepts a new parameter, "opt_namespace". If this parameter is specified, uploads will be placed in a folder named after the value of opt_namespace. If the parameter is unspecified, uploads will be placed in a folder named after the username of the uploading user. If a user has a created a third party application that uses a single token to connect to fme server, it should be recommended that they supply a random number or guid to the opt_namespace parameter. The FMEServer.js library has been updated to support this change on server: http://api.fmeserver.com/js/v1.2/FMEServer.js
takashi
Evangelist
  • takashiEvangelist
  • Author
  • April 4, 2018
jlutherthomas wrote:

This issue has now been fixed in 2018.0.0.1 - Build 18292+

The data upload service now accepts a new parameter, "opt_namespace". If this parameter is specified, uploads will be placed in a folder named after the value of opt_namespace. If the parameter is unspecified, uploads will be placed in a folder named after the username of the uploading user. If a user has a created a third party application that uses a single token to connect to fme server, it should be recommended that they supply a random number or guid to the opt_namespace parameter. The FMEServer.js library has been updated to support this change on server: http://api.fmeserver.com/js/v1.2/FMEServer.js
Good to hear. I'll test the new library.

 

Thanks for the quick fix.

 

takashi
Evangelist
  • takashiEvangelist
  • Author
  • April 5, 2018

I tested the behavior of getSession and dataUpload in the FMEServer.js Library Version 1.2 and FME Server 2018.0.0.1 build 18295, with this script.

<!DOCTYPE html>
<html>
<head><title>FME Server File Upload</title></head>
<body onload="onLoadBody();">
<div>
  <input id="selectFile" type="file" onchange="uploadFile(this);"></input>
</div>
<div>
    <table>
        <thead><th>Method</th><th>Session ID</th><th>Directory Path</th></thead>
        <tbody id="result" />
    </table>
</div>
<script type="text/javascript" src="http://api.fmeserver.com/js/v1.2/FMEServer.js"></script>
<script type="text/javascript">
var host = 'http://**********************************'; // Replace with your FME Server URL.
var token = '****************************************'; // Replace with you token.
var repository = "Tests"; // Replace with a valid repository name in the server.
var workspace = "fme-parameters.fmw"; // Replace with a valid workspace name in the repository.
    
function onLoadBody() {
    FMEServer.init({server: host, token: token});
}

function uploadFile(fileInput) {
    FMEServer.getSession(repository, workspace, function (obj) {
        var res = obj.serviceResponse;
        appendResultRow('getSession', res.session, res.files.folder[0].path);

        // Option 1: Use Session ID returned by the getSession method.
        var session = res.session;
        
        // Option 2: Use a string value defined in the client application.
        // var session = 'Custom Session ID';
        
        // Option 3: Use the null or the empty string.
        // var session = null;        
        
        FMEServer.dataUpload(repository, workspace, fileInput, session, function (obj) {
            appendResultRow('dataUpload', obj.serviceResponse.session, '');
        });
    });
    
    function appendResultRow(method, session, path) {
        var data = [method, session, path];
        var tr = document.createElement('tr');
        for (var i = 0; i < data.length; i++) {
            var td = document.createElement('td');
            td.innerText = data[i];
            tr.appendChild(td);
        }
        document.getElementById('result').appendChild(tr);
    }
}
</script>
</body></html>

In my observation, when the getSession method is called, the FME Server generates a unique identifier (session ID) of 10 digits and creates this directory.

$(FME_DATA_REPOSITORY)/<repository>/<workspace>/<session ID>

Then, the response from the getSession method contains the session ID and the directory path.

Regarding the dataUpload method, there seems to be three options on the 'session' parameter setting.

  1. Pass the session ID returned by the getSession method. The uploaded file will be saved into the directory which has been created when the getSession method was called. i.e. "$(FME_DATA_REPOSITORY)/<repository>/<workspace>/<session ID>"
  2. Specify a string value defined in the client application. The uploaded file will be saved into a directory "$(FME_DATA_REPOSITORY)/<repository>/<workspace>/<specified string>".
  3. Set the null or the empty string. The uploaded file will be saved into a directory "$(FME_DATA_REPOSITORY)/<repository>/<workspace>/<current FME Server user name>".

Is my observation above correct?

takashi
Evangelist
  • takashiEvangelist
  • Author
  • April 9, 2018
jlutherthomas wrote:

This issue has now been fixed in 2018.0.0.1 - Build 18292+

The data upload service now accepts a new parameter, "opt_namespace". If this parameter is specified, uploads will be placed in a folder named after the value of opt_namespace. If the parameter is unspecified, uploads will be placed in a folder named after the username of the uploading user. If a user has a created a third party application that uses a single token to connect to fme server, it should be recommended that they supply a random number or guid to the opt_namespace parameter. The FMEServer.js library has been updated to support this change on server: http://api.fmeserver.com/js/v1.2/FMEServer.js
FME 2018.0.0.1, the new parameter 'opt_namespace' and the new FMEServer.js library version 1.2 seem to work as expected. I upgraded all FME Servers under my management to 2018.0.0.1 build 18295.

 

Reply


Helpful Members This Week

Community Stats

31,468
Posts
119,284
Replies
39,083
Members

Latest FME

Terms & ConditionsCookie settingsAccessibility statement

Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.