Solved

Problem with notification that is not "activated"

lau
Badge +2

Hi all,

I encounter an issue with topic on FME Server:

I have one workspace (the controller) that triggers another (the worker) via a jobsubmitter.

The worker posts on the topic "reporter_database" in case of success. It sends some json code.

Case 1: When I run the worker via the controller, the topic is not "activated".

Case 2: If I launch the worker directly via the web browser interface, it runs correctly and activates the topic.

I remarked there was a difference between the two approaches (see picture). On the rigth (case2), there are some FME Directives and Notification Manager Directives while there is nothing on case1...

Does someone have an idea how I can solve the case 1?

Thank you !

Laurent

icon

Best answer by larry 16 June 2016, 16:13

View original

13 replies

david_r
Rank
Userlevel 4

Hi

You should use the FMEServerNotifier after the FMEServerJobSubmitter. You will find the json data from the child workspace referenced in the output_data{} list that is returned by the FMEServerJobSubmitter (attention: this list is only returned when the master workspace is running on FME Server!).

David

lau
Badge +2

Hi David,

Thank you for your answer. Unfortunately, my jobsubmitters are set to "wait for the job"= No. So I think your method won't work...

Do you know if there is another possibility? Maybe I can use the Shutdown Python Script? Is there a way to know in the python code if the translation was succesful?

david_r
Rank
Userlevel 4

Hi David,

Thank you for your answer. Unfortunately, my jobsubmitters are set to "wait for the job"= No. So I think your method won't work...

Do you know if there is another possibility? Maybe I can use the Shutdown Python Script? Is there a way to know in the python code if the translation was succesful?

Yes, you're right, you will have to set Wait for job = Yes for this to work. Any particular reason for not being able to?

Anything is possible using a Python script, but it's going to get fiddly unless you're already comfortable writing Python code.

There are a couple of techniques for passing workspace variables to a shutdown script, the easiest is to use Python global variables.

lau
Badge +2

I found another workaround. With my worker, I can create the json file in a separate folder. I can add a directory watch to this folder that posts to the topic when a modification is done in the folder... Not really fine but it should work...

If someone has a best idea don't hesitate to share. Thanks!

jpvo
Badge

Hi lau,

I could reproduce the problem. It sounds odd to me:

In the first image I start a worker via UI, the job starts and post on the TOPIC the right JSON information. If I trigger the same JSONPOST.fmw via a transformer fmeserverJobSubmitter the topic never gets activated.

WHY?

FME Server 2015.1.3.1 - Build 15573 - win64

 

Copyright (c) 1994 - 2015, Safe Software Inc.

Thanks for any help.

M

Hi Laurant @lau

You say...

Case 1: When I run the worker via the controller, the topic is not "activated".

My question is, how are you running the controller? Are you running it on FME Desktop (ie within Workbench) or are you running it in the FME Server web interface?

The difference is that if you run the controller in FME Workbench, then it runs immediately, but workers get put into a queue. If you run the controller in FME Server then the controller is put into the queue, but workers are run immediately.

That difference affects how directives are applied. I don't know, but it might affect notifications too. So I'd be interested to see how you (and @jpvo) are running the controller and if there is a difference if you run it a different way.

Regards

Mark

lau
Badge +2

Hi Laurant @lau

You say...

Case 1: When I run the worker via the controller, the topic is not "activated".

My question is, how are you running the controller? Are you running it on FME Desktop (ie within Workbench) or are you running it in the FME Server web interface?

The difference is that if you run the controller in FME Workbench, then it runs immediately, but workers get put into a queue. If you run the controller in FME Server then the controller is put into the queue, but workers are run immediately.

That difference affects how directives are applied. I don't know, but it might affect notifications too. So I'd be interested to see how you (and @jpvo) are running the controller and if there is a difference if you run it a different way.

Regards

Mark

Hi Mark,

Thank you for your answer.

I tried to run the controller both on server and desktop. The result is the same.

I also tried to post on a topic on failure. I get the same observations. When I run the controller, the topic is not activated. When I run directly the runner, the topic is activated and an email is sent.

It's really annoying because if I have a workspace that crashes I cannot receive an email... Do you have a suggestion on how I can workaround that?

Thanks in advance.

Laurent

M

Hi Mark,

Thank you for your answer.

I tried to run the controller both on server and desktop. The result is the same.

I also tried to post on a topic on failure. I get the same observations. When I run the controller, the topic is not activated. When I run directly the runner, the topic is activated and an email is sent.

It's really annoying because if I have a workspace that crashes I cannot receive an email... Do you have a suggestion on how I can workaround that?

Thanks in advance.

Laurent

I'm afraid I don't, and nothing I can find in our databases explains what is happening. So I have asked our server support team to take a look at this question and see if they can help. Apologies for any inconvenience. I hope they get in touch soon.

fmelizard
Rank
Userlevel 4
Badge +13

I found another workaround. With my worker, I can create the json file in a separate folder. I can add a directory watch to this folder that posts to the topic when a modification is done in the folder... Not really fine but it should work...

If someone has a best idea don't hesitate to share. Thanks!

I would really encourage you to look at setting the FMEServerJobSubmitter transformer to wait or discuss it with us, but here is an alternative...

You can use an HTTPCaller transformer to run the workspace using the Job Submitter web service. The URL to the web service can be found from the Web UI > Run page for your workspace. This is a less standard approach, but it would result in the exact same format of JSON being passed to the topic.

I hope this helps. Thanks for contributing to the FME community.

L
Badge

If I understand correctly, I had to work on a similar issue a few days ago.

My problem was that my workspace is published to the jobsubmitter service and set to trigger two topics on success or failure.

If I run the workspace using the FME Server web interface, the jobsubmitter service is used and topics are triggered correctly.

However, if the workspace is run through the FMEServerJobSubmitter transformer, the REST service is used and the topics are not triggered because they are tied to the jobsubmitter service.

I worked around that by not using the FMEServerJobSubmitter and using a HTTPCaller:

Full Request URL:

$(CCD_FME_SERVER_URL)/fmejobsubmitter/$(CCD_FME_REPOSITORY)/$(CCD_FME_WS_CONVERSION)?opt_servicemode=async&opt;_responseformat=json&tm;_priority=$(CCD_FME_JOB_PRIORITY)

Hope this will help.

Regards,

Larry

david_r
Rank
Userlevel 4

If I understand correctly, I had to work on a similar issue a few days ago.

My problem was that my workspace is published to the jobsubmitter service and set to trigger two topics on success or failure.

If I run the workspace using the FME Server web interface, the jobsubmitter service is used and topics are triggered correctly.

However, if the workspace is run through the FMEServerJobSubmitter transformer, the REST service is used and the topics are not triggered because they are tied to the jobsubmitter service.

I worked around that by not using the FMEServerJobSubmitter and using a HTTPCaller:

0684Q00000ArK2PQAV.png

Full Request URL: 

$(CCD_FME_SERVER_URL)/fmejobsubmitter/$(CCD_FME_REPOSITORY)/$(CCD_FME_WS_CONVERSION)?opt_servicemode=async&opt;_responseformat=json&tm;_priority=$(CCD_FME_JOB_PRIORITY)

Hope this will help.

Regards,

Larry

That makes sense. I suspect you're quite right about the difference between the jobsubmitter and the transformations/commands/submit services.

I think the issue is that the REST API assumes that you will explicitely specify any topics yourself in the FMEDirectives part of the JSON that you submit with the published parameters, etc.

From the API help, here's an example:

{
  "FMEDirectives": {},
  "NMDirectives": {
    "failureTopics": [],
    "successTopics": []
  },
  "TMDirectives": {
    "rtc": false,
    "description": "This is my description",
    "priority": 5,
    "tag": "linux"
  },
  "publishedParameters": [
    {
      "name": "MAXY",
      "value": "42"
    },
    {
      "name": "THEMES",
      "value": [
        "airports",
        "cenart"
      ]
    }
  ],
  "subsection": "REST_SERVICE"
} 

As you can see, "failureTopics" and "successTopics" must be set explicitely when executing a workspace this way.

J

I have the same issue and resolved it as well using an HTTP-caller... It's odd that the FMEServerJobSubmitter is not behaving correctly. I hope they fix this soon.

A

Enhancement idea https://knowledge.safe.com/idea/79163/fmeserverjobsubmitter-post-to-topic.html. Please vote!

Reply


Community Stats

28,662
Posts
109,327
Replies
36,529
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.