Solved

Get JSON for a Topic

  • 30 November 2015
  • 5 replies
  • 6 views
J
Rank
Badge +1
  • jimo Participant
  • Participant
  • 126 replies

I have a publication in FME Server that monitors a directory and sends a message to a topic when the contents of the directory change - for example, when a file gets updated.

When I start the monitor for the topic in FME Server, I get this JSON output:

Total: 1 Shown: 1 Test_DirWatch_Topic arrived at 7:40:03 { "ws_topic": "Test_DirWatch_Topic", "dirwatch_publisher_path": "C:\\\\ProgramData\\\\Safe Software\\\\FME Server\\\\Data\\\\xls\\\\New Text Document.txt", "dirwatch_publisher_content": "ENTRY_MODIFY C:\\\\ProgramData\\\\Safe Software\\\\FME Server\\\\Data\\\\xls\\\\New Text Document.txt", "dirwatch_publisher_action": "MODIFY", "fns_type": "dirwatch_publisher" }I also have am FME script that subscribes to the topic and takes action when the topic receives the JSON above. However, I only want the script to take action if the directory change is a "modify" to a certain file (in this case, "New Text Document.txt"). There are other files in this directory and I don't want the script to take action when any of them change. Is it possible for my FME script to get access to this JSON? Then I can break it down and determine which directory change occurred.Thanks
icon

Best answer by jimo 2 December 2015, 18:04

View original

5 replies

mark2atsafe
Rank
Userlevel 4
Badge +25

Absolutely. You need to add a textfile Reader to the workspace.

Then when you publish the workspace, click the notification properties button and select the Textfile Reader as the Notification Data Reader.

Now when you workspace is triggered by the notification, the JSON message will be passed into the workspace through this Reader. You can then process it using JSONFlattener transformers, or similar, to find out details of the notification.

J
Rank
Badge +1

Absolutely. You need to add a textfile Reader to the workspace.

Then when you publish the workspace, click the notification properties button and select the Textfile Reader as the Notification Data Reader.

Now when you workspace is triggered by the notification, the JSON message will be passed into the workspace through this Reader. You can then process it using JSONFlattener transformers, or similar, to find out details of the notification.

This works fine up until the jsonflattener. Then I get errors in my log file like:

JSONFlattener(JSONQueryFactory): A JSON syntax error was found at line 1, column 1

...and

JSONFlattener(JSONQueryFactory): The JSON data is incomplete: Unexpectedly encountered the end of JSON data

I have also tried parsing the text_line_data in a PythonCaller using the built-in json module with the same result. I think the problem is that the jsonflattener and the Python module don't understand the newlines in the notification output. Has anyone at Safe actually got this to work?

mark2atsafe
Rank
Userlevel 4
Badge +25

This works fine up until the jsonflattener. Then I get errors in my log file like:

JSONFlattener(JSONQueryFactory): A JSON syntax error was found at line 1, column 1

...and

JSONFlattener(JSONQueryFactory): The JSON data is incomplete: Unexpectedly encountered the end of JSON data

I have also tried parsing the text_line_data in a PythonCaller using the built-in json module with the same result. I think the problem is that the jsonflattener and the Python module don't understand the newlines in the notification output. Has anyone at Safe actually got this to work?

Some quick thoughts. Maybe set the TextFile Reader to "read whole file at once". It might be reading it as a multi-line dataset, and trying to parse a line at a time (how many features does it read?) - also perhaps set a Character Encoding for the Reader? Finally, can you put a Logger transformer in before the JSONFlattener to get a better idea of what is being returned? I suspect the problem is somehow in how the TextFile reader is reading the data.

J
Rank
Badge +1

That was the problem, Mark. I set the TextFile Reader to "read whole file at once" and it read the whole notification JSON, which the JSONFlattener was then able to parse.

Thanks.

mark2atsafe
Rank
Userlevel 4
Badge +25

That was the problem, Mark. I set the TextFile Reader to "read whole file at once" and it read the whole notification JSON, which the JSONFlattener was then able to parse.

Thanks.

Awesome. Glad to hear it's working.

Reply


Community Stats

28,724
Posts
109,520
Replies
36,564
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.