Solved

Feature Service from FME Database/logs

  • 17 September 2019
  • 10 replies
  • 3 views
runneals
Rank
Badge +8

I was wondering if there was further documentation on the FME server database. I'm looking to create a REST service from the database that we can use in conjunction with Operations Dashboard to show statistics about FME server in real-time (specifically things such as last time a job was ran, the status of the job, errors from the job (if any), run time, etc). Currently I'm harvesting data nightly to get a summary of the performance over the day, but it would be nice to be able to create a rest service that would show all the data together without having to heavily use an FME engine to push data to ArcGIS Enterprise (Portal) or ArcGIS Online.

icon

Best answer by jlutherthomas 19 November 2019, 05:43

View original

10 replies

stalknecht
Rank
Userlevel 1
Badge +18

You can use the REST interface of FME server: https://docs.safe.com/fme/html/FME_REST/apidoc/v3/index.html

If there is still something missing the default port, user and password are: 7082, fmeserver, fmeserver

good luck

david_r
Rank
Userlevel 4

I agree with @stalknecht, that's what the REST API is for. Yes, it's possible to dig directly in the database tables, but

  1. they're not documented
  2. it's not recommended (as far as I know)
  3. they're subject to change without notice between FME versions

 

runneals
Rank
Badge +8

I agree with @stalknecht, that's what the REST API is for. Yes, it's possible to dig directly in the database tables, but

  1. they're not documented
  2. it's not recommended (as far as I know)
  3. they're subject to change without notice between FME versions

 

The problem with the REST service is that it's not a native Feature Service that I could add to an ArcGIS Operations Dashboard. Would be nice if there was something like this maybe already in FME Server ;)

J
Badge +2

@runneals If you're willing to give me a day I might have something you could test for me...

runneals
Rank
Badge +8

@runneals If you're willing to give me a day I might have something you could test for me...

Yeah, no rush! It's been low on the to-do list, but would be nice to have headed into winter when we have to start monitoring server more closely.

J
Badge +2

@runneals I am 99% sure I have a way you can do this.... (It's not easy)

 

 

 

Is there an FME Server REST API call you're particular interested in?

 

What one of these were you thinking of? I'm an Operations Dashboard newb so just testing to see if it'd work.

 

J
Badge +2

Hi @runneals

 

 

So I managed to get it working.

 

Here is a really basic dashboard (based on the last 1000 jobs):

 

 

Basically, Esri have something called Koop that you can use to 'spoof' Feature Services.

 

 

Koop does allow caching (benefits for spatial data so you're not always running FME Server jobs), but basically every time you view this dashboard, FME Server will get called (workspace calling the REST API) and get you the latest data. If the cache is expired it will go and get new data (you can configure cache timeout).

 

 

 

I'm hoping that soon I'll be able to put up an article on how to do this step-by-step for FME Server with a github repository.

For this you need to configure to Koop on Node JS (to connect to FME Server) behind NGINX (configured for reverse proxy and HTTPS) which makes Esri Feature Services, as long as the workspace returns JSON in the correct format.

 

 

I was initially playing with spatial geoservices for use in AGOL, but remembered this and adapted it for non-spatial too. If the set up doesn't scare you, then I'll let you know when the article/github repo goes up!
nielsgerrits
Rank
Userlevel 6
Badge +32

Hi @runneals

 

 

So I managed to get it working.

 

Here is a really basic dashboard (based on the last 1000 jobs):

 

 

Basically, Esri have something called Koop that you can use to 'spoof' Feature Services.

 

 

Koop does allow caching (benefits for spatial data so you're not always running FME Server jobs), but basically every time you view this dashboard, FME Server will get called (workspace calling the REST API) and get you the latest data. If the cache is expired it will go and get new data (you can configure cache timeout).

 

 

 

I'm hoping that soon I'll be able to put up an article on how to do this step-by-step for FME Server with a github repository.

For this you need to configure to Koop on Node JS (to connect to FME Server) behind NGINX (configured for reverse proxy and HTTPS) which makes Esri Feature Services, as long as the workspace returns JSON in the correct format.

 

 

I was initially playing with spatial geoservices for use in AGOL, but remembered this and adapted it for non-spatial too. If the set up doesn't scare you, then I'll let you know when the article/github repo goes up!

Inspiring. Thanks for sharing!

runneals
Rank
Badge +8

Hi @runneals

 

 

So I managed to get it working.

 

Here is a really basic dashboard (based on the last 1000 jobs):

 

 

Basically, Esri have something called Koop that you can use to 'spoof' Feature Services.

 

 

Koop does allow caching (benefits for spatial data so you're not always running FME Server jobs), but basically every time you view this dashboard, FME Server will get called (workspace calling the REST API) and get you the latest data. If the cache is expired it will go and get new data (you can configure cache timeout).

 

 

 

I'm hoping that soon I'll be able to put up an article on how to do this step-by-step for FME Server with a github repository.

For this you need to configure to Koop on Node JS (to connect to FME Server) behind NGINX (configured for reverse proxy and HTTPS) which makes Esri Feature Services, as long as the workspace returns JSON in the correct format.

 

 

I was initially playing with spatial geoservices for use in AGOL, but remembered this and adapted it for non-spatial too. If the set up doesn't scare you, then I'll let you know when the article/github repo goes up!

That's what I was pretty much looking for!

Is it possible/advisable to install on FME server machine itself or should it be a separate machine? Mainly looking for internal network access only, with only 1-3 users max of it.

J
Badge +2

That's what I was pretty much looking for!

Is it possible/advisable to install on FME server machine itself or should it be a separate machine? Mainly looking for internal network access only, with only 1-3 users max of it.

The application itself is pretty lightweight so I don't think there would be any issues having it on the same machine, as long as node and nginx are configured to use different ports that won't conflict with FME Server. However, for best practice and just-in-case, it'd probably be better to keep them separate. I also haven't tried installing them all on the same machine, so everything I've said is pure speculation.

 

 

It does also (depending on caching, but even with it turned on) make a lot of engine calls, however they are quick (depending on the streaming workspace you build), so you'd maybe want to dedicate an engine or queue to it if you have the capacity (or use an existing queue for quick, fast-response streaming jobs).

Reply


Community Stats

28,670
Posts
109,366
Replies
36,531
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.