Skip to main content
Question

FME Server REST API - Job History limited to 1000?

  • September 18, 2018
  • 5 replies
  • 99 views

virtualcitymatt
Celebrity
Forum|alt.badge.img+35

I'm trying to use the FME Server REST service to get information about Job History,

 

I'm using this call: 
https://<fmeserver_host>/fmerest/v3/transformations/jobs/completed?limit=-1&offset;=-1

The problem is that it's only returning information about 1000 jobs when there are much more in there. We're running FME Server version 17725. I can't see anything about limits in the documentation and I'm wondering if this is a bug or if it's just undocumented. Can anyone test this call on their server to see if you see the same thing? I'm following it with a JSONFragmenter with 'json[*][*]' as the query to split up the response. I'd rather not build a workaround for this if it's a bug. 

5 replies

nielsgerrits
VIP
Forum|alt.badge.img+54

My guess would be room for improvement in the documentation.

It is quite common for webservices to limit the number of returned features per request to prevent timeouts etc. You need to iterate through the results. I must admit I didn't work out this specific case so I can't give you the exact details without diving in. 

First 100 results:

https://<fmeserver_host>/fmerest/v3/transformations/jobs/completed?limit=100&offset;=0

Second 100 results:

https://<fmeserver_host>/fmerest/v3/transformations/jobs/completed?limit=100&offset;=100

When no results are returned you are through. Tested on Server 2016 / V2.


Forum|alt.badge.img+2

Hi @virtualcitymatt

I believe 1000 is the limit, it's just undocumented. I believe this is because otherwise there could potentially be 1000s and 1000s of records being returned which would impact performance of the server core.

You would have to use the offset and limits that @nielsgerrits has shown, to get all of the jobs. There should be a total count response, so you'd know how many times to call the api.


virtualcitymatt
Celebrity
Forum|alt.badge.img+35
jlutherthomas wrote:

Hi @virtualcitymatt

I believe 1000 is the limit, it's just undocumented. I believe this is because otherwise there could potentially be 1000s and 1000s of records being returned which would impact performance of the server core.

You would have to use the offset and limits that @nielsgerrits has shown, to get all of the jobs. There should be a total count response, so you'd know how many times to call the api.

Thanks @jlutherthomas and @nielsgerrits! - I'll change up my request

nic_ran
Contributor
Forum|alt.badge.img+16
  • Contributor
  • May 14, 2020

I've just come up against this limitation myself and see that the documentation still hasn't been updated. It would be great to see the documentation updated to acknowledge this limit.

While you're in there, it would be nice if the documentation could also describe the "-1" values for the limit and offset parameters. Although these values are implied by the example given in the API test page, they are not actually documented.

Finally, do the limit/offset values start at 0 or 1?

All of this information can be discovered via trial and error but updated documentation would avoid the need to do this...

FYI, I submitted an "Idea" to have the documentation updated:

https://knowledge.safe.com/idea/113993/update-documentation-for-fme-server-rest-api-v3-jo.html


nic_ran
Contributor
Forum|alt.badge.img+16
  • Contributor
  • May 14, 2020

Front Point Consulting have built a custom transformer to make it easier to get all job history items...

https://hub.safe.com/publishers/front-point-consulting/transformers/get-all-fme-server-completed-jobs


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings