I am trying to publish workspaces via the REST api. Ideally I would like to programmatically get the workspaces dependencies before publishing to make sure the resources are on the server. When I manually publish via Desktop I get a warning about dependencies, I would like to access this info before publishing with REST. I am making the REST calls with Python so if I had a way to get the dependencies with Python that would be real neat.
Solved
Query a workspace for resource dependencies
Best answer by jlutherthomas
So, @2dews and @larry I've got the scoop on what the differences are between REST API and FME Publishing Wizard (I will edit my previous answer)
View original
You can publish to FME Server using the REST API, however this isn't something that we recommend, as the publishing wizard is doing additional stuff to the REST API.
What the FME Desktop Publishing Wizard can do, that the REST API can't:
- Source dataset paths are updated if files are uploaded to FME Server
- Named connections may be renamed when uploaded to FME Server, due to character restrictions. This also includes named connections used in web-as-a-filesystem workflows
- Service registration options are cached within the workspace so they can be used in a future publish. They don't affect the workspace behaviour itself
- Disable destination redirects (to visualizer or to FFS)
- Cache last published repository name and workspace name
So the publishing wizard has methods to search the workspace for all of the workspace dependencies that are part of the publishing process, however this is not available through an API call. So if you use the REST API to publish a workspace, you will need to tell the API everything it will need to know to run the workspace as you want. The API calls out to FME Server, where in this case the API wouldn't be valid as it needs to look at a workspace before it's published.
Because of this, we do not recommend using the REST API to publish workspaces to FME Server, as it isn't guaranteed that it will behave as expected compared to publishing through the Publishing Wizard. So I wouldn't be surprised if you get situations where it doesn't work correctly.
+2Hi @2dews
We don't actually support publishing a workspace via the REST API at the moment, however this is something we'd like to do in the future. Also determining all of the workspace dependencies, and FME Server object dependencies (connections, topics, etc) is something we'd like to be able to do.
You can add this as an idea here.
jlutherthomas wrote:
Hi @2dews
We don't actually support publishing a workspace via the REST API at the moment, however this is something we'd like to do in the future. Also determining all of the workspace dependencies, and FME Server object dependencies (connections, topics, etc) is something we'd like to be able to do.
You can add this as an idea here.
+22dews wrote:
Hi @2dews
In the call it has "Note: This version of the FME Server REST API does not replicate all of the functionality available in the Publish to FME Server wizard of FME Workbench. For more information, see "Publishing to FME Server" in the FME Desktop Help (https://docs.safe.com/fme/html/FME_Desktop_Documentation/FME_Workbench/Workbench/publishing_to_fme_server.htm)"
It's hard to tell going through the api docs and the desktop docs where the difference is, and what functionality is missing. One of those could well be the dependencies issue that you're referring to.
It also looks like things like web or database connections need to already be published to FME Server and referred to during this call, where as during publish from desktop I believe it also publishes the web connections.
Also with services it looks like you can specify the service, but no further settings around it.
However, I have not tested this REST call myself, I would definitely recommend using the desktop wizard to be on the safe side. I've asked my colleagues if any of them have tested this REST call.
Aside from dependencies, have you successfully got this call working to publish and run a workspace?
jlutherthomas wrote:
Hi @2dews
We don't actually support publishing a workspace via the REST API at the moment, however this is something we'd like to do in the future. Also determining all of the workspace dependencies, and FME Server object dependencies (connections, topics, etc) is something we'd like to be able to do.
You can add this as an idea here.
POST /repositories/<repository>/items
The documentation does say that This version of the FME Server REST API does not replicate all of the functionality available in the Publish to FME Server wizard of FME Workbench, which is great, but it would be ideal if the differences were desceribed in a bit more detail.
jlutherthomas wrote:
Hi @2dews
We don't actually support publishing a workspace via the REST API at the moment, however this is something we'd like to do in the future. Also determining all of the workspace dependencies, and FME Server object dependencies (connections, topics, etc) is something we'd like to be able to do.
You can add this as an idea here.
jlutherthomas wrote:
Hi @2dews
We don't actually support publishing a workspace via the REST API at the moment, however this is something we'd like to do in the future. Also determining all of the workspace dependencies, and FME Server object dependencies (connections, topics, etc) is something we'd like to be able to do.
You can add this as an idea here.
Hi @2dews,
in the FME Server REST API v3 we added an additional endpoint for Projects. This still doesn't let you query dependencies of workspaces, but it allows you to bundle a workspace with all parts the workspace needs to run, and move it as a Project. Here's the REST endpoint:
https://docs.safe.com/fme/html/FME_REST/apidoc/v3/#!/projects
Not sure if this solves your problem, but I think it's worth to take a look at.
@larry this might be interesting for your workflow too. Maybe it makes a few things easier compared to REST API v2.
gerhardatsafe wrote:
Hi @2dews,
in the FME Server REST API v3 we added an additional endpoint for Projects. This still doesn't let you query dependencies of workspaces, but it allows you to bundle a workspace with all parts the workspace needs to run, and move it as a Project. Here's the REST endpoint:
https://docs.safe.com/fme/html/FME_REST/apidoc/v3/#!/projects
Not sure if this solves your problem, but I think it's worth to take a look at.
@larry this might be interesting for your workflow too. Maybe it makes a few things easier compared to REST API v2.
+2So, @2dews and @larry I've got the scoop on what the differences are between REST API and FME Publishing Wizard (I will edit my previous answer)
You can publish to FME Server using the REST API, however this isn't something that we recommend, as the publishing wizard is doing additional stuff to the REST API.
What the FME Desktop Publishing Wizard can do, that the REST API can't:
- Source dataset paths are updated if files are uploaded to FME Server
- Named connections may be renamed when uploaded to FME Server, due to character restrictions. This also includes named connections used in web-as-a-filesystem workflows
- Service registration options are cached within the workspace so they can be used in a future publish. They don't affect the workspace behaviour itself
- Disable destination redirects (to visualizer or to FFS)
- Cache last published repository name and workspace name
So the publishing wizard has methods to search the workspace for all of the workspace dependencies that are part of the publishing process, however this is not available through an API call. So if you use the REST API to publish a workspace, you will need to tell the API everything it will need to know to run the workspace as you want. The API calls out to FME Server, where in this case the API wouldn't be valid as it needs to look at a workspace before it's published.
Because of this, we do not recommend using the REST API to publish workspaces to FME Server, as it isn't guaranteed that it will behave as expected compared to publishing through the Publishing Wizard. So I wouldn't be surprised if you get situations where it doesn't work correctly.
jlutherthomas wrote:
So, @2dews and @larry I've got the scoop on what the differences are between REST API and FME Publishing Wizard (I will edit my previous answer)
You can publish to FME Server using the REST API, however this isn't something that we recommend, as the publishing wizard is doing additional stuff to the REST API.
What the FME Desktop Publishing Wizard can do, that the REST API can't:
- Source dataset paths are updated if files are uploaded to FME Server
- Named connections may be renamed when uploaded to FME Server, due to character restrictions. This also includes named connections used in web-as-a-filesystem workflows
- Service registration options are cached within the workspace so they can be used in a future publish. They don't affect the workspace behaviour itself
- Disable destination redirects (to visualizer or to FFS)
- Cache last published repository name and workspace name
So the publishing wizard has methods to search the workspace for all of the workspace dependencies that are part of the publishing process, however this is not available through an API call. So if you use the REST API to publish a workspace, you will need to tell the API everything it will need to know to run the workspace as you want. The API calls out to FME Server, where in this case the API wouldn't be valid as it needs to look at a workspace before it's published.
Because of this, we do not recommend using the REST API to publish workspaces to FME Server, as it isn't guaranteed that it will behave as expected compared to publishing through the Publishing Wizard. So I wouldn't be surprised if you get situations where it doesn't work correctly.
+22dews wrote:
That's unfortunate. We will continue on the path of publishing with the REST api though. We are looking to move towards a continuous integration setup where our workspaces are published to server and tested when checked into Git. Not to mention we want to be able to teardown and rebuild our server completely from code. I would rather write more code to make sure we push everything to the server than require someone to sit down and publish 25 workspaces when we make updates.
2dews wrote:
That's unfortunate. We will continue on the path of publishing with the REST api though. We are looking to move towards a continuous integration setup where our workspaces are published to server and tested when checked into Git. Not to mention we want to be able to teardown and rebuild our server completely from code. I would rather write more code to make sure we push everything to the server than require someone to sit down and publish 25 workspaces when we make updates.
larry wrote:
Same here. We are able to rebuild our FME Server from scratch (9 roles, 9 users, 5 resources folders containing 3766 resources files, 9 custom transformers and python files, 9 repositories and permissions, 17 topics and permissions, 17 email subscriptions with HTML templates, 21 workspaces and 1 schedule) in a few minutes using REST while it was taking hours in the past. This allowed us to implement continuous integration and automatically rebuild our server and run a test suite every night.
2dews wrote:
@larry If you have any blog posts, GitHub repos, or other material highlighting your CI or tests it would be great to see. Sounds like you are already at where we are hoping to get to.
2dews wrote:
@larry If you have any blog posts, GitHub repos, or other material highlighting your CI or tests it would be great to see. Sounds like you are already at where we are hoping to get to.
larry wrote:
Same here. We are able to rebuild our FME Server from scratch (9 roles, 9 users, 5 resources folders containing 3766 resources files, 9 custom transformers and python files, 9 repositories and permissions, 17 topics and permissions, 17 email subscriptions with HTML templates, 21 workspaces and 1 schedule) in a few minutes using REST while it was taking hours in the past. This allowed us to implement continuous integration and automatically rebuild our server and run a test suite every night.
2dews wrote:
@larry do you use REST to set up database connections? I am having trouble figuring that out.
Reply
Rich Text Editor, editor1
Editor toolbars
Press ALT 0 for help
Helpful Members This Week
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.