Question

FME Best Practice Validation Project (You Can Help!)

  • 21 December 2016
  • 50 replies
  • 26 views
Show first post

50 replies

L
Badge
Fantastic. I like that. I wish I could be there to see the presentation.

 

@jdh This tool is a command line application and does not require a specific version control. It can be integrated in any source control tool accepting external compare program to compare two version of a file as we did with subversion.

 

jdh
Rank
Badge +22
Fantastic. I like that. I wish I could be there to see the presentation.

 

Ah, when I asked I was told it required svn.

 

 

L
Badge
Fantastic. I like that. I wish I could be there to see the presentation.

 

As the developer of the tool, I can assure you that SVN is not required. I'll make sure that the presenter is aware of this for the future.

 

 

sigtill
Rank
Badge +21
The attachment seems to have gone missing. Do you have a copy of it still?

 

 

I hate that i dont get notifications when someone answers my posts (answerhub-issue) - just saw this. However I could successfully download the *.zip file .... is it ok now? And I cant seem to (a)mark2atsafe.

 

 

The real SigTill
L
Badge
Fantastic. I like that. I wish I could be there to see the presentation.

 

Hi @KenAtSafe,

 

I'll try to see if I can find the presentation. It may be in french. No the code is not available but we are planning to open this as a web service for the beginning. I'll post the URL here as soon as it is done.

 

L
Badge
Fantastic. I like that. I wish I could be there to see the presentation.

 

Hi @KenAtSafe,

 

The presentation is strictly based on a demo, I'll try to make the web service live ASAP. In the mean time, if you want to have a little private demo or two workspaces compared, just email them to me.

 

fmelizard
Rank
Userlevel 4
Badge +13

Thanks @larry ! I'll be in touch

runneals
Rank
Badge +14
I just filed PR#76897 - a request for a Notification service that is a Repository Watcher. It would be like a Directory Watcher but would issue a notification if a workspace is published to a particular repository.

 

 

That way you could run a testing workspace in response - but you also get the ability to be notified in general when a workspace is published or updated.

 

 

Validating before publishing to Server - well that would need us to incorporate this whole idea into Desktop functionality (and I just don't see that happening - at least not as easily as a Repository Watcher might).

 

You can kind of do this with a directory watch to the repositories directory
H
Badge +2
Hi @KenAtSafe,

 

The presentation is strictly based on a demo, I'll try to make the web service live ASAP. In the mean time, if you want to have a little private demo or two workspaces compared, just email them to me.

 

Hi @larry,

We are currently looking into possible solutions for comparing workspace versions. Would you be available for a demonstration of this tool? Please let me know if you are happy to do this and I can reach out via email to pin down a date.

L
Badge

Hi @larry,

We are currently looking into possible solutions for comparing workspace versions. Would you be available for a demonstration of this tool? Please let me know if you are happy to do this and I can reach out via email to pin down a date.

Hello @hollyatsafe,

I'm no longer working with Consortech but you can contact them as they are a SAFE partner.

Ping me back if you can't reach them or if they do not get back to you.

Regards,

Larry

 

 

mrclaudepicard

Hi @larry,

We are currently looking into possible solutions for comparing workspace versions. Would you be available for a demonstration of this tool? Please let me know if you are happy to do this and I can reach out via email to pin down a date.

Hello @hollyatsafe,

I'd gladly demo the Workspace Comparator for you.

Contact me by email to set a date.

C.

B
Badge

@mark2atsafe I left some issues on the Github page, but what is the preferred place for it? I am working on an updated version, with more checks and would like to get involveld.

mark2atsafe
Rank
Userlevel 4
Badge +25

@mark2atsafe I left some issues on the Github page, but what is the preferred place for it? I am working on an updated version, with more checks and would like to get involveld.

Yes, sorry about not responding before. It's just a busy time at the moment. I really would like to get this project back up and running. It's an important one to me. I think Github is the right place for it right now. Eventually it might live on the FME Hub (which is getting some great new updates for 2019 that will really help). I think for now if you can post changes back to Github that would be great. Once the FME 2019 release is done and I have all my training updates and blog posts complete, I'll take the time to dive back into this and see where we're at.

Thanks so much for getting involved. I do think this is a useful project and between us we'll get it fixed up and enhanced for sure.

B
Badge

Yes, sorry about not responding before. It's just a busy time at the moment. I really would like to get this project back up and running. It's an important one to me. I think Github is the right place for it right now. Eventually it might live on the FME Hub (which is getting some great new updates for 2019 that will really help). I think for now if you can post changes back to Github that would be great. Once the FME 2019 release is done and I have all my training updates and blog posts complete, I'll take the time to dive back into this and see where we're at.

Thanks so much for getting involved. I do think this is a useful project and between us we'll get it fixed up and enhanced for sure.

Thanks, I will do that soon.

runneals
Rank
Badge +14

Working on creating a formalized document with best practices. Stuck on Web/Database Connections and using them on server. I think ideally there would only be 1 connection per account that it's for (ie. 1 connection to 1 user on ArcGIS Online). BUT how do you manage access to them if that's the case? Or is it easier and better to just let anyone create their own connections and automatically get access since they own it? Is there any pros/cons to doing it the former vs latter?

lauraatsafe
Rank
Badge +9

Working on creating a formalized document with best practices. Stuck on Web/Database Connections and using them on server. I think ideally there would only be 1 connection per account that it's for (ie. 1 connection to 1 user on ArcGIS Online). BUT how do you manage access to them if that's the case? Or is it easier and better to just let anyone create their own connections and automatically get access since they own it? Is there any pros/cons to doing it the former vs latter?

Hi @runneals, In my experience, it's easier to let anyone create their own connections when working on FME Server. That ensures that only the account that created the connection can use or modify it by default. The owner of a connection does have the ability to share it with other users/roles as well, so if they wanted to allow others to have access to it, they can do so.

Having one connection per account could be helpful if you don't want your FME Server users to have their own credentials for the service (like AGOL) being used. For that, the owner of the connection can share it with a specific role to let any users part of that role have access to the connection. The drawback to this is that we currently don't have 'read only' access for connections, so sharing it would mean that any user could edit it.

runneals
Rank
Badge +14

Would be great if this was run right after you hit "Publish to FME Server" on the File menu of FME Workbench - so you could validate it BEFORE you publish it to an FME Server. Or more in generall - add an option to run a custom workspace before "Publish to FME Server" to validate something in that particular fmw workspace - for instance that all your database-connections are connected to staging/dev/prod and with the correct user / paths etc. When uploading to multiple FME Servers dev/stage/prod this can easily be forgotten!

You could probably set up an automation, and then send an email to the user that uploaded it.

runneals
Rank
Badge +14

Thanks to all you fine folks who provided comments here. I have put together a formal best practices guide.

Check out the updated version 1.1 (thanks to @mmccart for some additional feedback/ideas that I originally overlooked and merged with my original): https://github.com/runneals/GISProjects/tree/master/fme/FME%20Best%20Practices

sigtill
Rank
Badge +21

Bumping this post up, because this can be useful to many also in 2019!

The real SigTill
R
@sigtill @redgeographics and others - I'm just wondering about the best way to set this up on GitHub. If there are commit conflicts then it's very hard to assess what the differences are using GitHub itself. You'd need to open them in FME and do a visual inspection to see any differences (writing a DIFF tool for FME workspaces - that sounds like another challenge!)

 

 

Anyway, I think one way to help minimize that - and to make the project more manageable - is to split off each of the sections into exported custom transformers. It becomes a little harder to 'install' into FME, but I think the other advantages outweigh that.

 

 

So I'm going to do that and see if I can get it into GitHub sometime today.

 

@larry​ - Can you please share which tool you're describing here for FMW DIFF? Since this post is quite old, I'm not sure if there's a new tool available to diff purposes. I tried GIT but it's a great deal with FMW file format.

L
Badge
@sigtill @redgeographics and others - I'm just wondering about the best way to set this up on GitHub. If there are commit conflicts then it's very hard to assess what the differences are using GitHub itself. You'd need to open them in FME and do a visual inspection to see any differences (writing a DIFF tool for FME workspaces - that sounds like another challenge!)

 

 

Anyway, I think one way to help minimize that - and to make the project more manageable - is to split off each of the sections into exported custom transformers. It becomes a little harder to 'install' into FME, but I think the other advantages outweigh that.

 

 

So I'm going to do that and see if I can get it into GitHub sometime today.

 

Hello Ravin,

 

This is an old internal tool Consortech was developing while I was working there.

My best suggestion is for you to try to contact them and see if this is maintained and still available.

And please et me know if you can't have an answer, I can give you some hints if you want to develop your own tool.

 

Regards,

Larry

mark2atsafe
Rank
Userlevel 4
Badge +25
@sigtill @redgeographics and others - I'm just wondering about the best way to set this up on GitHub. If there are commit conflicts then it's very hard to assess what the differences are using GitHub itself. You'd need to open them in FME and do a visual inspection to see any differences (writing a DIFF tool for FME workspaces - that sounds like another challenge!)

 

 

Anyway, I think one way to help minimize that - and to make the project more manageable - is to split off each of the sections into exported custom transformers. It becomes a little harder to 'install' into FME, but I think the other advantages outweigh that.

 

 

So I'm going to do that and see if I can get it into GitHub sometime today.

 

Check out FME2022 (still in beta, but will be released shortly). It has a visual comparison tool that sounds like it would help.

R
@sigtill @redgeographics and others - I'm just wondering about the best way to set this up on GitHub. If there are commit conflicts then it's very hard to assess what the differences are using GitHub itself. You'd need to open them in FME and do a visual inspection to see any differences (writing a DIFF tool for FME workspaces - that sounds like another challenge!)

 

 

Anyway, I think one way to help minimize that - and to make the project more manageable - is to split off each of the sections into exported custom transformers. It becomes a little harder to 'install' into FME, but I think the other advantages outweigh that.

 

 

So I'm going to do that and see if I can get it into GitHub sometime today.

 

That would be great! Thanks. I'll try that out and post results.

R
@sigtill @redgeographics and others - I'm just wondering about the best way to set this up on GitHub. If there are commit conflicts then it's very hard to assess what the differences are using GitHub itself. You'd need to open them in FME and do a visual inspection to see any differences (writing a DIFF tool for FME workspaces - that sounds like another challenge!)

 

 

Anyway, I think one way to help minimize that - and to make the project more manageable - is to split off each of the sections into exported custom transformers. It becomes a little harder to 'install' into FME, but I think the other advantages outweigh that.

 

 

So I'm going to do that and see if I can get it into GitHub sometime today.

 

Thanks Larry

L
Badge
@sigtill @redgeographics and others - I'm just wondering about the best way to set this up on GitHub. If there are commit conflicts then it's very hard to assess what the differences are using GitHub itself. You'd need to open them in FME and do a visual inspection to see any differences (writing a DIFF tool for FME workspaces - that sounds like another challenge!)

 

 

Anyway, I think one way to help minimize that - and to make the project more manageable - is to split off each of the sections into exported custom transformers. It becomes a little harder to 'install' into FME, but I think the other advantages outweigh that.

 

 

So I'm going to do that and see if I can get it into GitHub sometime today.

 

I'm glad to hear that. Not using FME these days but I'll give it a try as soon as I get back to it!

Reply


Community Stats

28,732
Posts
109,535
Replies
36,570
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.