Skip to main content
Question

FME Challenge: 12 Days of FME Hackathon

  • November 30, 2017
  • 79 replies
  • 247 views

Show first post
This post is closed to further activity.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.

79 replies

runneals
Supporter
Forum|alt.badge.img+29
  • Supporter
  • December 14, 2017

TITLE:

 

Predicting football results using FME. Reducing hours of analysis down to 10 seconds.

 

 

BRIEF DESCRIPTION:

 

There are hundreds of football games taking place across Europe each weekend. To determine the best matches to place a wager can be very difficult. Using open data (upcoming fixtures compared with historical form) I am using FME to quickly analyse the matches that are best to bet on each game week. What used to take hours to analyse is now being achieved using an FME workspace that takes under 10 seconds to run. I have it set up on FME Server to send me (and my friends!) an email each time there are new decent bets available. It also now tweets from a new dedicated account @deadcerts.

 

I'm sharing the workspace here, and hopefully I will get some feedback and improvements from other tied up with FME and Football as their main work/hoby! If you make any improvements, which I'm sure you will, then maybe we can collaborate on a Github project? You can contact me @pinpointalerts

 

DOWNLOAD THE WORKSPACE:

 

fme-football-predictor.zip

 

I mentor a high school FIRST Robotics team and was pretty close to trying to integrate FME into our scouting system to do something similar except more for scouting and strategizing than betting :P, but never got it out of an idea. Interesting use for it.

runneals
Supporter
Forum|alt.badge.img+29
  • Supporter
  • December 14, 2017
Now having FME Server ring you back to tell the answer (trying to skip any voicemail services and retrying every five minutes as long as no real life contact was made)! Well done.

 

 

 

More like having FME Server call you to let you know it is sick (jobs failing) and needs some attention would be awesome... But all those phone calls :P At least it could get you out of some important meetings xD

revesz
Contributor
Forum|alt.badge.img+21
  • Contributor
  • December 14, 2017
Ooh, I wonder what DanRSecret.fmw does. I suppose you'll tell me it's a secret!!

 

 

Yep, because Dan says it is a secret

 


runneals
Supporter
Forum|alt.badge.img+29
  • Supporter
  • December 14, 2017

Inspiration for this project was: https://www.3ders.org/articles/20161220-3d-printed-gingerbread-house-lets-you-celebrate-christmas-the-scandinavian-way.html

a) GingerbreadhouseGenerator b) Converts your fancy 3D-modelled house (for instance from Sketchup) to a 2D-model possible to print and build a gingerbreadhouse from the parts! c) Workspace can be sent upon requests by commenting this article. Below are the steps used in the workspace! I do not guarantee for the printed papers and the finished 3D Gingerbreadhouse! 1. Select area with house:

2. Look at the result in Sketcup

3. Get a single house:

4. Explode the house into individual parts:

5. Workspace with description

6. End result - one PDF per part - and the simple PDF

7. Rent a bakery and print all the roughly 40 000 000 + parts to all the 2 770 000+ houses in Norway (TODO)

TODO if enough interest:

- Bake the house

- Rotate the geoemetries to "flat surface" by using the SurfaceNormalCalculator and 3DAffiner to get correct geometries.

I just got to looking and that's a lot of wasted paper and gingerbread if you have 32+ sheets xD

runneals
Supporter
Forum|alt.badge.img+29
  • Supporter
  • December 14, 2017
Interesting.

 

I recommend better error catching if there is an invalid port.  Currently it is

 

2017-12-11 15:09:53| 0.6| 0.0|WARN |Python Exception <TypeError>: Input must be a string or unicode.
2017-12-11 15:09:53| 0.6| 0.0|WARN |Traceback (most recent call last):
File "<string>", line 98, in close
File "<string>", line 23, in throw
TypeError: Input must be a string or unicode.
2017-12-11 15:09:53| 0.6| 0.0|FATAL |f_70(PythonFactory): PythonFactory failed to close properly
2017-12-11 15:09:53| 0.6| 0.0|ERROR |f_70(PythonFactory): A fatal error has occurred. Check the logfile above for details

 

I would also be very interested in checking all the licences on a specific server in one shot, or at least all the licences for a specific version of flexlm.  (We have 6 software requiring 3 versions of flexlm on the same license server)

 

Is that what that error meant? It seemed more like a python bug than putting in an invalid port number.

runneals
Supporter
Forum|alt.badge.img+29
  • Supporter
  • December 14, 2017

A title.

Server log saver and timeline visualizer

A brief description.

We have more and more schedules and I was less and less enthusiast to go through long-long list of logs (instead of building a snowman army for example now ???) when someone didn't like something

Half a year ago I opted not to care with a few SLAs and build something visual (no, I cannot build nice tings) that save me some boring time ?.

I end up with this.

The workspace saves the Server log to a staging table (on request) for future reference. No cleanup now, the size of the table is the discretion whoever cares with it.

The timeline control is Google Charts (sorry anyone from China). It is not my dream as a timeline (no control over individual colours, no navigation, etc) but I just couldn't find anything decent. Until I hack one (that would look as nice as FME Server Schedules Dashboard inspiration) I chose a quick win solution temporarily.

fme-job-history-to-google-timeline.fmw

Why do you dive into the server db? Why not use the API to request server logs like the workspace does for the FME dashboard?

ebygomm
Influencer
Forum|alt.badge.img+46
  • Influencer
  • December 14, 2017

Inspiration for this project was: https://www.3ders.org/articles/20161220-3d-printed-gingerbread-house-lets-you-celebrate-christmas-the-scandinavian-way.html

a) GingerbreadhouseGenerator b) Converts your fancy 3D-modelled house (for instance from Sketchup) to a 2D-model possible to print and build a gingerbreadhouse from the parts! c) Workspace can be sent upon requests by commenting this article. Below are the steps used in the workspace! I do not guarantee for the printed papers and the finished 3D Gingerbreadhouse! 1. Select area with house:

2. Look at the result in Sketcup

3. Get a single house:

4. Explode the house into individual parts:

5. Workspace with description

6. End result - one PDF per part - and the simple PDF

7. Rent a bakery and print all the roughly 40 000 000 + parts to all the 2 770 000+ houses in Norway (TODO)

TODO if enough interest:

- Bake the house

- Rotate the geoemetries to "flat surface" by using the SurfaceNormalCalculator and 3DAffiner to get correct geometries.

Going to have to make a sketchup model of the Innderdalshytta so I can compare the fme produced pattern to my hand made one :-)

 

 

https://www.flickr.com/photos/ebygomm/5336924990/in/photolist-98B9ef-98xZqe-98B8S7

 

 

 

 

 

 


revesz
Contributor
Forum|alt.badge.img+21
  • Contributor
  • December 15, 2017
Why do you dive into the server db? Why not use the API to request server logs like the workspace does for the FME dashboard?
Yes, the API is a good point for a solution in FME dashboard showing the job history residing in the FME DB.

 

However I wanted to see jobs older than what the cleanup settings keep.

 

I don't want to overload the FME DB job history table either, so I keep all historic job data in a separate "staging" table. (It holds 55k+ records from the last half year). It is more convenient to query the new records from the FME DB table and push to the staging one than converting the json, too.

 


takashi
Celebrity
  • December 15, 2017
Going to have to make a sketchup model of the Innderdalshytta so I can compare the fme produced pattern to my hand made one :-)

 

 

https://www.flickr.com/photos/ebygomm/5336924990/in/photolist-98B9ef-98xZqe-98B8S7

 

 

 

 

 

 

Fantastic!

 


sigtill
Supporter
Forum|alt.badge.img+25
  • Supporter
  • December 15, 2017
Going to have to make a sketchup model of the Innderdalshytta so I can compare the fme produced pattern to my hand made one :-)

 

 

https://www.flickr.com/photos/ebygomm/5336924990/in/photolist-98B9ef-98xZqe-98B8S7

 

 

 

 

 

 

Your Gingebreadhouse and the image from David is more impressive than the 3D-model, thats for sure! For some reason I can not upload an image right now, I`ll try again later - but it can also be viewed here: https://imgur.com/qrBOkhn

 

 


sigtill
Supporter
Forum|alt.badge.img+25
  • Supporter
  • December 15, 2017
I just got to looking and that's a lot of wasted paper and gingerbread if you have 32+ sheets xD
yup, v2 will include more parts on a single paper - to make an environmentalfriendly gingerhousetemplate

 

 


david_r
Celebrity
  • December 18, 2017
yup, v2 will include more parts on a single paper - to make an environmentalfriendly gingerhousetemplate

 

 

For when the time comes ;-)

 

https://en.wikipedia.org/wiki/Bin_packing_problem

jdh
Contributor
Forum|alt.badge.img+40
  • Contributor
  • December 18, 2017
Is that what that error meant? It seemed more like a python bug than putting in an invalid port number.
If you put in an invalid port number, then you get a python exception error.

 

 

I only know this because I was testing the various licences we have, and I accidentally used the wrong flexlm/port combination.

 

 

It would be great if the exception was caught and a better error message returned.

 

 


sigtill
Supporter
Forum|alt.badge.img+25
  • Supporter
  • December 18, 2017
Oh, already pythonized the problem! https://github.com/secnot/rectpack

 

 


sigtill
Supporter
Forum|alt.badge.img+25
  • Supporter
  • December 18, 2017
Going to have to make a sketchup model of the Innderdalshytta so I can compare the fme produced pattern to my hand made one :-)

 

 

https://www.flickr.com/photos/ebygomm/5336924990/in/photolist-98B9ef-98xZqe-98B8S7

 

 

 

 

 

 

Now I can post the image:

 

 


fmelizard
Safer
Forum|alt.badge.img+21
  • Author
  • Safer
  • December 18, 2017

Inspiration for this project was: https://www.3ders.org/articles/20161220-3d-printed-gingerbread-house-lets-you-celebrate-christmas-the-scandinavian-way.html

a) GingerbreadhouseGenerator b) Converts your fancy 3D-modelled house (for instance from Sketchup) to a 2D-model possible to print and build a gingerbreadhouse from the parts! c) Workspace can be sent upon requests by commenting this article. Below are the steps used in the workspace! I do not guarantee for the printed papers and the finished 3D Gingerbreadhouse! 1. Select area with house:

2. Look at the result in Sketcup

3. Get a single house:

4. Explode the house into individual parts:

5. Workspace with description

6. End result - one PDF per part - and the simple PDF

7. Rent a bakery and print all the roughly 40 000 000 + parts to all the 2 770 000+ houses in Norway (TODO)

TODO if enough interest:

- Bake the house

- Rotate the geoemetries to "flat surface" by using the SurfaceNormalCalculator and 3DAffiner to get correct geometries.

This actually works, amazed.

 

 


davideagle
Contributor
Forum|alt.badge.img+22
  • Contributor
  • December 18, 2017
If you put in an invalid port number, then you get a python exception error.

 

 

I only know this because I was testing the various licences we have, and I accidentally used the wrong flexlm/port combination.

 

 

It would be great if the exception was caught and a better error message returned.

 

 

There is definitely a problem with the Python, I use it to handle a pivot of the license usage stats and that part needs cleaning up before I release this properly. But, thank you very much for the comments as that highlights what I need to test when I give this more than the "hack" I've given it so far. Happy Christmas all!

 

 


davideagle
Contributor
Forum|alt.badge.img+22
  • Contributor
  • December 18, 2017
Interesting.

 

I recommend better error catching if there is an invalid port.  Currently it is

 

2017-12-11 15:09:53| 0.6| 0.0|WARN |Python Exception <TypeError>: Input must be a string or unicode.
2017-12-11 15:09:53| 0.6| 0.0|WARN |Traceback (most recent call last):
File "<string>", line 98, in close
File "<string>", line 23, in throw
TypeError: Input must be a string or unicode.
2017-12-11 15:09:53| 0.6| 0.0|FATAL |f_70(PythonFactory): PythonFactory failed to close properly
2017-12-11 15:09:53| 0.6| 0.0|ERROR |f_70(PythonFactory): A fatal error has occurred. Check the logfile above for details

 

I would also be very interested in checking all the licences on a specific server in one shot, or at least all the licences for a specific version of flexlm.  (We have 6 software requiring 3 versions of flexlm on the same license server)

 

@jdh so in that case would you want to be able to point this at multiple license server root directories at the same time or, would more than 1 FlexNetAnalyser in the same process with shared parameters be acceptable? Thanks for the feedback!

 


jdh
Contributor
Forum|alt.badge.img+40
  • Contributor
  • December 18, 2017
There is definitely a problem with the Python, I use it to handle a pivot of the license usage stats and that part needs cleaning up before I release this properly. But, thank you very much for the comments as that highlights what I need to test when I give this more than the "hack" I've given it so far. Happy Christmas all!

 

 

I'll be happy to beta test. This is the "hack" I'm most likely to benefit from - even more than my own :)

davideagle
Contributor
Forum|alt.badge.img+22
  • Contributor
  • December 18, 2017
Yeah, what's up with the password? Got some dirty tricks to hide, Dave? ;-)
No, just some dirty Python I need to give some more attention! But then you know all about that kind of thing eh! ;-)

 

 


jdh
Contributor
Forum|alt.badge.img+40
  • Contributor
  • December 18, 2017
@jdh so in that case would you want to be able to point this at multiple license server root directories at the same time or, would more than 1 FlexNetAnalyser in the same process with shared parameters be acceptable? Thanks for the feedback!

 

In my dream world I would be able to point it to a machine, and it would magically figure out all the flexnet licenses in use.

 

 

In a more realistic world, I would point it to a specific license server root directory, but not have to specify the port, and it would return all the licenses visible on that instance.

davideagle
Contributor
Forum|alt.badge.img+22
  • Contributor
  • December 18, 2017
In my dream world I would be able to point it to a machine, and it would magically figure out all the flexnet licenses in use.

 

 

In a more realistic world, I would point it to a specific license server root directory, but not have to specify the port, and it would return all the licenses visible on that instance.
Your dreams are partly intreguing and partly unnerving... I'll see what I can do!

 

 


Forum|alt.badge.img
  • December 18, 2017
In my dream world I would be able to point it to a machine, and it would magically figure out all the flexnet licenses in use.

 

 

In a more realistic world, I would point it to a specific license server root directory, but not have to specify the port, and it would return all the licenses visible on that instance.
How about the dream world where FME doesn't use Flexnet? :) #comingsoon

 

 


davideagle
Contributor
Forum|alt.badge.img+22
  • Contributor
  • December 18, 2017
In my dream world I would be able to point it to a machine, and it would magically figure out all the flexnet licenses in use.

 

 

In a more realistic world, I would point it to a specific license server root directory, but not have to specify the port, and it would return all the licenses visible on that instance.
Thankfully this transformer also works with your Esri and Autodesk license servers too... #aintgoingawayanytimesoon

 

 


jdh
Contributor
Forum|alt.badge.img+40
  • Contributor
  • December 18, 2017
How about the dream world where FME doesn't use Flexnet? :) #comingsoon

 

 

Honestly, not something I'm really looking forward to, as I still need flexnet for my esri, autodesk, presagis and hexagon software, so it'll just mean I need to manage licenses in different interfaces.