Skip to main content
Question

How to pull the table data from nextgenstats.nfl.com?

  • November 13, 2017
  • 8 replies
  • 31 views

Forum|alt.badge.img

So I am having some issue's in pulling the table data from www.nextgenstats.nfl.com.

I have tried the HTMLTable reader, and I keep receiving an error message when trying to connect to it. Any suggestions would be great!

This post is closed to further activity.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.

8 replies

LizAtSafe
Safer
Forum|alt.badge.img+18
  • Safer
  • 1542 replies
  • November 13, 2017

Hi @jasonberneathy what error message are you recieving?


Forum|alt.badge.img
  • Author
  • 4 replies
  • November 13, 2017

Hi @jasonberneathy what error message are you recieving?

hello @lizsanderson So trying to connect to this table at: https://nextgenstats.nfl.com/stats/passing#yards

 

I keep recieving a "Failed to retrieve feature types." Error...

 

With the following listed in the log.

 

HTML Table Reader: Could not open HTML file 'https://nextgenstats.nfl.com/stats/passing#yards' for reading. ' ' : 'No such file or directory'

 

followed by:

 

Failed to obtain any schemas from reader 'HTMLTABLE' from 1 dataset. This may be due to invalid datasets or format accessibility issues due to licensing, dependencies, or module loading. See logfile for more information.

 

 


jeroen
Contributor
Forum|alt.badge.img+15
  • Contributor
  • 97 replies
  • November 13, 2017

If you just need the data for one time a workaround for you could be the following. The Table is accessible in the console /inspector of your browser (i used firefox for this try). I got the information you needed by going to the options - developer - inspector. Then go to the section you want and right click on it and choose edit as html and save the text. After this you can parse the data in FME and get it back into the same structure.

https://developer.mozilla.org/nl/docs/Tools/Page_Inspector/How_to/Open_the_Inspector


Forum|alt.badge.img
  • Author
  • 4 replies
  • November 13, 2017

If you just need the data for one time a workaround for you could be the following. The Table is accessible in the console /inspector of your browser (i used firefox for this try). I got the information you needed by going to the options - developer - inspector. Then go to the section you want and right click on it and choose edit as html and save the text. After this you can parse the data in FME and get it back into the same structure.

https://developer.mozilla.org/nl/docs/Tools/Page_Inspector/How_to/Open_the_Inspector

Thank you for your answer, and it would work as a temporary solution. However I would like to pull this data in real time (since it changes from week to week) when running the workbench rather than creating a bunch of HTML files to run the workbench.

 

 


steveatsafe
Safer
Forum|alt.badge.img+13
  • Safer
  • 422 replies
  • November 13, 2017

Hi @jasonberneathy what error message are you recieving?

I had a look at the URL also...same error. Seems it isn't finding any Tables. I always thought this only worked with static content but I see reviewing the help that it should work with content that is created on the fly.

mark2atsafe
Safer
Forum|alt.badge.img+56
  • Safer
  • 2554 replies
  • November 14, 2017
I had a look at the URL also...same error. Seems it isn't finding any Tables. I always thought this only worked with static content but I see reviewing the help that it should work with content that is created on the fly.
Although I'm not sure if it would raise that error message, is it actually a HTML "table" the web page is creating? Or are they creating something that looks like a table, but isn't a HTML table tag?

 


Forum|alt.badge.img
  • Author
  • 4 replies
  • November 14, 2017
Although I'm not sure if it would raise that error message, is it actually a HTML "table" the web page is creating? Or are they creating something that looks like a table, but isn't a HTML table tag?

 

It appears to be pulling the data from json. However I am unaware of how to access it remotely with FME.

 

 


Forum|alt.badge.img
  • 43 replies
  • November 15, 2017
It appears to be pulling the data from json. However I am unaware of how to access it remotely with FME.

 

 

Hi, you can find the JSON link in Firefox/Chrome dev console with request type XHR. (https://appapi.ngs.nfl.com/statboard/passing?season=2017&seasonType;=REG)

 

Then in FME use JSON reader, put the JSON link for JSON source dataset, then Schema Scan mode to JSON Query, and add one Feature Type 'json_featuretype' with Query 'json["stats"][*]'