Question

Extract an HTML table from a website using HTML Reader

  • 30 November 2018
  • 3 replies
  • 6 views
durshe80

Good morning, I'm trying to extract a table from a website, but am having a difficult time doing so. The issue is that the website has a pull down option with multiple tables. I'm able to use HTML Table reader to extract the table that comes up as a default, but I need to pull one of the other tables. The URL that I'm using is https://eds.ospi.k12.wa.us/DirectoryEDS.aspx. In upper left, you have the option to select Reports by either District or School. District comes up as the default; but I am actually interested in the school table. Can someone help me achieve this? Using the HTML Reader, I'm able to see the default table that comes up on the website which is gvESDDirectory. However, the table that I'm looking for is called gvSchoolDirectory. I can see it using developer tools in Chrome. I'm just not able to figure out how to get FME to see that table.

 

Thanks!

Eugene D.

3 replies

fmelizard
Rank
Userlevel 4
Badge +13

Sadly it doesn't look like that web page can be loaded with School as the default. I snooped in the page source and it has this in it:

<select name="ctl00$PageContent$ddlReport" onchange="javascript:setTimeout(&#39;
__doPostBack(\&#39;ctl00$PageContent$ddlReport\&#39;,\&#39;\&#39;)&#39;, 0)" id="PageContent_ddlReport">
<option selected="selected" value="0">District</option>
<option value="1">School</option>

 

So it seems like it only refreshes the report when the choice is changed.

Maybe someone out there knows a trick that could be used to put something on the URL to get it to load up with School instead of District as the default -- that is really the only way to get at it. I checked the source of the page when it first loads, before one changes that selection, and the schools aren't in there at that point.

 

durshe80

Thank you Dale. I was hoping that I was just missing some parameter in a transformer. I did see that snippet of code via developer tools and was hoping that maybe there was a way to use the javascript transformer to get around it, but don't have enough knowledge on possibilities.

J
Badge +2

Hi @durshe80

 

 

I agree with what @daleatsafe said. I've been having a look to see what the request looks like when you select School, or Export to Excel. Unfortunately, the header/body in the request are so big it's causing my computer issues with building the request in FME or Postman, so I haven't actually been able to confirm that that method would work.

 

 

You may have to work with the data after exporting it to Excel manually I'm afraid.

Reply


Community Stats

28,710
Posts
109,487
Replies
36,559
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.