Solved

html reporter not designed to handle large records

S
Badge

I have followed the tutorial attached link to build a dashboard to report on 250,000 records in a html report. Unfortunately this is my only option since we don't have FME server 2020.

 

I am spending far too long to work out a way to stop the html file crashing as it tries to load all 250,000 records in the table. I am trying to work out how to convert the table to an array or other in javascript but I believe htmlreportgenerator is not designed for this. If anyone has a solution I can insert in the custom html that would be great.

 

<div id="myTable" >
               <table class="table table-striped">
                  <thead>
                     <tr>
                        <th>Document Number</th>
                        <th>TASK ID</th>
                        <th>Document Stage</th>
                        <th>Document Version</th>
                        <th>Date Completed</th>
                        <th>Plan Status</th>
                     </tr>
                  </thead>
                  <tbody>
                     <tr>
                        <td>PS722713</td>
                        <td>3664</td>
                        <td></td>
                        <td>1</td>
                        <td>20131118000000</td>
                        <td>New</td>
                     </tr>

 


https://community.safe.com/s/article/how-to-customize-html-reports

icon

Best answer by chrisatsafe 8 July 2021, 01:33

View original

2 replies

chrisatsafe
Rank
Badge +2

Hi @sylvia​ ,

The reason the browser is having a tough time keeping up with a 250,000 record table is because the resulting html page is ~54MB in size. An html file of that size will load slowly simply due to the number of lines in the file. A better solution in this case would be serving up the data using a server side pagination so the client's browser doesn't load slowly/freeze/crash or embedding the data in a Google Sheet in the output HTML report using an iFrame (I have done this before and it works great).

 

I have attached an example that works great with smaller table sizes but notice that the page still loads the original html table first then places it in the Javascript table. Not bad for a table up to ~1,000 features but keep in mind, more features = longer loading time. Note: I take no credit for the JS as it was originally posted here. I simply used the concept they showed and implemented it into a workspace.

 

I've also attached the two example outputs where the 1,000 feature output is ~217KB whereas the full 250,000 feature dataset is ~54MB.

 

Hope that helps!

1 Attachment
S
Badge

Hi @sylvia​ ,

The reason the browser is having a tough time keeping up with a 250,000 record table is because the resulting html page is ~54MB in size. An html file of that size will load slowly simply due to the number of lines in the file. A better solution in this case would be serving up the data using a server side pagination so the client's browser doesn't load slowly/freeze/crash or embedding the data in a Google Sheet in the output HTML report using an iFrame (I have done this before and it works great).

 

I have attached an example that works great with smaller table sizes but notice that the page still loads the original html table first then places it in the Javascript table. Not bad for a table up to ~1,000 features but keep in mind, more features = longer loading time. Note: I take no credit for the JS as it was originally posted here. I simply used the concept they showed and implemented it into a workspace.

 

I've also attached the two example outputs where the 1,000 feature output is ~217KB whereas the full 250,000 feature dataset is ~54MB.

 

Hope that helps!

thank you very much Chris, this is very helpful. I had read about pagination on other forums but I wasn't sure what was the best way forward. Thanks for much for the workbench as well.

Reply


Community Stats

28,661
Posts
109,322
Replies
36,528
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.