Question

How to scrape data from websites with a “Load More” button?

  • 2 April 2020
  • 1 reply
  • 8 views

I'm trying to get imdb movie reviews , for example the movie "The Matrix" has 4K reviews and I need to retrieve all of it.


1 reply

Badge +2

Hi @dani_kelevra,

It looks like IMDb has a number of unofficial APIs which are free for a certain number of requests. It may be worth exploring this as another option to work with instead of scraping data directly from the web page, since there are some examples and documentation it may be easier to work with pagination.

 

 

E.g. https://rapidapi.com/apidojo/api/imdb8?endpoint=apiendpoint_a4ba0a9f-9314-45d1-b50f-3a6b7cb34c69

Reply