Wait page to load before getting data with … – Stack Overflow Selenium is good way to solve that, but accepted answer is quite deprecated. As @Seth mentioned in comments headless mode of Firefox/Chrome (or possibly other browsers) should be used instead of PhantomJS. First of all you need to […]
Tag: beautifulsoup wait for javascript
Web Scraping Python Beautifulsoup Javascript
Web-scraping JavaScript page with Python – Stack Overflow I personally prefer using scrapy and selenium and dockerizing both in separate containers. This way you can install both with minimal hassle and crawl modern websites that almost all contain javascript in one form or another. Here’s an example: Use the scrapy […]