Free Google Search scraper and SERP API – Apify
FeaturesThis SERP API actor crawls Google Search Result Pages (SERP or SERPs) and extracts data from the HTML in a structured format such as JSON, XML or Excel. Specifically, the actor extracts the following data from each SERP:
Organic results
Ads
Product ads
Related queries
People also ask
Price, reviews rating and count (under productInfo field if available)
Additional custom attributes
Note that the actor doesn’t support special types of Google searches, such as Google Shopping,
Google Images or Google News.
SERP APIOur Google Search Results Scraper gives you a RESTful SERP API that provides real-time results optimized for structured JSON output that you can download and use any way you want.
Cost of usageThe actor is free to use, but to scrape SERPs effectively, you should use Apify Proxy and need to have a sufficient limit for Google SERP queries (you can see the limit on your Account page).
New Apify users have a free trial of Apify Proxy and Google SERPs, so you can use the actor for free at the beginning.
Once the Apify Proxy trial expires, you’ll need to subscribe to a paid plan in order to keep using the actor. If you need to increase your Google SERP limit or have any questions, please email
Use casesGoogle Search processes over 3. 5 billion searches per day, so it’s really important for businesses to know how they rank on Google. Unfortunately, Google Search does not provide a public API, so the only way to monitor search results and ranking is to use web scraping.
Our free googlescraper tool gives you your own, customizable SERP scraper. You can do whatever you want with the SERP data once you extract and download it.
Typical use cases include:
Search engine optimization (SEO)
— Monitor how your website performs on Google for certain queries over time.
Analyze display ads for a given set of keywords.
Monitor your competition in both organic and paid results.
Build a URL list for certain keywords. This is useful if, for example, you need good relevant starting points when scraping web pages containing specific phrases.
Read more in the How to scrape Google Search blog post.
Number of resultsYou can change the number of results per page by using the resultsPerPage parameter. The default is 10 but allowed values are 10-100. You can also set maxPagesPerQuery to get more results for each query.
Please keep in mind that, although Google shows that it internally found millions of results, Google will never display more than a few hundred results per single search query. You can test this in your browser. If you need to get as many results as possible, try to create many similar queries and combine different parameters and locations.
Input settingsThe actor gives you fine-grained control over what kind of Google Search results you’ll get.
You can specify the following settings:
Query phrases or raw URLs
Country
Language
Exact geolocation
Number of results per page
Mobile or desktop version
For a complete description of all settings of the actor, see the input specification.
ResultsThe actor stores its result in the default dataset associated with the actor run, from which you can export it
to various formats, such as JSON, XML, CSV or Excel.
The results can be downloaded from the Get dataset items API endpoint:
DATASET_ID]/items? format=[FORMAT]
where [DATASET_ID] is the ID of the dataset and [FORMAT]
can be csv, html, xlsx, xml, rss or json.
For each Google Search results page, the dataset will contain a single record, which in JSON format looks as follows. Keep in mind that some fields have example values:
{
“searchQuery”: {
“term”: “Hotels in Prague”,
“page”: 1,
“type”: “SEARCH”,
“domain”: “”,
“countryCode”: “cz”,
“languageCode”: “en”,
“locationUule”: null,
“resultsPerPage”: “10”},
“url”: “,
“hasNextPage”: false,
“resultsTotal”: 138000000078,
“relatedQueries”: [
“title”: “cheap hotels in prague”,
“url”: “},
“title”: “best hotels in prague old town”,
//… ],
“paidResults”: [
“title”: “2280 Hotels in Prague | Best Price Guarantee | “,
“displayedUrl”: “,
“description”: “Book your Hotel in Prague online. No reservation costs. Great rates. Bed and Breakfasts. Support in 42 Languages. Hotels. Motels. Read Real Guest Reviews. 24/7 Customer Service. 34+ Million Real Reviews. Secure Booking. Apartments. Save 10% with Genius. Types: Hotels, Apartments, Villas. £0 – £45 Hotels – up to £45. 00/day – Book Now · More£45 – £90 Hotels – up to £90. 00/dayBook Now£130 – £180 Hotels – up to £180. 00/dayBook Now£90 – £130 Hotels – up to £130. 00/dayBook Nowup to £45. 00/dayup to £90. 00/dayup to £180. 00/dayup to £130. 00/day”,
“siteLinks”: [
“title”: “Book apartments and more”,
“description”: “Bookings instantly confirmed! Instant confirmation, 24/7 support”},
“title”: “More than just hotels”,
“description”: “Search, book, stay – get started! Hotels when and where you need them”}]},
“title”: “Hotels In Prague | ™ Official Site”,
“description”: “Hotels In Prague Book Now! Collect 10 Nights and Get 1 Free. Budget Hotels. Guest Reviews. Last Minute Hotel Deals. Luxury Hotels. Exclusive Deals. Price Guarantee. Photos & Reviews. Travel Guides. Earn Free Hotel Nights. No Cancellation Fees. Types: Hotel, Apartment, Hostel. “,
“siteLinks”: []},
“paidProducts”: [],
“organicResults”: [
“title”: “30 Best Prague Hotels, Czech Republic (From $11) – “,
“displayedUrl”: ” › Czech Republic”,
“description”: “Great savings on hotels in Prague, Czech Republic online. Good availability and great rates. Read hotel reviews and choose the best hotel deal for your stay. “,
“siteLinks”: [],
“productInfo”: {
“price”: “$123”,
“rating”: 4. 7,
“numberOfReviews”: 4510}, },
“title”: “The 30 best hotels & places to stay in Prague, Czech Republic… “,
“productInfo”: {}, },
“peopleAlsoAsk”: [
“question”: “What is the name of the best hotel in the world? “,
“answer”: “Burj Al Arab Jumeirah, Dubai. Arguably Dubai’s most iconic hotel, the Burj Al Arab rises above the Persian Gulf on its own man-made island like a giant sail. Everything here is over-the-top, from the gilded furnishings in its guest rooms to the house fleet of Rolls-Royces. “,
“title”: “Best 100 Hotels: World’s Best Hotels 2020 | Travel + Leisure | Travel… “,
“date”: “Jul 8, 2020”}],
“customData”: {
“pageTitle”: “Hotels in Prague – Google Search”}},
How to get one search result per rowIf you are only interested in organic Google Search results and want to get just one organic or paid result per row on output, simply query the fields=searchQuery, organicResults and unwind=organicResults query parameters to the API endpoint URL:
DATASET_ID]/items? format=[FORMAT]&fields=searchQuery, organicResults&unwind=organicResults
The API will return a result like this (in JSON format):
[
“term”: “Restaurants in Prague”,
//… },
“title”: “THE 10 BEST Restaurants in Prague 2019 – TripAdvisor”,
“description”: “Best Dining in Prague, Bohemia: See 617486 TripAdvisor traveler reviews of 6232 Prague restaurants and search by cuisine, price, location, and more. “,
“title”: “The 11 Best Restaurants in Prague | Elite Traveler”,
“description”: “Jan 16, 2018 – With the regional fare certainly a highlight of dining in Prague, a great number of superb international eateries have touched down to become… “,
//… ]
When using a tabular format such as csv or xls, you’ll get a table where each row contains just one organic result. For more details about exporting and formatting the dataset records, please see the documentation of the Get dataset items API endpoint.
Tips and tricks
Crawling the second and subsequent results pages might be slower than the first page.
If you need to scrape a lot of results for a single query, you can greatly improve the speed of the crawl by setting Results per page (resultsPerPage) to 100, instead of crawling 10 pages each with 10 results.
ChangelogGoogle Search Results Scraper is under active development and we regularly introduce new features and fix bugs. We also often have to hotfix the extractor when Google changes the page layout. Check the Changelog for recent updates.
Free Digital Marketing Tools – URL Profiler
We have developed a range of digital marketing tools which are completely free and available to download by anyone – for Windows and Mac users. Each of these tools has been developed to solve a specific problem, so they are very quick and easy to use.
Simple SERP Scraper
The Simple SERP Scraper does exactly what it says – scrapes Google search engine results pages. It is very straightforward to use – simply add in your search prospecting queries (one per line) and scrape Google’s results. The settings allow you to determine the locality and how many results you want to pull The tool has been designed to work without proxies. If you don’t use proxies, make sure to leave the delay on random. If you intend to do bulk prospecting (more than 100 search queries) we’d recommend you use anonymous private proxies.
Twitter List Scraper
Twitter lists are user-generated groups of individual users on Twitter, typically based on a common interest or theme. With the Twitter List Scraper, simply paste in URLs of the member pages, and the tool will return Twitter usernames and profile links of all the members.
Read more on how to use the Twitter List Scraper in this blog post.
10 Best SERP API to Scrape Real-time Search Engine Results …
Google constantly keeps on changing its SERP structure and overall algorithm, so it’s essential to scrape search results via accurate sources.
And it’s all the same with other search engines as well. Most of the things that work right now will soon become a thing of the past. In that case, if you’ll keep on relying on an outdated method of scraping SERP data, you’ll be lost among the trenches.
To avoid that from happening, you need to jump on the solution that works now and in the future.
Before we get started, let’s get the basics right.
What is SERP?
SERP stands for Search Engine Results Pages.
Take this for an example: Whenever you search for something on Google (or other search engines), you get a page full of helpful information related to your query. If you’re searching for cat photos, you’ll get cat photos are shown and relevant articles.
Google SERP has evolved recently. You get the results on the Google search page. This means you don’t need to visit another site to find out what you are looking for. Ex: if you search for “open source vulnerability scanner, ” you get the list of a scanner on the page as shown below.
This is called featured snippet, one of the SERP features. There are many other SERP features available like knowledge panel, site links, image pack, a tweet box, etc.
How to check SERP manually?
Manually checking SERP data used to be easy and reliable in the past. However, these days, it’s not the same anymore. You don’t often get accurate results from it because a lot of elements like your search history, device, and location affect the process.
And no, you can’t even rely on the glorious incognito mode.
However, following the below actions might possibly land you with accurate data:
Sign out of your Gmail account
Clear cookies and history
Change your location to the one you’re searching in
Open a new tab or window for every new search
Even if it does throw up accurate results, it’s still a lot of manual work. I am a huge fan of saving time, so here’s what you need to know about using SERP API.
Saves time
Imagine spending half of your day’s time trying to do things manually and ending up not focusing on aspects that are more important for your business. That’d be terrible.
Fortunately, using SERP API can ridiculously save you a ton of time and cut down on the labor work.
Accurate results
As I mentioned earlier, checking SERP manually can be a hit or miss sometimes. There are a lot of factors that you need to take care of to make sure you’re getting the right results. However, such is not the case with SERP API. You’re guaranteed only to receive the most accurate data, every time.
No rocket science
Using a SERP API does not require you to have any hardcore coding knowledge. Most of the technical stuff is already done by the source from which you’re getting the API. This makes sure you don’t have to touch those “complicated” areas.
Enough of theory. Let’s explore the API solutions.
Serpstack
With real-time and super accurate Google search results, Serpstack is hands down one of my favorites in this list. It is completed based on JSON REST API and goes well with every programming language out there.
Moreover, it covers all of the critical search engine elements like:
Web
Images
Videos
News
Shopping
Some of the features that are worth talking about are:
No-waiting processing of high-volume API requests
Various customization options
Robust proxy network enabled
Super tight security with 256-bit SSL encryption
Straightforward integration
Lightning-fast speed of API requests
You can use this API free for the first 100 requests/month after that pricing plans start from just $29. 99 per month.
Oxylabs
With a Trustpilot score of 4. 5/5 stars, Oxylabs’s SERP Scraper API does an excellent job of giving you public data from the main search engines out there.
Whether you want to use it for brand monitoring, ads data tracking, keyword data mining, or other reason, it lets you collect data effortlessly. Here are some main features of it:
Receive data from both, organic and paid results
100% data delivery
Structured data in CSV and as well as JSON format
Highly scalable and easy to integrate
Hassle-free data extraction on their part so you don’t have to do much
On top of that, you get your own dedicated account manager, 24/7 support, and any other necessary help to ensure a smooth flow of the process.
Zenserp
Built with the intention of “speed” in mind, Zenserp is another popular choice that makes scraping Google search results a breeze. You can easily integrate this solution via browser, CURL, Python,, or PHP.
It lets you scrape through the latest SERP elements like:
Organic results
Paid results
Answer box
Featured snippet
Maps
Zenserp comes with some really impressive features like:
Real-time results
Ability to view location-based results
Super accurate data
Ability to handle high volume requests
This API also costs $29. 99 for the starting plan and comes with a free version as well with limitations.
SERPMaster
SERPMaster prides itself as being the lowest cost Google SERP scraper with guaranteed data delivery for each request sent. Getting started with their API is easy, as code examples are provided for cURL, Python, PHP, and integrations, as well as making requests from one’s web browser.
Features
Data from all SERP features (organic, ads, featured snippet, etc)
Global location support
SERPs from any device or browser
Real-time data scraping
JavaScript rendering
Batch requests
24/7 support
SERPMaster plans provide the best value for money on the market. You can start using their API for just $20 per month.
SerpWow
With over 620 satisfied customers, SerpWow can be a great choice for reliable SERP data. It lets you scrape various search engines like:
Google
Yahoo
Bing
Baidu
Yandex
Naver
It allows scraping desktop, mobile, and as well as tablet results. Other features include:
Worldwide locations
Structured data via JSON, CSV or HTML
Easy integration via Python, PHP,, CURL or Galang
Works seamlessly without proxies
Can handle high volume API requests with ease
You can make use of their free trial first and then pick one of their premium plans starting from just $9 per month.
SerpAPI
Trusted by companies like IBM, Harvard University, and The Hoth, SerpAPI is packed with advanced features that can make scraping SERP super smooth. Here are some of them:
No waiting, real-time results
Solves CAPTCHAs automatically
Ability to get results from any location
JSON Results like maps, shopping and knowledge graph
Super security with Legal US Shield
Premium plans start from just $50/month, and you can test it out with the free trial available.
DataForSeo
Another brilliant option for SERP API, DataForSEO supports most of the major search results like:
Featured snippets
Rich snippets
Answer boxes
Top stories
Knowledge graph
This API has one of the most support teams that’ll help you throughout the process of testing and integrating. Some features that are worth mentioning are:
Accurate data
Fast processing of results
All worldwide locations results
Postback/Pingback options
Organized view of top 100 organic and paid results
Very easy to integrate
Informational stats that indicate the usage of your API
DataForSEO uses the “Pay Per Use” pricing structure, which means you only pay for what you’ve used. Nothing more, nothing less. The entry-level payment amount starts at $50-$100 and has a free trial as well.
I recommend checking out their pricing page for a better understanding.
Apify
Apify’s Google search scraper crawls the SERPs and returns back various data in the groups of:
Ads
Organic
Product ads
Related queries
It has some pretty useful features like the ability to search within a specific location and extract custom attributes. In addition, you can keep a watch on what your competitors are ranking and also analyze ads for your chosen keywords.
There is a lot more to it, and you can find everything out by taking their free trial. Premium pricing plans start from just $49/month.
GeoRanker
With the ability to scrape through maps, organic, or ad results, GeoRanker is definitely a great contender that can simplify the task of checking the SERP. Some of the main features include:
Simple integration method
Works for any search engines
Supports all locations
And much more!
They have a free trial that you can get into and then book one of their paid plans by talking to their representatives.
SERPhouse
Last but not least, SERPhouse API works for Google and Bing and is very easy to use since it displays results in the REST JSON format. You can scrape results like:
Carousel
People also ask
They also recently rolled out the ability to fetch images, which is pretty cool. With this API, you get access to features like:
Multiple device selection
Postback/Pingback option
Location and language-specific option
Automatically solves CAPTCHAs
No waiting request processing
It comes with a free plan with limitations, and as well as paid plans starting at just $9. 99/month.
Conclusion
There you have it. I hope the above API will help you save time by automating the process of checking SERP and that, too, accurately. If you are interested in building your own scraper then check out these popular frameworks.