Cheerio Web Scraping

To scrape a webpage using Cheerio, you need two things:

1. Axios (or any HTTP client) to fetch the HTML of the page.
2. Cheerio to parse the HTML and extract data using jQuery-like syntax.

1. Prerequisites

First, initialize your project and install the necessary packages:

2. The Scraper Code

Here is a complete example that scrapes a website (in this case, a practice site) to get titles and prices.

3. Key Functions Explained

• axios.get(url): Downloads the raw HTML source code of the page.
• cheerio.load(data): Parses the HTML so you can use the $ variable just like jQuery.
• $(selector).each(): Loops through every element that matches the CSS selector.
• .find(): Searches for elements inside the current container.
• .text(): Extracts the text content of an element.
• .attr('attributeName'): Gets the value of an attribute (like href, src, or title).

4. Important Limitations

• Static HTML Only: Cheerio cannot scrape websites that require JavaScript to render (like React or Vue apps). If the page is blank when you scrape it, you might need Puppeteer or Playwright.
• User-Agents: Some websites block scrapers. To fix this, you may need to add a "User-Agent" header to your axios request:

5. How to run it

Save the code as scrape.js and run it in your terminal: