Case study · Web Scraping

RegExp Scraper

Extract custom data patterns from static and JavaScript-rendered websites using configurable regular expressions.

Overview

Scrapes data from supplied URLs using user-defined regular expressions. It supports both static and dynamically rendered pages and can recursively follow links according to a configurable crawl depth.

Problem

Different websites and extraction tasks require different matching rules, and some targets require JavaScript rendering while others can be processed directly from HTML.

Solution

Provides configurable regular-expression patterns together with selectable crawler modes for static or JavaScript-rendered websites.

How it works

- User-defined regular-expression extraction - Static website support - JavaScript-rendered website support - Selectable Cheerio crawler - Selectable Puppeteer and Chrome crawler - Configurable crawl depth - Multiple start URLs - Proxy configuration - Duplicate match prevention - Structured dataset output