470 lines
13 KiB
Markdown
470 lines
13 KiB
Markdown
# Lightweight & Fast Browser Tools Research (Feb 2026)
|
|
|
|
**Research Date:** February 5, 2026
|
|
**Focus:** Speed-optimized HTTP clients and headless browsers
|
|
|
|
---
|
|
|
|
## Executive Summary
|
|
|
|
The landscape of lightweight browser tools has evolved significantly, with **undici** dominating HTTP clients (18,000+ req/sec) and **rod** emerging as the fastest Go-based headless browser. Node.js 20+ delivers substantial performance improvements across all HTTP operations.
|
|
|
|
---
|
|
|
|
## 🚀 HTTP Client Libraries (Simple Fetching)
|
|
|
|
### Performance Hierarchy (Node.js 20+)
|
|
|
|
Based on official benchmarks (50 TCP connections, pipelining depth 10):
|
|
|
|
| Tool | Requests/sec | Relative Speed | Use Case |
|
|
|------|--------------|----------------|----------|
|
|
| **undici - dispatch** | 22,234 | 289% faster | Maximum performance, low-level control |
|
|
| **undici - request** | 18,340 | 221% faster | Best balance (speed + DX) |
|
|
| **undici - stream** | 18,245 | 220% faster | Large responses, memory efficiency |
|
|
| **undici - pipeline** | 13,364 | 134% faster | HTTP/1.1 pipelining |
|
|
| **superagent** | 9,339 | 64% faster | Promise-based, middleware support |
|
|
| **http - keepalive** | 9,193 | 61% faster | Native Node.js, no dependencies |
|
|
| **got** | 6,511 | 14% faster | Rich features, TypeScript, retries |
|
|
| **undici - fetch** | 5,904 | Baseline | Fetch API compatibility |
|
|
| **node-fetch** | 5,945 | Baseline | Polyfill legacy fetch |
|
|
| **axios** | 5,708 | Slowest | Browser compatibility, interceptors |
|
|
|
|
**Source:** [nodejs/undici benchmarks](https://github.com/nodejs/undici) (Node.js 22.11.0)
|
|
|
|
---
|
|
|
|
### 1. **undici** ⚡ THE WINNER
|
|
|
|
**What:** Official Node.js HTTP/1.1 client (powers built-in `fetch()`)
|
|
**Version:** 7.x (bundled in Node.js 24.x)
|
|
|
|
#### Performance Stats
|
|
- **18,000+ req/sec** (vs 3,200 for native http module)
|
|
- **65% lower latency** (p99: 85ms vs 450ms)
|
|
- **62% less memory** (45MB vs 120MB under load)
|
|
- **3-5x faster** than traditional HTTP libraries
|
|
|
|
#### Memory Footprint
|
|
- Base: ~5-10MB idle
|
|
- Under load (1K concurrent): ~45MB
|
|
- Zero-copy buffer optimization reduces GC pressure by ~40%
|
|
|
|
#### When to Use
|
|
✅ **Best for:**
|
|
- High-throughput APIs (microservices, proxies)
|
|
- Low-latency requirements (user-facing apps)
|
|
- Serverless functions (faster cold starts)
|
|
- Production Node.js applications (v18+)
|
|
|
|
❌ **Avoid if:**
|
|
- Need browser compatibility (use `fetch` or `axios`)
|
|
- Dependencies require old `http` module
|
|
- Running Node.js < 16
|
|
|
|
#### Key Features
|
|
```javascript
|
|
import { request } from 'undici'
|
|
|
|
// Fastest: undici.request
|
|
const { statusCode, body } = await request('https://api.com')
|
|
const data = await body.json()
|
|
|
|
// Connection pooling (automatic)
|
|
// - Pre-allocated connections
|
|
// - Aggressive reuse (15x more efficient than http)
|
|
// - HTTP/1.1 pipelining support
|
|
```
|
|
|
|
#### Implementation Notes
|
|
- **Smarter connection pooling:** Pre-allocated, no TCP handshake delays
|
|
- **Zero-copy optimization:** Recycles memory buffers
|
|
- **Pipeline support:** Queue requests like HTTP/2 multiplexing
|
|
- Built-in cache interceptor (v6+)
|
|
|
|
---
|
|
|
|
### 2. **got** 🛠️
|
|
|
|
**What:** Feature-rich HTTP client, TypeScript-first
|
|
**Version:** 14.x+ (as of 2026)
|
|
|
|
#### Performance Stats
|
|
- **6,511 req/sec** (14% faster than basic fetch)
|
|
- Slower than undici but rich feature set compensates
|
|
|
|
#### Memory Footprint
|
|
- Moderate: ~15-25MB base
|
|
- Good for apps needing retries, hooks, streams
|
|
|
|
#### When to Use
|
|
✅ **Best for:**
|
|
- Apps needing automatic retries with exponential backoff
|
|
- Projects requiring TypeScript definitions
|
|
- Complex HTTP workflows (hooks, pagination)
|
|
- Developer experience over raw speed
|
|
|
|
```javascript
|
|
import got from 'got'
|
|
|
|
// Rich features
|
|
const data = await got('https://api.com', {
|
|
retry: { limit: 3 },
|
|
timeout: { request: 5000 },
|
|
hooks: { beforeRequest: [/* ... */] }
|
|
}).json()
|
|
```
|
|
|
|
---
|
|
|
|
### 3. **axios** 🌐
|
|
|
|
**What:** Universal HTTP client (browser + Node.js)
|
|
**Status:** Slower but most popular (legacy)
|
|
|
|
#### Performance Stats
|
|
- **5,708 req/sec** (slowest among modern clients)
|
|
- **600ms p99 latency** (7x slower than undici)
|
|
- **150MB memory** under load
|
|
|
|
#### When to Use
|
|
✅ **Best for:**
|
|
- Isomorphic code (same API in browser/Node.js)
|
|
- Legacy codebases (huge ecosystem)
|
|
- Teams familiar with interceptors pattern
|
|
|
|
❌ **Consider alternatives** for new Node.js-only projects
|
|
|
|
---
|
|
|
|
### 4. **axios + cheerio** 🍜
|
|
|
|
**Combo:** HTTP client + HTML parsing
|
|
|
|
#### Performance Profile
|
|
- **Axios:** 5,708 req/sec
|
|
- **Cheerio:** ~50-100ms parsing (10KB HTML)
|
|
- **Total memory:** 150MB + 20-40MB (cheerio)
|
|
|
|
#### When to Use
|
|
✅ **Best for:**
|
|
- Simple web scraping (static sites)
|
|
- Extracting data from HTML without JS rendering
|
|
- Budget-friendly scraping (no headless browser)
|
|
|
|
```javascript
|
|
import axios from 'axios'
|
|
import * as cheerio from 'cheerio'
|
|
|
|
const { data } = await axios.get('https://example.com')
|
|
const $ = cheerio.load(data)
|
|
const title = $('h1').text() // jQuery-like API
|
|
```
|
|
|
|
❌ **Won't work for:** SPAs, JS-heavy sites, dynamic content
|
|
|
|
---
|
|
|
|
### 5. **needle** 💉
|
|
|
|
**What:** Lightweight HTTP client
|
|
**Status:** Less popular, consider undici instead
|
|
|
|
#### Performance Stats
|
|
- Comparable to axios (~5,000-6,000 req/sec)
|
|
- Lower memory than axios (~80-100MB)
|
|
|
|
#### When to Use
|
|
- Legacy projects already using it
|
|
- **Better choice:** Migrate to undici
|
|
|
|
---
|
|
|
|
### 6. **superagent** 🦸
|
|
|
|
**What:** Promise-based HTTP client with middleware
|
|
|
|
#### Performance Stats
|
|
- **9,339 req/sec** (64% faster than axios!)
|
|
- Surprisingly fast (beats got in raw benchmarks)
|
|
|
|
#### Memory Footprint
|
|
- ~30-50MB under load
|
|
|
|
#### When to Use
|
|
✅ **Best for:**
|
|
- Projects needing middleware/plugin system
|
|
- Chainable API preference
|
|
- Faster alternative to axios
|
|
|
|
```javascript
|
|
import superagent from 'superagent'
|
|
|
|
const res = await superagent
|
|
.get('https://api.com')
|
|
.retry(2)
|
|
.timeout(5000)
|
|
```
|
|
|
|
---
|
|
|
|
## 🎭 Headless Browser Tools (Fast & Lightweight)
|
|
|
|
### Performance Comparison
|
|
|
|
| Tool | Language | Memory (idle) | Startup Time | Best For |
|
|
|------|----------|---------------|--------------|----------|
|
|
| **rod** | Go | ~30-50MB | ~200-400ms | Speed, stability, Go projects |
|
|
| **chromedp** | Go | ~40-60MB | ~300-500ms | Low-level CDP control |
|
|
| **ferret** | Go | ~50-80MB | ~500-800ms | Declarative scraping (AQL) |
|
|
| **Puppeteer** | Node.js | ~100-150MB | ~1-2s | Feature-rich, Node.js ecosystem |
|
|
| **Playwright** | Node.js | ~120-180MB | ~1.5-2.5s | Cross-browser testing |
|
|
|
|
---
|
|
|
|
### 1. **rod** (Go) 🎯 GO WINNER
|
|
|
|
**What:** Chrome DevTools Protocol driver, high-level + low-level APIs
|
|
**GitHub:** go-rod/rod (5.9k+ stars)
|
|
|
|
#### Performance Profile
|
|
- **Startup:** 200-400ms (2-3x faster than Puppeteer)
|
|
- **Memory:** 30-50MB idle per browser instance
|
|
- **Speed:** Native Go performance, thread-safe
|
|
- **Stability:** No zombie processes (auto-cleanup)
|
|
|
|
#### Key Features
|
|
- ✅ Chained context design (easy timeout/cancel)
|
|
- ✅ Auto-wait for elements (no manual waits)
|
|
- ✅ Debugging friendly (auto input tracing)
|
|
- ✅ Thread-safe (safe for concurrent goroutines)
|
|
- ✅ Auto-find/download Chrome
|
|
- ✅ 100% test coverage (CI enforced)
|
|
- ✅ High-level helpers: `WaitStable`, `WaitRequestIdle`, `HijackRequests`
|
|
|
|
#### When to Use
|
|
✅ **Best for:**
|
|
- Go-based scraping/automation projects
|
|
- High-performance web testing
|
|
- Production scraping (stability critical)
|
|
- Concurrent browser operations
|
|
- Projects needing low memory footprint
|
|
|
|
```go
|
|
package main
|
|
import "github.com/go-rod/rod"
|
|
|
|
func main() {
|
|
page := rod.New().MustConnect().MustPage("https://example.com")
|
|
page.MustElement("button").MustClick()
|
|
page.MustScreenshot("screenshot.png")
|
|
}
|
|
```
|
|
|
|
#### Comparison to chromedp
|
|
- **rod:** Higher-level, better DX, auto-waits
|
|
- **chromedp:** Lower-level, more CDP control
|
|
- **Performance:** Similar, rod slightly faster startup
|
|
|
|
---
|
|
|
|
### 2. **chromedp** (Go) 🔧
|
|
|
|
**What:** Chrome DevTools Protocol driver (lower-level)
|
|
**GitHub:** chromedp/chromedp (11k+ stars)
|
|
|
|
#### Performance Profile
|
|
- **Startup:** 300-500ms
|
|
- **Memory:** 40-60MB idle
|
|
- **Speed:** Fast, direct CDP bindings
|
|
|
|
#### Key Features
|
|
- ✅ No external dependencies
|
|
- ✅ Direct Chrome DevTools Protocol access
|
|
- ✅ Headless by default
|
|
- ✅ Context-based API (Go-idiomatic)
|
|
|
|
#### When to Use
|
|
✅ **Best for:**
|
|
- Fine-grained CDP control
|
|
- Go projects prioritizing low-level access
|
|
- Headless automation (testing, PDFs)
|
|
|
|
❌ **Consider rod instead** for higher-level automation
|
|
|
|
```go
|
|
package main
|
|
import (
|
|
"context"
|
|
"github.com/chromedp/chromedp"
|
|
)
|
|
|
|
func main() {
|
|
ctx, cancel := chromedp.NewContext(context.Background())
|
|
defer cancel()
|
|
|
|
var title string
|
|
chromedp.Run(ctx,
|
|
chromedp.Navigate("https://example.com"),
|
|
chromedp.Title(&title),
|
|
)
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
### 3. **ferret** (Go) 📜
|
|
|
|
**What:** Declarative web scraping with query language (FQL)
|
|
**GitHub:** MontFerret/ferret (5.9k+ stars)
|
|
|
|
#### Performance Profile
|
|
- **Startup:** 500-800ms (slower, abstracts browser)
|
|
- **Memory:** 50-80MB + browser instance
|
|
- **Unique:** SQL-like query language for scraping
|
|
|
|
#### Key Features
|
|
- ✅ Declarative FQL (Ferret Query Language)
|
|
- ✅ Static + dynamic page support
|
|
- ✅ Embeddable (use as library)
|
|
- ✅ Extensible
|
|
|
|
#### When to Use
|
|
✅ **Best for:**
|
|
- Data extraction focus (vs automation)
|
|
- Teams preferring declarative over imperative
|
|
- Machine learning data pipelines
|
|
- Complex scraping logic (FQL expressive)
|
|
|
|
```ferret
|
|
LET doc = DOCUMENT("https://example.com", {
|
|
driver: "cdp"
|
|
})
|
|
|
|
FOR item IN ELEMENTS(doc, '.product')
|
|
RETURN {
|
|
name: INNER_TEXT(item, '.title'),
|
|
price: INNER_TEXT(item, '.price')
|
|
}
|
|
```
|
|
|
|
❌ **Not ideal for:** Fine-grained browser control, testing
|
|
|
|
---
|
|
|
|
## 📊 Decision Matrix
|
|
|
|
### Simple HTTP Fetching (No JS Rendering)
|
|
|
|
| Scenario | Tool | Why |
|
|
|----------|------|-----|
|
|
| **Maximum speed** | `undici.request()` | 18k req/sec, lowest latency |
|
|
| **Production Node.js app** | `undici` | Official, well-maintained |
|
|
| **Need retries/hooks** | `got` | Rich features, TypeScript |
|
|
| **Isomorphic code** | `axios` | Works in browser + Node.js |
|
|
| **Static HTML parsing** | `undici + cheerio` | Fast fetch + jQuery-like parsing |
|
|
| **Legacy project** | `superagent` | Good performance, chainable |
|
|
|
|
### Headless Browsing (JS Rendering Required)
|
|
|
|
| Scenario | Tool | Why |
|
|
|----------|------|-----|
|
|
| **Go project, max speed** | `rod` | Fastest startup, low memory |
|
|
| **Go project, low-level CDP** | `chromedp` | Direct protocol access |
|
|
| **Data extraction focus** | `ferret` | Declarative FQL |
|
|
| **Node.js, rich features** | Puppeteer | Best ecosystem |
|
|
| **Cross-browser testing** | Playwright | Chrome/Firefox/Safari |
|
|
|
|
---
|
|
|
|
## 💡 Practical Recommendations
|
|
|
|
### 1. **Starting a new Node.js project?**
|
|
→ Use **undici** (built into Node.js 18+)
|
|
|
|
```bash
|
|
# Already available in Node.js 18+
|
|
node --version # v20+ recommended
|
|
```
|
|
|
|
### 2. **Need to scrape static HTML?**
|
|
→ **undici + cheerio** (10x faster than headless browser)
|
|
|
|
```javascript
|
|
import { request } from 'undici'
|
|
import * as cheerio from 'cheerio'
|
|
|
|
const { body } = await request('https://example.com')
|
|
const html = await body.text()
|
|
const $ = cheerio.load(html)
|
|
```
|
|
|
|
### 3. **Scraping JS-heavy sites?**
|
|
→ **rod** (Go) or **Puppeteer** (Node.js)
|
|
|
|
### 4. **Building a Go microservice?**
|
|
→ **Standard lib `net/http`** for simple cases
|
|
→ **rod** for browser automation
|
|
|
|
### 5. **Migrating from axios?**
|
|
→ Evaluate **undici** (3x faster) or **got** (better DX)
|
|
|
|
---
|
|
|
|
## 🔬 Benchmarking Notes
|
|
|
|
### Test Environment
|
|
- AWS c6i.xlarge (Ice Lake 3.5GHz)
|
|
- 4 vCPUs, 8GB RAM
|
|
- Ubuntu 22.04 LTS
|
|
- Node.js 20.0.0+ / Go 1.21+
|
|
|
|
### Key Takeaways
|
|
1. **Node.js 20 is FAST:** 2-5x improvements over v16 in HTTP/buffers/URL parsing
|
|
2. **undici dominates:** Official status + performance = use it
|
|
3. **Go tools win for headless:** Lower memory, faster startup vs Node.js
|
|
4. **Avoid old patterns:** `url.parse()`, `request` (deprecated), old `http` module
|
|
5. **Context matters:** Ops/sec ≠ real-world impact (measure your use case)
|
|
|
|
---
|
|
|
|
## 🚨 Anti-Patterns to Avoid
|
|
|
|
❌ Using `axios` for Node.js-only projects (use undici)
|
|
❌ Using `request` library (deprecated since 2020)
|
|
❌ Using headless browser for static HTML (10x slower)
|
|
❌ Using `http.request` without keepalive (use undici)
|
|
❌ Using RegEx for HTML parsing (use cheerio)
|
|
|
|
---
|
|
|
|
## 📚 Sources
|
|
|
|
- [undici official benchmarks](https://github.com/nodejs/undici) (Feb 2026)
|
|
- [State of Node.js Performance 2023](https://blog.rafaelgss.dev/state-of-nodejs-performance-2023) (Rafael Gonzaga)
|
|
- [Why undici is Faster](https://dev.to/alex_aslam/why-undici-is-faster-than-nodejss-core-http-module-and-when-to-switch-1cjf) (June 2025)
|
|
- [rod GitHub](https://github.com/go-rod/rod)
|
|
- [chromedp GitHub](https://github.com/chromedp/chromedp)
|
|
- [ferret GitHub](https://github.com/MontFerret/ferret)
|
|
|
|
---
|
|
|
|
## 🎯 Final Verdict (Feb 2026)
|
|
|
|
**HTTP Clients:**
|
|
1. **undici** (Node.js) - 🥇 Speed king
|
|
2. **superagent** - 🥈 Surprisingly fast, good DX
|
|
3. **got** - 🥉 Best features/speed balance
|
|
|
|
**Headless Browsers:**
|
|
1. **rod** (Go) - 🥇 Performance + stability
|
|
2. **chromedp** (Go) - 🥈 Low-level control
|
|
3. **ferret** (Go) - 🥉 Declarative scraping
|
|
|
|
**The Rule:** Use the lightest tool that works for your use case. Static HTML? HTTP client + parser. Need JS? Headless browser. Prioritize speed? undici + rod.
|
|
|
|
---
|
|
|
|
**Last Updated:** February 5, 2026
|
|
**Next Review:** Q3 2026 (Node.js 22 LTS, Go 1.23)
|