Skip to main content
Home/Recipes/Multi-page crawl
Extraction

2.Multi-page crawl

DesignerDeveloperQA

Overview

A homepage extraction misses what lives on pricing pages, dashboards, and docs. Crawling maps the full visual surface of a product, not just what's above the fold.

A single-page extraction only captures tokens visible on the landing page. Most products have additional colors, font weights, spacing values, and component styles that only appear on pricing pages, dashboards, documentation, or error states. The --crawl flag follows internal links up to N pages deep and merges all discovered tokens into a single output. Use --sitemap to discover pages from sitemap.xml instead of following links. You can also use crawl to compare two separate extractions from the same site. For example a marketing page vs. the app, or compare two competing products with identical page structure.

Crawl up to 5 pages

Terminal
dembrandt stripe.com --crawl 5 --save-output

Discover via sitemap

Terminal
dembrandt stripe.com --sitemap --save-output
AI agent prompt: compare internal pages
# Extract two separate sections, then:
"Compare these two extractions from the same product.
One is the marketing site, one is the app.
Which tokens are shared, which differ, and does the
visual language stay consistent across the product boundary?"
Output

Merged token set across all crawled pages. Catches brand values that only appear in secondary UI.

Browse all

All recipes →

43 workflows