Bulk catalog crawling is turned off on this deployment. You can still browse products, search, and add listings manually where supported.

Bulk catalog crawl

Discover product detail pages by following links from seed URLs on the same host, then import each listing with embeddings (same pipeline as manual add). While a run is in progress, the live activity panel shows each step with icons (fetches, saves, skips, errors). Broad seeds like homepages are automatically narrowed to Fliesen-related entry points when possible.

Legal and practical limits

Automated crawling can violate a site's terms of service or overload their servers. Use only data you are allowed to use, keep delay respectful, and prefer official feeds or APIs when available. This tool uses rule-based HTML parsing (JSON-LD, Open Graph, link following) — not an LLM browsing the site. Large runs are best done locally with next dev and conservative limits.

Live activity

Newest first · last 48 events

Nothing running yet

Start a crawl to see fetches, saves, skips, and errors here — each row uses an icon so you can scan the run at a glance.

Seed URLs (one per line)

Start from category or search result pages on the same host; the crawler enqueues same-origin links and saves URLs that look like product detail pages.

Allowed hosts (optional, one per line)

Max pages to fetch

Max products to create

Max queued URLs

Delay between requests (ms)