Technical documentation
Crawler identity
How KlippradarBot identifies itself and how it behaves when crawling merchant sites.
User-Agent
KlippradarBot/1.0 (+https://klippradar.se/crawler; mailto:[email protected]) - Nordic price comparison crawler. Neutral, no CPC/affiliate. Respects robots.txt.
This is the exact User-Agent you will see in your access logs when our
crawler is probing whether you accept direct requests again after a previous
429 / 403. The purpose suffix is there so your on-call engineer can
identify the bot from the log line alone, without leaving the log viewer.
The contact email in the User-Agent reaches our crawl team directly.
Core rules
- We respect
robots.txt. - We use merchant-specific pacing, concurrency limits, and backoff.
- We slow down or pause on repeated
403,429,5xx, or timeout patterns. - We prefer conditional requests where available, including
ETagandLast-Modified.
Contact
For crawler coordination, whitelisting, or merchant-specific guidance: