CRAWLER MANIFEST //
StoreXrayBot Crawler
The storexray.app domain utilizes a dedicated search crawler to analyze the tech stack and basic public metrics of open Shopify storefronts.
User-Agent Identification
Our crawler identifies itself with the following User-Agent header:
StoreXrayBot/1.0; +https://storexray.app/bot
Polite Crawling Principles
- We respect crawl rules defined in the target website's robots.txt file.
- We respect request delays between pages to prevent overloading target servers.
- We only aggregate publicly accessible data (public product feeds, metadata, review widgets).
How to Block StoreXrayBot
If you wish to prevent our crawler from scanning your storefront, add the following directives to your robots.txt file:
User-agent: StoreXrayBot Disallow: /
Contact Us
If you have any questions, traffic complaints, or crawl rate optimization requests, please contact us at: hello@storexray.app.