Crawler Policy
ExamsKiTayari.com operates a polite web crawler, ExamsKiTayariBot, to monitor official exam and recruitment websites for updates.
Good-citizen principles
- Respects robots.txt. ExamsKiTayariBot reads and honours each site’s
robots.txtdirectives, including disallowed paths and crawl-delay. - Rate limited. The crawler paces its requests to avoid placing load on official servers, with conservative concurrency and back-off on errors.
- Identifies itself. Every request sends a descriptive User-Agent string with a contact URL so site operators can reach us.
- Read-only. The crawler only reads publicly available pages. It never submits forms, logs in, or attempts to access restricted content.
User-agent string
Our crawler identifies itself using a User-Agent in the following format:
ExamsKiTayariBot/1.0 (+https://examskitayari.com/bot)Questions or concerns
If you operate an official website and have questions about our crawler, or would like to adjust how we access your site, please reach out via the contact information on our platform. We respond promptly to all requests from site operators.