RentBeagleBot

RentBeagleBot reads the prices that apartment complexes post on their own websites, so renters can see current rents, specials and price history in one place. RentBeagle is free for renters and takes no money from properties.

How it behaves

• It identifies itself in every request: RentBeagleBot/0.1 (+https://rentbeagle.com/bot; renter price tracker).
• It obeys robots.txt, including Crawl-delay, and reads nothing a robots.txt disallows. If a site's robots.txt can't be loaded because of a server error, it reads nothing from that site that day.
• It runs once a day, starting around 6am US Central, and reads only each complex's public floor-plan and pricing pages. It waits at least 5 seconds between requests to an apartment complex's own website, or longer if your Crawl-delay asks. Shared platforms that serve hundreds of complexes from one host (for example a management company's listings site or a floor-plan widget) get at most a few requests per second.
• It does not log in, fill in forms, solve CAPTCHAs, run page scripts or switch IP addresses to get past a block. A site that turns it away (a 401, 403 or 429 answer, or a CAPTCHA) is not asked again that day, and a site that turns it away two runs in a row is tried only once a week after that.
• It runs on GitHub Actions, so its IP addresses change; check the signature rather than the address.
• It never collects from listing services, and it keeps no personal information.
• Requests are signed with HTTP Message Signatures (Web Bot Auth); our public keys are at /.well-known/http-message-signatures-directory.

Blocking or limiting it

Add this to your robots.txt and it will stop on its next visit:

User-agent: RentBeagleBot
Disallow: /

To slow it down instead, add a Crawl-delay (in seconds) under User-agent: RentBeagleBot.

Or email bot@rentbeagle.com with your website address and we'll add it to our do-not-collect list within 3 business days. The same address takes any other question or complaint about the bot.