• Daily deals
  • Price alerts
  • Categories
  • Stores
  • Insights
lowerSpendings lowerSpendings
Sign in
Daily dealsPrice alertsCategoriesStoresInsights

lowerSpendingsBot

lowerSpendingsBot collects public product prices for lowerSpendings, a price comparison site for Australian shoppers. If you saw lowerSpendingsBot/1.0 in your logs, this page explains what it does.

What it fetches

  • Public product listings: name, price, availability, product identifiers (such as GTINs and manufacturer part numbers), the main image link and the page link.
  • Where a store has a catalogue endpoint or sitemap, it uses that instead of crawling pages. It never logs in, adds to carts or submits forms.
  • It keeps a short excerpt of a product description at most, and every listing on lowerSpendings links back to the store.

How it behaves

  • It reads your robots.txt and follows it, including Crawl-delay.
  • It makes one request at a time to a site, at least 5 seconds apart, usually once a day, and sends conditional requests (ETag / If-Modified-Since) so unchanged pages cost little.
  • If a site answers with 403, 429 or a challenge page, it stops for the rest of the day. It always uses this name and the same addresses: it never changes its identity, IP address or headers to get past a block.

Opting out or asking us to slow down

To stop it, add this to your robots.txt:

User-agent: lowerSpendingsBot
Disallow: /

To ask for a different rate, offer a product feed or anything else, email michael@lowerspendings.com.

lowerSpendings Compare prices, track price drops and spend less.
About Insights How we make money How we rank Stores we compare Privacy Terms Contact Sell on lowerSpendings

lowerSpendings is free to use. We may earn a commission when you buy through our links, and we show ads. This doesn't change the price you pay or the order of our results. Prices come from the stores and can change, so check the price at the store before you buy.