The User-Agent
PatineCompsBot/0.1 (+https://getpatine.com/bot; comparable-sales index; contact info@getpatine.com)What it is for
Patine is an iPhone app that identifies antiques and collectibles from photographs. The rule the whole product is built on is that we never show somebody a value without showing where it came from — which means a price is only ever displayed next to a real past sale, with the name of the house that sold it, the date, and a link back to your page.
The crawler exists to build that index of past results. Every entry it creates carries your name and a link to your lot. It is not a way of republishing your catalogue; it is the reason our users are sent to it.
What it fetches
Past-sale lot pages, and the sale and index pages needed to find them. From a lot page it keeps the title, the realised price, the sale date, your name as the source, the URL, and where one exists a thumbnail. Nothing else.
It does not fetch images at full size, does not crawl anything behind a login, does not submit forms, and does not touch current or live sales.
How it behaves
- It reads
robots.txtand does what it says. A disallowed path is a disallowed path, including when the file it would produce is exactly the file we want. One house’s results page references an endpoint that would return an entire sale’s prices in a single request; that endpoint sits under a disallowed prefix, and we do not use it. - It honours
crawl-delay, and where none is given it is slower than one request every few seconds. - A bot challenge is an answer, not an obstacle. One house we would have liked to include serves its auction archive behind a managed challenge. We treat that as a refusal and we do not route around it — that house is simply not in the index, and the spec that names it as an example has been amended to say why.
- It identifies itself honestly on every request, with a contact address that a person reads.
- It runs occasionally, in small passes. It is not a continuous crawl and it will not appear in your logs as sustained traffic.
How to stop it
Add this to your robots.txt and it will stop, on its next pass, without argument:
User-agent: PatineCompsBot
Disallow: /If you would rather it stopped now, or stopped only on certain paths, or would rather send us a feed than be crawled at all — that last one is genuinely our preference — write to info@getpatine.com and a person will answer.
If you think something is wrong
If an entry misattributes a lot, quotes a price incorrectly, or links somewhere it should not, tell us and we will correct or remove it. We would rather have a smaller index that is right.