InfatiumBot
The crawler that reads open sources for users' feeds. Here is how to recognise it, what it reads and how to keep it out.
What it is called
In robots.txt the crawler answers to the name InfatiumBot. The full User-Agent string:
InfatiumBot/1.0 (+https://infatium.ru/bot)What it reads
Only the sources users have added to their own feeds:
- RSS and Atom feeds a site publishes for its readers
- site pages — the list of articles and the articles themselves
- public Telegram channels — through Telegram, not over HTTP, so robots.txt does not apply to them
How to opt out
Disallow the crawler a site or a section in robots.txt. A rule for InfatiumBot takes precedence over a rule for “*”:
User-agent: InfatiumBot
Disallow: /A single page is closed with meta tags. noindex — the page is not saved; nosnippet and noai — the text is not saved, the title and the link remain; noimageai — images are not saved:
<meta name="robots" content="noindex">
<meta name="infatiumbot" content="noai, noimageai">The same can be sent in the X-Robots-Tag response header, also addressed to us: “X-Robots-Tag: infatiumbot: noindex”.
In Telegram, turn on “Restrict saving content” in the channel settings — such a channel cannot be added as a source, and one already added stops being read. The same flag on a single message excludes only that message.
A message deleted in the channel is deleted on our side too.
Where to write
If the crawler gets in the way of your site, or you want a source excluded entirely, write to support@infatium.ru or leave a request in the form on the page “Copyright”.