📆Date created: 04/02/2015
There should be a unified way for subs to configure what kind of bots they want and don't want. Good bots should respect the wishes of the community.
⚡ Found 1 subreddits like r/robotstxt
Sorted by: Best match
TOP CONTENT FROM R/ROBOTSTXT
💬 See what people in r/robotstxt are talking about