http: Implement robots.txt
To prevent search engines from being reported and blocked, implement and serve /robots.txt via get that will prevent valid search engines from trying to reach resources on fake http server.
See robotstxt.org
Edited by Vojtech Myslivec