[Xfce-i18n] [Site] PO directories are indexed by Google

Nick Schermer nickschermer at gmail.com
Mon Feb 22 12:05:35 CET 2010


On Mon, Feb 22, 2010 at 9:45 AM, Алекс <eskaer_spamsink at ngs.ru> wrote:
> Is it possible to change the site so search engines do not crawl inside po directories? They contain E-Mail addresses which are indexed and possibly harvested.

Add a robots.txt to transifex:
https://translations.xfce.org/robots.txt. But this will not prevent
spam bots from crawling the pages.

Nick



More information about the Xfce-i18n mailing list