[ZendTo] Re: allow google to index the list?

[Julian Field]

I've changed the file to read

User-agent: *
Allow: /pipermail/zendto/
Disallow: /

User-agent: ECSbot
Allow: /

which I think should allow everyone to index the ZendTo list archive but 
nothing else, which is what we want.

Agree?

Cheers,
Jules.

On 04/10/2012 07:34, Brendon Baumgartner wrote:
> I think this should do it. Also, here are the docs from google.
> https://developers.google.com/webmasters/control-crawl-index/docs/robots_txt
>
>
> User-agent: googlebot
> Allow: /
>
> User-agent: ECSbot
> Allow: /
>
> User-agent: *
> Disallow: /
>
>
>
>
> On Thu, Sep 27, 2012 at 5:04 PM, Brendon Baumgartner <b at brendon.com 
> <mailto:b at brendon.com>> wrote:
>
>     Can you please allow google to index the list. I see you have it
>     disabled here:
>
>     http://mailman.ecs.soton.ac.uk/robots.txt
>
>     It would be nice to actually search the mailing list for past
>     problems/solutions.
>     Thanks
>     -Brendon
>
>
>
>
> _______________________________________________
> ZendTo mailing list
> ZendTo at zend.to
> http://mailman.ecs.soton.ac.uk/mailman/listinfo/zendto

Jules

-- 
Jules Field MEng MBCS CITP CEng
email+iMessage: Jules at ecs.soton.ac.uk
Twitter: @JulesFM

Senior Tutor, Postmaster
Electronics and Computer Science
University of Southampton SO17 1BJ, UK

'That is the land of lost content,
  I see it shining plain,
  The happy highways where I went,
  And cannot come again.' - A.E. Houseman

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ecs.soton.ac.uk/pipermail/zendto/attachments/20121004/e773d983/attachment.html