mercredi 21 octobre 2015

Can I put robots.txt for "sorry" server?

I'm considering whether I should put following robots.txt for my "sorry server" that returns some sorry message to our customer that we are under maintenance.

User-agent: *
Disallow: /

So here's my concerns/questions:

  1. Won't it tell crawlers to not to index our site forever in spite of our server is ready after the maintenance is done?

  2. If I put the robots.txt for my sorry server, should I put another robots.txt for our regular server that tells crawlers to "please index our site"?




Aucun commentaire:

Enregistrer un commentaire