Posts Tagged ‘robots.txt’
Using robots.txt file to block search engines.
All of us want our new websites to be indexed by search engines as soon as possible. But there are instances when we do not want our web pages to be indexed by search engines. For example, if you are developing a new section to your website and the web pages are still under construction you do not [...]
