As you my already aware that Search Engine Optimization (SEO) is never ending working for blogger. There are always something that we can – and should – optimize so our blog have more exposure in search engine. One of important part for SEO is to tell search engine what to do with our blog content when crawl our blog. Yeah – surely search engine still know what to do with our blog even we do not tell how but of course, The result will be not optimum. We can optimizing the result by tell search engine what exactly to do with our blog. Which one it is allowed, and which one is not. Having robots.txt file in our blog is the solutions for it.
How Does Robots.txt Work?
We can tell search engine clearly how to treat our blog content by using robots.txt file. Which content search engine should be indexed and which content that should be not indexed. In short word, robots.txt tell search engine robot what allowed and what not allowed. Robots.txt command quite simple. There are only three comment that we use to optimize it.