META Name Robots
To make spider of a search engine to index only the page you want to index and don’t want it to follow the links in the page, you can do that with the description so called ROBOTS tag.
The robots Meta tag is not the same as the file called robots.txt file that you use to control the reach of the spider. You should use these two together. Both are used by the search engines like Yahoo and Google. If you use this Meta tag the wrong way you might shut the search engines out. So the influence of this Meta tag is significant.
Examples of Robots TAG
Add the following meta tag in the HTML source of your page:
- The spider will now index your whole website.
- The spider will not only index the first webpage of your website but also all your other webpages.
you are also allowed to type it like this:
By changing the index to no-index and the follow to no-follow your are able to influence the behaviour of the spider. If you don’t want the search engine spider to crawl through your whole website you use the following meta tag :
The spider will now only look at this page and stops there.
The spider will not look at this page but will crawl through the rest of the pages on your website.
The spider will not look at this page and will NOT crawl through the rest of your webpages.
Description
You add this robot tag on the first index page and you tell the spider if you want your whole website to be crawled or not.
Make sure that on every page relevant meta tags are added. Add keywords and phrases that are relevant and correspond to the text and the language on that specific page. It might be a lot of work to add specific meta tags to each page but you will notice in time that it works!
