Today Google added a new feature to their Webmaster Tools, allowing us to see the cumulative number of pages indexed over the past year. This feature also has an advanced section which shows not only totals of indexed pages, but also the cumulative number of pages crawled, the number of pages that Google knows about which are not crawled because they are blocked by robots.txt, and also the number of pages that were not selected for inclusion in their results.
This new section is located under the “Health” section of Webmaster Tools, and is called “Index Status.”
All this data can be used to identify and debug a variety of indexing-related problems. For example, if some of your content doesn’t appear any more on Google and you notice that the graph of pages indexed has a sudden drop, that may be an indication that you introduced a site-wide error when using meta=”noindex” and now Google isn’t including our content in the search results. The “not selected” part of this graph can also indicate URL’s from your site that redirect to other pages or URL’s whose contents are substantially similar to other pages. (Duplicate Content)
Overall, this is an important tool to utilize on a regular basis to monitor Google’s indexing of your websites. Watching the trends will help you to determine what changes are having a positive effect with Google and which ones are not. It should be a matter of routine to monitor this graph across for all your websites.