The SearchEngineKeywordsPerformance plugin offers new Crawling overview reports:
Crawling overview report show how Search engines bots crawl your websites. This crawling overview displays the following key metrics: crawled pages, total pages in index, total inboud links, robots.txt exclusion page count, page redirects (301, 302 http status).
Crawling errors report which shows any error encountered by Search bots: crawl errors, DNS failures, connection timeouts, error pages (4xx http status), internet error pages (5xx http status).
These metrics are regularly as often as possible according to how fast Search Engines provide us with the data:
Google updates and provides the crawling overview metrics frequently, but only provides the most recent KPI values. There is no possibility to get historical stats. Nevertheless Piwik imports them each day and saves them, allowing you to keep full control over your historical data.
Bing updates the crawling overview metrics daily and also provides historical data. All available data will be imported on the first import. New data will then be imported every day in your Piwik.