So a few days ago I was idly looking at the “StatPress” page on my site to look how few people were visiting to see something rather surprising :-
What was happening here ? Have I been slashdotted ? Is my income from those silly ads down the side going to shoot through the roof to a level worth letting a certain company send me the payments ?
No. Nothing so exciting. All (well, the overwhelming majority) were from an MSN robot – presumably indexing this site. Well fair enough, but why so many accesses ? It is not as if my site has much on it – nowhere near the nearly 100,000 page fetches they tried. A closer look at the Apache logs shows that the msn robot repeatedly fetched many pages including one page 1,300 times!
Sounds like Microsoft has a bug somewhere.