They proved that Microsoft uses clickstream data to rank websites, and in 7% of manufactured cases that's all the data they have?
I think this is exactly as petty and silly as last week's news, and now they get to spend a week explaining how this occurred and that they do obey robots.txt.
Everyone else in this thread has already explained that microsoft is not properly using robots.txt by having case insensitive urls, hence why these urls were indexed.
I think this is exactly as petty and silly as last week's news, and now they get to spend a week explaining how this occurred and that they do obey robots.txt.