Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

They proved that Microsoft uses clickstream data to rank websites, and in 7% of manufactured cases that's all the data they have?

I think this is exactly as petty and silly as last week's news, and now they get to spend a week explaining how this occurred and that they do obey robots.txt.



Everyone else in this thread has already explained that microsoft is not properly using robots.txt by having case insensitive urls, hence why these urls were indexed.


"Be strict in what you emit, liberal in what you accept."

Bing failed the first part, Google the second. Both should fix that.


The Bing context might suck for you guys, but this is your problem - case insensitivity is every Windows server, not just the Bing website.

Why would anyone hosted on Windows have to specify every possible spelling variation to keep search engines out of a folder or file?

Here's another example:

http://www.ifma.org/robots.txt

These guys are disallowing /pv/

http://www.google.com/search?sourceid=chrome&ie=UTF-8...

You guys are indexing /PV/


Maybe I'm misunderstanding your argument, but I think you're confusing the Windows file system with the URLs that a web service provides.


The problem exists when that web service is on Windows - ASP.NET, Cold Fusion, static html sites, probably a negligible percent of PHP sites etc -

/pv is /PV is /pV is /Pv


Except that this time it isn't Microsoft themselves crying foul.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: