Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

yup. Anchor text is fair game. Keywords taken from anchor texts are usually a good indication on what the linked page is about.

A page can have a high page rank without ever being crawled - given that other high ranked pages have linked to it with good keywords in the anchor text. Such pages can often be identified by the lack of snippet texts.

Edit: I may have misunderstood you, you can only use the data found in links, not follow them.



If a page is not crawlable, but listed in the search results based on anchor text (or the URL), then it will not have any snippets. You can show snippets only if you crawl the page itself.




Consider applying for YC's Fall 2026 batch! Applications are open till July 27.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: