Crawlers (or bots) are used to collect data out there on the web. By using web site navigation menus, and studying inner and external hyperlinks, the bots begin to understand the context of a page. Of course, the words, photographs, and different knowledge on pages additionally assist search engines