Distributed Web Crawling

Distributed Web Crawling

List price: US$51.99

Currently unavailable

Add to wishlist

AbeBooks may have this title (opens in new window).

Try AbeBooks

Description

Please note that the content of this book primarily consists of articles available from Wikipedia or other free sources online. Distributed web crawling is a distributed computing technique whereby Internet search engines employ many computers to index the Internet via web crawling. The idea is to spread out the required resources of computation and bandwidth to many computers and networks. With this type of policy, a central server assigns new URLs to different crawlers dynamically. This allows the central server to, for instance, dynamically balance the load of each crawler. With dynamic assignment, typically the systems can also add or remove downloader processes. The central server may become the bottleneck, so most of the workload must be transferred to the distributed crawling processes for large crawls.show more

Product details

  • Paperback | 116 pages
  • 152 x 229 x 7mm | 181g
  • DIC Press
  • United States
  • English
  • 6135863371
  • 9786135863376