Release Name: 0.1.0
Notes:
A Ruby web-spider framework for easily developing programs to crawl a domain and gather useful statistics.
Changes:
* Support for performing a focused crawl, by selecting which links to follow on each page
* Page stores HTTP response headers
* Ability to specify multiple starting URLs
|
