crawler2
所属分类：Java编程
开发工具：Java
文件大小：573KB
下载次数：10
上传日期：2009-02-12 05:13:33
上 传 者：
canhelp2001
说明： 利用广度优先遍历搜索一定范围内的所有网页,可用于建立搜索引擎和查找网络错误.
(Our webcrawler will traversal a certain range of website from a given source URL by Breadth First Search)
文件列表:
crawler (0, 2006-11-01)
crawler\.classpath (354, 2006-10-23)
crawler\.project (383, 2006-10-20)
crawler\bin (0, 2006-10-27)
crawler\bin\crawler.properties (836, 2006-11-01)
crawler\bin\db (0, 2006-10-26)
crawler\bin\db\PsqlPool.class (5585, 2006-10-27)
crawler\bin\html (0, 2006-10-23)
crawler\bin\html\AnchorElement.class (1045, 2006-10-31)
crawler\bin\html\AttributeList.class (5553, 2006-10-31)
crawler\bin\html\Element.class (1714, 2006-10-31)
crawler\bin\html\HtmlPage.class (6383, 2006-10-27)
crawler\bin\io (0, 2006-10-25)
crawler\bin\io\MyPrintStream.class (529, 2006-10-25)
crawler\bin\log (0, 2006-10-25)
crawler\bin\log\LogItem.class (1653, 2006-10-27)
crawler\bin\log\PgDBLogger.class (3902, 2006-11-01)
crawler\bin\test (0, 2006-10-27)
crawler\bin\WebCrawler (0, 2006-10-27)
crawler\bin\WebCrawler\Crawler.class (7761, 2006-11-01)
crawler\bin\WebCrawler\Link.class (5231, 2006-10-27)
crawler\bin\WebCrawler\LinkProducerConsumer.class (2800, 2006-10-27)
crawler\crawler.jar (33363, 2006-11-01)
crawler\crawler.properties (836, 2006-11-01)
crawler\dbsetup.sql (1742, 2006-10-23)
crawler\lib (0, 2006-10-23)
crawler\lib\postgresql-8.1-405.jdbc2.jar (363688, 2006-05-10)
crawler\lib\Tidy.jar (177868, 2001-08-01)
crawler\run.bat (125, 2006-10-25)
crawler\src (0, 2006-10-27)
crawler\src\crawler.properties (119, 2006-10-23)
crawler\src\db (0, 2006-10-26)
crawler\src\db\PsqlPool.java (6360, 2006-10-27)
crawler\src\html (0, 2006-10-20)
crawler\src\html\AnchorElement.java (600, 2006-10-31)
crawler\src\html\AttributeList.java (9149, 2006-10-31)
crawler\src\html\Element.java (1369, 2006-10-31)
crawler\src\html\HtmlPage.java (6995, 2006-10-27)
crawler\src\io (0, 2006-10-25)
... ...
Prerequisite:
1. PostgreSQL is installed. Make a note of port number, user name, and password during installation.
2. Set up the crawler database using the psql tool and dbsetup.sql file.
psql -U postgres template 1 -f ./dbsetup.sql
3. Update the parameters in crawler.properties as you want.
4. Execute the program using run.bat.
run -p
-o
近期下载者：
相关文件：
收藏者：