A New Framework for Focused Web Crawling

来源 :Wuhan University Journal of Natural Sciences | 被引量 : 0次 | 上传用户:zjlong8888
下载到本地 , 更方便阅读
声明 : 本文档内容版权归属内容提供方 , 如果您对本文有版权争议 , 可与客服联系进行内容授权或下架
论文部分内容阅读
Focused crawlers are important tools to support applications such as specialized Web portals, online searching, and Web search engines. A topic driven crawler chooses the best URLs and relevant pages to pursue during Web crawling. It is difficult to deal with irrelevant pages. This paper presents a novel focused crawler framework. In our focused crawler, we propose a method to overcome some of the limitations of dealing with the irrelevant pages. We also introduce the implementation of our focused crawler and present some important metrics and an evaluation function for ranking pages relevance. The experimental result shows that our crawler can obtain more “important” pages and has a high precision and recall value. Focused crawlers are important tools to support applications such as specialized Web portals, online searching, and Web search engines. A topic driven crawler chooses the best URLs and relevant pages to pursue not Web crawling. It is difficult to deal with irrelevant pages. This paper presents a novel focused crawler framework. In our focused crawler, we propose a method to overcome some of the limitations of dealing with the irrelevant pages. We also introduce the implementation of our focused crawler and present some important metrics and an evaluation function for rating pages relevance. The experimental result shows that our crawler can obtain more “important” pages and has a high precision and recall value.
其他文献
【内容摘要】语文学习的特殊之处,在于学生的课外学习对学习效果的影响非常巨大,懂得自主学习的学生不满足于有限的课本知识,会按照自己的意愿去学习更多的相关内容。长期的积累让这部分学生在语文学习上遥遥领先,所以初中语文教师要鼓励学生自主学习,激发他们的学习兴趣。  【关键词】初中语文 自主学习 兴趣培养  众所周知,现在中小学生的学习任务很繁重,学生都存在不同程度的厌学情绪,很少有学生会在课后自主学习。
INTRODUCTION In the absence of other intracranial tuberculomas or systemic manifestation of tuberculosis isolated brainstem tuberculoma is a rare entity,that ac
“孩子不可能不犯错。如果孩子犯了错,那也是我的错。”如果家长有了这样的认识,恐怕结果就大不一样。故事一:警告失灵儿子5岁,非常调皮。他最近狂爱玩枪,他会随身携带着他那
自 198 9年Choo发现了丙型肝炎病毒 (HCV)以来 ,对丙型肝炎的认识有了逐步深入的了解。全球HCV流行率平均约为 3% (0 1% - 5 % ) [1] ,我国流行率平均为 3 2 %。男性高于女
如果做个有心人,将一年中的大小节日、纪念日统计一下,我们会发现,特别的日子真多!我们常常在不平凡的日子中度过,然而除了在电视、报刊中了解到一些与之相关的信息外,我们的生活却并不因这些日子的到来而格外鲜活生动。相反,好多日子已离我们愈来愈远,我们的情感不会因它而起半点波澜。比如:3月5日。在毫无准备的情况下,突然问别人:“这是什么日子?”多半会有人莫名其妙地睁大双眼。也许说到2月14日这个并不属于我
为推动现代农业产业基地建设,培养和造就一大批农业产业大户和现代农民企业家,2010年四川省决定在40个县实施农民创业培训。10月9日,四川省级农民创业培训示范班开班仪式在省
Communicating Sequential Processes (CSP) is a kind of process algebra, which is suitable for modeling and verifying Web service composition. This paper describe
In this paper, we propose a flexible location-based service (LBS) middleware framework to make the development and deployment of new location based applications
为了解 2 0 0 1年冬季北京市城区居民流行性感冒 (流感 )样病例发病及流感疫苗接种情况 ,使用北京市 8个城区的电话局向号和随机数字相结合产生的电话号码 ,对城区居民进行电
绥德学用半月刊收效明显陕西绥德县工商局连续3年来把学习《工商行政管理》当作一件大事来抓,每周三、周六的学习制度一直坚持至今。通过学习,使系统上下开阔了视野,业务水平大为