Hedley, Y., Younas, M., James, A. and Sanderson, M. (2004) A two-phase sampling technique for information retrieval from hidden web databases. In: Web Information Systems – WISE 2004 : 5th International Conference on Web Information Systems Engineering, Brisbane, Australia, November 22-24, 2004. Proceedings. Lecture Notes in Computer Science, System (3306). Springer , Berlin / Heidelberg . ISBN 978-3-540-23894-2
The larger amount of high quality and specialised information on the Web is stored in document databases, which is not indexed by general-purpose search engines such as Google and Yahoo. Such information is dynamically generated as a result of submitting queries to databases — which are referred to as Hidden Web databases. This paper presents a Two-Phase Sampling (2PS) technique that detects Web page templates from the randomly sampled documents of a database. It generates terms and frequencies that summarise the database content with improved accuracy. We then utilise such statistics to improve the accuracy of text similarity computation in categorisation. Experimental results show that 2PS effectively eliminates terms contained in Web page templates, and generates terms and frequencies with improved accuracy. We also demonstrate that 2PS improves the accuracy of text similarity computation required in the process of database categorisation.
|Institution:||The University of Sheffield|
|Academic Units:||The University of Sheffield > Faculty of Social Sciences (Sheffield) > Information School (Sheffield)|
|Depositing User:||Repository Officer|
|Date Deposited:||22 Sep 2008 17:51|
|Last Modified:||22 Sep 2008 17:51|