White Rose University Consortium logo
University of Leeds logo University of Sheffield logo York University logo

A two-phase sampling technique for information retrieval from hidden web databases

Hedley, Y., Younas, M., James, A. and Sanderson, M. (2004) A two-phase sampling technique for information retrieval from hidden web databases. In: Web Information Systems – WISE 2004 : 5th International Conference on Web Information Systems Engineering, Brisbane, Australia, November 22-24, 2004. Proceedings. Lecture Notes in Computer Science, System (3306). Springer , Berlin / Heidelberg . ISBN 978-3-540-23894-2

Full text not available from this repository.


The larger amount of high quality and specialised information on the Web is stored in document databases, which is not indexed by general-purpose search engines such as Google and Yahoo. Such information is dynamically generated as a result of submitting queries to databases — which are referred to as Hidden Web databases. This paper presents a Two-Phase Sampling (2PS) technique that detects Web page templates from the randomly sampled documents of a database. It generates terms and frequencies that summarise the database content with improved accuracy. We then utilise such statistics to improve the accuracy of text similarity computation in categorisation. Experimental results show that 2PS effectively eliminates terms contained in Web page templates, and generates terms and frequencies with improved accuracy. We also demonstrate that 2PS improves the accuracy of text similarity computation required in the process of database categorisation.

Item Type: Book Section
Institution: The University of Sheffield
Academic Units: The University of Sheffield > Faculty of Social Sciences (Sheffield) > Information School (Sheffield)
Depositing User: Repository Officer
Date Deposited: 22 Sep 2008 17:51
Last Modified: 22 Sep 2008 17:51
Published Version: http://dx.doi.org/10.1007/b103344
Status: Published
Publisher: Springer
Refereed: Yes
Identification Number: 10.1007/b103344
URI: http://eprints.whiterose.ac.uk/id/eprint/4543

Actions (repository staff only: login required)