Winkler, J.R. orcid.org/0000-0002-4629-8928 and Mitrouli, M. (2025) Analysis and implications of a negative parameter in Tikhonov regularisation. Communications in Statistics - Theory and Methods. ISSN: 0361-0926
Abstract
The application of Tikhonov regularisation to the least squares (LS) problem arises frequently in machine learning, for example, in regression and the calculation of the excess risk (out-of-sample prediction error) from a given set of noisy observations. It requires the minimisation with respect to x of a function f(x, λ), where λ is the regularisation parameter. If λ≥0, there exists an optimal value λopt of λ such that the vector x(λopt) that minimises f(x, λ) is numerically stable and its error with respect to x(0) is small. It has been claimed that λopt may be negative, and the aim of this article is the analysis of the consequences of this condition. It is shown theoretically that the condition λ < 0 yields a family of solutions x(λ), each of whose members has a large error and is unstable. Furthermore, the L-curve, which is a method for the calculation of the value of λopt, yields a good result for λ≥0, and it also shows that λ < 0 yields unsatisfactory solutions. The L-curve implies, therefore, that λopt≥0, which is in accord with the theoretical analysis. Examples of LS problems that consider λ < 0 and λ≥0 are shown, and the unsatisfactory results for λ < 0 are evident.
Metadata
Item Type: | Article |
---|---|
Authors/Creators: |
|
Copyright, Publisher and Additional Information: | © 2025 The Author(s). Published with license by Taylor & Francis Group, LLC. This is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives License (http://creativecommons.org/licenses/by-nc-nd/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is properly cited, and is not altered, transformed, or built upon in any way. The terms on which this article has been published allow the posting of the Accepted Manuscript in a repository by the author(s) or with their consent. |
Keywords: | Least squares; regularisation; negative regularisation parameter; condition estimation; regularisation error; L-curve |
Dates: |
|
Institution: | The University of Sheffield |
Academic Units: | The University of Sheffield > Faculty of Engineering (Sheffield) > Department of Computer Science (Sheffield) |
Date Deposited: | 22 Jul 2025 14:22 |
Last Modified: | 01 Oct 2025 15:04 |
Status: | Published online |
Publisher: | Taylor and Francis Group |
Refereed: | Yes |
Identification Number: | 10.1080/03610926.2025.2540866 |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:229233 |
Download
Filename: Analysis_and_implications_of_a_negative_parameter_in_Tikhonov_regularisation.pdf
Licence: CC-BY-NC-ND 4.0