Abstract
The aim of this paper is twofold. In the first part, we recapitulate the main results regarding the shrinkage properties of partial least squares (PLS) regression. In particular, we give an alternative proof of the shape of the PLS shrinkage factors. It is well known that some of the factors are >1. We discuss in detail the effect of shrinkage factors for the mean squared error of linear estimators and argue that we cannot extend the results to PLS directly, as it is nonlinear. In the second part, we investigate the effect of shrinkage factors empirically. In particular, we point out that experiments on simulated and real world data show that bounding the absolute value of the PLS shrinkage factors by 1 seems to leads to a lower mean squared error.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Butler N, Denham M (2000) The peculiar shrinkage properties of partial least squares regression. J R Stat Soc Ser B 62(3):585–593
de Jong S (1995) PLS shrinks. J Chemom 9:323–326
Frank I, Friedman J (1993) A statistical view of some chemometrics regression tools. Technometrics 35:109–135
Helland I (1988) On the structure of partial least squares regression. Commun Stat Simul Comput 17(2):581–607
Höskuldsson A (1988) PLS regression methods. J Chemom 2:211–228
Lanczos C (1950) An iteration method for the solution of the eigenvalue problem of linear differential and integral operators. J Res Natl Bur Stand 45:225–280
Lingjaerde O, Christopherson N (2000) Shrinkage structures of partial least squares. Scand J Stat 27:459–473
Parlett B (1998) The symmetric eigenvalue problem. Soc Ind Appl Math
Phatak A, de Hoog F (2002) Exploiting the connection between PLS, Lanczos, and conjugate gradients: alternative proofs of some properties of PLS. J Chemom 16:361–367
Rosipal R, Krämer N (2006) Overview and recent advances in partial least squares. In: Subspace, latent structure and feature selection techniques. Lecture Notes in Computer Science, Springer, Heidelberg, pp 34–51
Wold H (1975) Path models with latent variables: the NIPALS approach. In: HB et al. (ed) Quantitative sociology: international perspectives on mathematical and statistical model building. Academic, New York, pp 307–357
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Krämer, N. An overview on the shrinkage properties of partial least squares regression. Computational Statistics 22, 249–273 (2007). https://doi.org/10.1007/s00180-007-0038-z
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00180-007-0038-z