U md @sdZddlmZmZddlZddlmZmZddlZ ddl m Z ddl m Z mZmZddl mZdd l mZdd lmZmZdd lmZdd lmZdd lmZddlmZmZddlmZmZddl m!Z!dddgZ"eedkrddl m#Z$n ddl m$Z$ddZ%d.ddZ&dd Z'd/d"d#Z(d$d%Z)Gd&d'd'eeeee ed(Z*Gd)dde*Z+Gd*dde*Z,Gd+d,d,e*Z-Gd-ddeee Z.dS)0zG The :mod:`sklearn.pls` module implements Partial Least Squares (PLS). )IntegralRealN)ABCMetaabstractmethod)svd) BaseEstimatorRegressorMixinTransformerMixin)MultiOutputMixin)ClassNamePrefixFeaturesOutMixin) check_arraycheck_consistent_length) sp_version) parse_version)svd_flip)check_is_fitted FLOAT_DTYPES)Interval StrOptions)ConvergenceWarning PLSCanonical PLSRegressionPLSSVDz1.7)pinv)pinv2c Cst|ddd\}}}|jj}ddd}t|||t|j}t||k}|ddd|f}||d|}t t t ||d|S)NF) full_matrices check_finiteg@@g.A)fd) rdtypecharlowernpmaxfinfoepssumZ transpose conjugatedot)ausZvhtfactorZcondZrankr/Y/home/sam/Atlas/atlas_env/lib/python3.8/site-packages/sklearn/cross_decomposition/_pls.py _pinv2_old&s  r1Aư>Fc st|jjztfdd|jD}Wn,tk rV}ztd|W5d}~XYnXd}|dkrvt|t|} } t|D]} |dkrt | |} nt |j|t ||} | t t | | } t || } |dkrt | | }nt |j| t | j| }|r*|t t ||}t ||t ||}| |}t |||ksp|j ddkrvq|| }q~| d}||krt dt| ||fS) a?Return the first left and right singular vectors of X'Y. Provides an alternative to the svd(X'Y) and uses the power method instead. With norm_y_weights to True and in mode A, this corresponds to the algorithm section 11.3 of the Wegelin's review, except this starts at the "update saliences" part. c3s&|]}tt|kr|VqdSN)r#anyabs).0colr&r/r0 Esz;_get_first_singular_vectors_power_method..Y residual is constantNdBz$Maximum number of iterations reached)r#r%r r&nextT StopIterationr1ranger)sqrtshapewarningswarnr)XYmodemax_itertolnorm_y_weightsZy_scoreeZ x_weights_oldZX_pinvZY_pinvi x_weightsZx_score y_weightsZx_weights_diffZn_iterr/r:r0(_get_first_singular_vectors_power_method8s8   "  rRcCs@t|j|}t|dd\}}}|dddf|dddffS)zbReturn the first left and right singular vectors of X'Y. Here the whole SVD is computed. FrNr)r#r)rAr)rHrICU_Vtr/r/r0_get_first_singular_vectors_svdssrXTcCs|jdd}||8}|jdd}||8}|rr|jddd}d||dk<||}|jddd}d||dk<||}n t|jd}t|jd}||||||fS)z{Center X, Y and scale if the scale parameter==True Returns ------- X, Y, x_mean, y_mean, x_std, y_std raxisr?)rZZddofg?)ZmeanZstdr#ZonesrE)rHrIscaleZx_meanZy_meanZx_stdZy_stdr/r/r0_center_scale_xy}s     r]cCs2tt|}t||}||9}||9}dS)z7Same as svd_flip but works on 1d arrays, and is inplaceN)r#Zargmaxr7sign)r+vZbiggest_abs_val_idxr^r/r/r0 _svd_flip_1dsr`c @seZdZUdZeeddddgdgeddhged d hged d hgeeddddgeed dddgdgdZe e d<e d%ddd d ddddddZ ddZ d&ddZd'ddZd(ddZd)dd Zed!d"Zd#d$ZdS)*_PLSaPartial Least Squares (PLS) This class implements the generic PLS algorithm. Main ref: Wegelin, a survey of Partial Least Squares (PLS) methods, with emphasis on the two-block case https://stat.uw.edu/sites/default/files/files/reports/2000/tr371.pdf r?Nleftclosedboolean regression canonicalr2r>rnipalsr n_componentsr\deflation_moderJ algorithmrKrLcopy_parameter_constraintsrTr3r4)r\rkrJrlrKrLrmc Cs4||_||_||_||_||_||_||_||_dSr5)rjrkrJr\rlrKrLrm) selfrjr\rkrJrlrKrLrmr/r/r0__init__s z _PLS.__init__c Cs|t|||j|tj|jdd}t|dtj|jdd}|jdkrT|dd}|j d}|j d}|j d}|j }|j d kr|n t |||}||krt d |d |d |j d k|_|j}t|||j\} } |_|_|_|_t||f|_t||f|_t||f|_t||f|_t||f|_t||f|_g|_t| jj} t |D]} |j!dkr"tj"t#| d| kdd} d| dd| f<z$t$| | |j%|j&|j'|d\}}}WnPt(k r}z0t)|dkrt*+d| WY qzW5d}~XYnX|j,|n|j!dkr`. .. versionadded:: 0.8 Parameters ---------- n_components : int, default=2 Number of components to keep. Should be in `[1, min(n_samples, n_features, n_targets)]`. scale : bool, default=True Whether to scale `X` and `Y`. max_iter : int, default=500 The maximum number of iterations of the power method when `algorithm='nipals'`. Ignored otherwise. tol : float, default=1e-06 The tolerance used as convergence criteria in the power method: the algorithm stops whenever the squared norm of `u_i - u_{i-1}` is less than `tol`, where `u` corresponds to the left singular vector. copy : bool, default=True Whether to copy `X` and `Y` in :term:`fit` before applying centering, and potentially scaling. If `False`, these operations will be done inplace, modifying both arrays. Attributes ---------- x_weights_ : ndarray of shape (n_features, n_components) The left singular vectors of the cross-covariance matrices of each iteration. y_weights_ : ndarray of shape (n_targets, n_components) The right singular vectors of the cross-covariance matrices of each iteration. x_loadings_ : ndarray of shape (n_features, n_components) The loadings of `X`. y_loadings_ : ndarray of shape (n_targets, n_components) The loadings of `Y`. x_scores_ : ndarray of shape (n_samples, n_components) The transformed training samples. y_scores_ : ndarray of shape (n_samples, n_components) The transformed training targets. x_rotations_ : ndarray of shape (n_features, n_components) The projection matrix used to transform `X`. y_rotations_ : ndarray of shape (n_features, n_components) The projection matrix used to transform `Y`. coef_ : ndarray of shape (n_features, n_targets) The coefficients of the linear model such that `Y` is approximated as `Y = X @ coef_ + intercept_`. intercept_ : ndarray of shape (n_targets,) The intercepts of the linear model such that `Y` is approximated as `Y = X @ coef_ + intercept_`. .. versionadded:: 1.1 n_iter_ : list of shape (n_components,) Number of iterations of the power method, for each component. n_features_in_ : int Number of features seen during :term:`fit`. feature_names_in_ : ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0 See Also -------- PLSCanonical : Partial Least Squares transformer and regressor. Examples -------- >>> from sklearn.cross_decomposition import PLSRegression >>> X = [[0., 0., 1.], [1.,0.,0.], [2.,2.,2.], [2.,5.,4.]] >>> Y = [[0.1, -0.2], [0.9, 1.1], [6.2, 5.9], [11.9, 12.3]] >>> pls2 = PLSRegression(n_components=2) >>> pls2.fit(X, Y) PLSRegression() >>> Y_pred = pls2.predict(X) rnrkrJrlrTr3r4r\rKrLrmc s tj||ddd|||ddS)Nrfr2rhrisuperrprorjr\rKrLrm __class__r/r0rptszPLSRegression.__init__cs"t|||j|_|j|_|S)rq)rrrZ x_scores_rZ y_scores_)rorHrIrr/r0rszPLSRegression.fit)r) rrrrrarnrrparampoprpr __classcell__r/r/rr0rs b csVeZdZUdZejZeed<dD]Ze eq"d dddddd fd d Z Z S) ra Partial Least Squares transformer and regressor. Read more in the :ref:`User Guide `. .. versionadded:: 0.8 Parameters ---------- n_components : int, default=2 Number of components to keep. Should be in `[1, min(n_samples, n_features, n_targets)]`. scale : bool, default=True Whether to scale `X` and `Y`. algorithm : {'nipals', 'svd'}, default='nipals' The algorithm used to estimate the first singular vectors of the cross-covariance matrix. 'nipals' uses the power method while 'svd' will compute the whole SVD. max_iter : int, default=500 The maximum number of iterations of the power method when `algorithm='nipals'`. Ignored otherwise. tol : float, default=1e-06 The tolerance used as convergence criteria in the power method: the algorithm stops whenever the squared norm of `u_i - u_{i-1}` is less than `tol`, where `u` corresponds to the left singular vector. copy : bool, default=True Whether to copy `X` and `Y` in fit before applying centering, and potentially scaling. If False, these operations will be done inplace, modifying both arrays. Attributes ---------- x_weights_ : ndarray of shape (n_features, n_components) The left singular vectors of the cross-covariance matrices of each iteration. y_weights_ : ndarray of shape (n_targets, n_components) The right singular vectors of the cross-covariance matrices of each iteration. x_loadings_ : ndarray of shape (n_features, n_components) The loadings of `X`. y_loadings_ : ndarray of shape (n_targets, n_components) The loadings of `Y`. x_rotations_ : ndarray of shape (n_features, n_components) The projection matrix used to transform `X`. y_rotations_ : ndarray of shape (n_features, n_components) The projection matrix used to transform `Y`. coef_ : ndarray of shape (n_features, n_targets) The coefficients of the linear model such that `Y` is approximated as `Y = X @ coef_ + intercept_`. intercept_ : ndarray of shape (n_targets,) The intercepts of the linear model such that `Y` is approximated as `Y = X @ coef_ + intercept_`. .. versionadded:: 1.1 n_iter_ : list of shape (n_components,) Number of iterations of the power method, for each component. Empty if `algorithm='svd'`. n_features_in_ : int Number of features seen during :term:`fit`. feature_names_in_ : ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0 See Also -------- CCA : Canonical Correlation Analysis. PLSSVD : Partial Least Square SVD. Examples -------- >>> from sklearn.cross_decomposition import PLSCanonical >>> X = [[0., 0., 1.], [1.,0.,0.], [2.,2.,2.], [2.,5.,4.]] >>> Y = [[0.1, -0.2], [0.9, 1.1], [6.2, 5.9], [11.9, 12.3]] >>> plsca = PLSCanonical(n_components=2) >>> plsca.fit(X, Y) PLSCanonical() >>> X_c, Y_c = plsca.transform(X, Y) rn)rkrJrTrhr3r4)r\rlrKrLrmc s tj||dd||||ddS)Nrgr2rir)rorjr\rlrKrLrmrr/r0rps zPLSCanonical.__init__)r rrrrrarnrrrrrprr/r/rr0rs _ csTeZdZUdZejZeed<dD]Ze eq"d dddddfd d Z Z S) CCAap Canonical Correlation Analysis, also known as "Mode B" PLS. Read more in the :ref:`User Guide `. Parameters ---------- n_components : int, default=2 Number of components to keep. Should be in `[1, min(n_samples, n_features, n_targets)]`. scale : bool, default=True Whether to scale `X` and `Y`. max_iter : int, default=500 The maximum number of iterations of the power method. tol : float, default=1e-06 The tolerance used as convergence criteria in the power method: the algorithm stops whenever the squared norm of `u_i - u_{i-1}` is less than `tol`, where `u` corresponds to the left singular vector. copy : bool, default=True Whether to copy `X` and `Y` in fit before applying centering, and potentially scaling. If False, these operations will be done inplace, modifying both arrays. Attributes ---------- x_weights_ : ndarray of shape (n_features, n_components) The left singular vectors of the cross-covariance matrices of each iteration. y_weights_ : ndarray of shape (n_targets, n_components) The right singular vectors of the cross-covariance matrices of each iteration. x_loadings_ : ndarray of shape (n_features, n_components) The loadings of `X`. y_loadings_ : ndarray of shape (n_targets, n_components) The loadings of `Y`. x_rotations_ : ndarray of shape (n_features, n_components) The projection matrix used to transform `X`. y_rotations_ : ndarray of shape (n_features, n_components) The projection matrix used to transform `Y`. coef_ : ndarray of shape (n_features, n_targets) The coefficients of the linear model such that `Y` is approximated as `Y = X @ coef_ + intercept_`. intercept_ : ndarray of shape (n_targets,) The intercepts of the linear model such that `Y` is approximated as `Y = X @ coef_ + intercept_`. .. versionadded:: 1.1 n_iter_ : list of shape (n_components,) Number of iterations of the power method, for each component. n_features_in_ : int Number of features seen during :term:`fit`. feature_names_in_ : ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0 See Also -------- PLSCanonical : Partial Least Squares transformer and regressor. PLSSVD : Partial Least Square SVD. Examples -------- >>> from sklearn.cross_decomposition import CCA >>> X = [[0., 0., 1.], [1.,0.,0.], [2.,2.,2.], [3.,5.,4.]] >>> Y = [[0.1, -0.2], [0.9, 1.1], [6.2, 5.9], [11.9, 12.3]] >>> cca = CCA(n_components=1) >>> cca.fit(X, Y) CCA(n_components=1) >>> X_c, Y_c = cca.transform(X, Y) rnrrTr3r4rc s tj||ddd|||ddS)Nrgr>rhrirrrr/r0rpysz CCA.__init__)rrr/r/rr0rs W rc@sfeZdZUdZeeddddgdgdgdZeed<dd d d d d Z ddZ dddZ dddZ dS)raPartial Least Square SVD. This transformer simply performs a SVD on the cross-covariance matrix `X'Y`. It is able to project both the training data `X` and the targets `Y`. The training data `X` is projected on the left singular vectors, while the targets are projected on the right singular vectors. Read more in the :ref:`User Guide `. .. versionadded:: 0.8 Parameters ---------- n_components : int, default=2 The number of components to keep. Should be in `[1, min(n_samples, n_features, n_targets)]`. scale : bool, default=True Whether to scale `X` and `Y`. copy : bool, default=True Whether to copy `X` and `Y` in fit before applying centering, and potentially scaling. If `False`, these operations will be done inplace, modifying both arrays. Attributes ---------- x_weights_ : ndarray of shape (n_features, n_components) The left singular vectors of the SVD of the cross-covariance matrix. Used to project `X` in :meth:`transform`. y_weights_ : ndarray of (n_targets, n_components) The right singular vectors of the SVD of the cross-covariance matrix. Used to project `X` in :meth:`transform`. n_features_in_ : int Number of features seen during :term:`fit`. feature_names_in_ : ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0 See Also -------- PLSCanonical : Partial Least Squares transformer and regressor. CCA : Canonical Correlation Analysis. Examples -------- >>> import numpy as np >>> from sklearn.cross_decomposition import PLSSVD >>> X = np.array([[0., 0., 1.], ... [1., 0., 0.], ... [2., 2., 2.], ... [2., 5., 4.]]) >>> Y = np.array([[0.1, -0.2], ... [0.9, 1.1], ... [6.2, 5.9], ... [11.9, 12.3]]) >>> pls = PLSSVD(n_components=2).fit(X, Y) >>> X_c, Y_c = pls.transform(X, Y) >>> X_c.shape, Y_c.shape ((4, 2), (4, 2)) r?Nrbrcrerjr\rmrnrT)r\rmcCs||_||_||_dSr5r)rorjr\rmr/r/r0rpszPLSSVD.__init__c Cs*|t|||j|tj|jdd}t|dtj|jdd}|jdkrT|dd}|j }t |j d|j d|j d}||krt d |d |d t |||j\}}|_|_|_|_t|j|}t|dd \}}}|d d d |f}|d |}t||\}}|j} ||_| |_|jj d|_|S)aJFit model to data. Parameters ---------- X : array-like of shape (n_samples, n_features) Training samples. Y : array-like of shape (n_samples,) or (n_samples, n_targets) Targets. Returns ------- self : object Fitted estimator. rrrrIFrsr?rvrrwrxryrSN)r{rr|r#r}rmr r~rrjrrErr]r\rrrrr)rArrrrr) rorHrIrjrrTrUr,rWVr/r/r0rsL    z PLSSVD.fitcCst||j|tjdd}||j|j}t||j}|dk rt|ddtjd}|j dkrh| dd}||j |j }t||j }||fS|S)a  Apply the dimensionality reduction. Parameters ---------- X : array-like of shape (n_samples, n_features) Samples to be transformed. Y : array-like of shape (n_samples,) or (n_samples, n_targets), default=None Targets. Returns ------- x_scores : array-like or tuple of array-like The transformed data `X_transformed` if `Y is not None`, `(X_transformed, Y_transformed)` otherwise. F)r rNrI)rtrur r?rv)rr|r#r}rrr)rr r~rrrr)rorHrIZXrrZYrrr/r/r0rs  zPLSSVD.transformcCs|||||S)aLearn and apply the dimensionality reduction. Parameters ---------- X : array-like of shape (n_samples, n_features) Training samples. y : array-like of shape (n_samples,) or (n_samples, n_targets), default=None Targets. Returns ------- out : array-like or tuple of array-like The transformed data `X_transformed` if `Y is not None`, `(X_transformed, Y_transformed)` otherwise. rrr/r/r0r/szPLSSVD.fit_transform)r)N)N) rrrrrrrnrrrprrrr/r/r/r0rs D8 )r2r3r4F)T)/rnumbersrrrFabcrrnumpyr#Z scipy.linalgrbaserr r r r utilsr rZ utils.fixesrrZ utils.extmathrZutils.validationrrZutils._param_validationrr exceptionsr__all__rrr1rRrXr]r`rarrrrr/r/r/r0sX           ;  hk