U mde@sbdZddlZddlmZddlmZmZmZm Z m Z dddgZ Gd ddeZ Gd ddeZ dS) aZ Multivariate Conditional and Unconditional Kernel Density Estimation with Mixed Data Types. References ---------- [1] Racine, J., Li, Q. Nonparametric econometrics: theory and practice. Princeton University Press. (2007) [2] Racine, Jeff. "Nonparametric Econometrics: A Primer," Foundation and Trends in Econometrics: Vol 3: No 1, pp1-88. (2008) http://dx.doi.org/10.1561/0800000009 [3] Racine, J., Li, Q. "Nonparametric Estimation of Distributions with Categorical and Continuous Data." Working Paper. (2000) [4] Racine, J. Li, Q. "Kernel Estimation of Multivariate Conditional Distributions Annals of Economics and Finance 5, 211-235 (2004) [5] Liu, R., Yang, L. "Kernel estimation of multivariate cumulative distribution function." Journal of Nonparametric Statistics (2008) [6] Li, R., Ju, G. "Nonparametric Estimation of Multivariate CDF with Categorical and Continuous Data." Working Paper [7] Li, Q., Racine, J. "Cross-validated local linear nonparametric regression" Statistica Sinica 14(2004), pp. 485-512 [8] Racine, J.: "Consistent Significance Testing for Nonparametric Regression" Journal of Business & Economics Statistics [9] Racine, J., Hart, J., Li, Q., "Testing the Significance of Categorical Predictor Variables in Nonparametric Regression Models", 2006, Econometric Reviews 25, 523-544 N)kernels) GenericKDEEstimatorSettingsgpke LeaveOneOut _adjust_shapeKDEMultivariateKDEMultivariateConditionalrc@sVeZdZdZdddZddZddfd d Zdd d Zdd dZddZ ddZ dS)r a Multivariate kernel density estimator. This density estimator can handle univariate as well as multivariate data, including mixed continuous / ordered discrete / unordered discrete data. It also provides cross-validated bandwidth selection methods (least squares, maximum likelihood). Parameters ---------- data : list of ndarrays or 2-D ndarray The training data for the Kernel Density Estimation, used to determine the bandwidth(s). If a 2-D array, should be of shape (num_observations, num_variables). If a list, each list element is a separate observation. var_type : str The type of the variables: - c : continuous - u : unordered (discrete) - o : ordered (discrete) The string should contain a type specifier for each variable, so for example ``var_type='ccuo'``. bw : array_like or str, optional If an array, it is a fixed user-specified bandwidth. If a string, should be one of: - normal_reference: normal reference rule of thumb (default) - cv_ml: cross validation maximum likelihood - cv_ls: cross validation least squares defaults : EstimatorSettings instance, optional The default values for (efficient) bandwidth estimation. Attributes ---------- bw : array_like The bandwidth parameters. See Also -------- KDEMultivariateConditional Examples -------- >>> import statsmodels.api as sm >>> nobs = 300 >>> np.random.seed(1234) # Seed random generator >>> c1 = np.random.normal(size=(nobs,1)) >>> c2 = np.random.normal(2, 1, size=(nobs,1)) Estimate a bivariate distribution and display the bandwidth found: >>> dens_u = sm.nonparametric.KDEMultivariate(data=[c1,c2], ... var_type='cc', bw='normal_reference') >>> dens_u.bw array([ 0.39967419, 0.38423292]) NcCs||_t|j|_t||j|_||_t|j\|_|_|j|jkrNt d|dkr\t n|}| ||j s~| ||_n |||_dS)NzGThe number of observations must be larger than the number of variables.)var_typelenk_varsrdata data_typenpshapenobs ValueErrorr _set_defaults efficient _compute_bwbw_compute_efficient)selfrr rdefaultsra/home/sam/Atlas/atlas_env/lib/python3.8/site-packages/statsmodels/nonparametric/kernel_density.py__init__es   zKDEMultivariate.__init__cCsXd}|dt|jd7}|dt|jd7}|d|jd7}|d|jd7}|S) Provide something sane to print.z KDE instance zNumber of variables: k_vars =  zNumber of samples: nobs = zVariable types: BW selection method: )strr rr _bw_methodrZrprrrr__repr__us zKDEMultivariate.__repr__cCs|SNrxrrr~zKDEMultivariate.cCsVt|j}d}t|D]8\}}t|| |j|ddf |jd}|||7}q| S)aX Returns the leave-one-out likelihood function. The leave-one-out likelihood function for the unconditional KDE. Parameters ---------- bw : array_like The value for the bandwidth parameter(s). func : callable, optional Function to transform the likelihood values (before summing); for the log likelihood, use ``func=np.log``. Default is ``f(x) = x``. Notes ----- The leave-one-out kernel estimator of :math:`f_{-i}` is: .. math:: f_{-i}(X_{i})=\frac{1}{(n-1)h} \sum_{j=1,j\neq i}K_{h}(X_{i},X_{j}) where :math:`K_{h}` represents the generalized product kernel estimator: .. math:: K_{h}(X_{i},X_{j}) = \prod_{s=1}^{q}h_{s}^{-1}k\left(\frac{X_{is}-X_{js}}{h_{s}}\right) rNr data_predictr )rr enumeraterr )rrfuncLOOLiX_not_if_irrrloo_likelihood~s zKDEMultivariate.loo_likelihoodc Cst|dkr|j}n t||j}g}tt|dD]2}|t|j|j||ddf|j d|j q2t |}|S)aT Evaluate the probability density function. Parameters ---------- data_predict : array_like, optional Points to evaluate at. If unspecified, the training data is used. Returns ------- pdf_est : array_like Probability density function evaluated at `data_predict`. Notes ----- The probability density is given by the generalized product kernel estimator: .. math:: K_{h}(X_{i},X_{j}) = \prod_{s=1}^{q}h_{s}^{-1}k\left(\frac{X_{is}-X_{js}}{h_{s}}\right) Nrr* rrr rangerrappendrrr rsqueeze)rr+pdf_estr0rrrpdfs  zKDEMultivariate.pdfc Csz|dkr|j}n t||j}g}tt|dD]8}|t|j|j||ddf|j dddd|j q2t |}|S)a Evaluate the cumulative distribution function. Parameters ---------- data_predict : array_like, optional Points to evaluate at. If unspecified, the training data is used. Returns ------- cdf_est : array_like The estimate of the cdf. Notes ----- See https://en.wikipedia.org/wiki/Cumulative_distribution_function For more details on the estimation see Ref. [5] in module docstring. The multivariate CDF for mixed data (continuous and ordered/unordered discrete) is estimated by: .. math:: F(x^{c},x^{d})=n^{-1}\sum_{i=1}^{n}\left[G(\frac{x^{c}-X_{i}}{h})\sum_{u\leq x^{d}}L(X_{i}^{d},x_{i}^{d}, \lambda)\right] where G() is the product kernel CDF estimator for the continuous and L() for the discrete variables. Used bandwidth is ``self.bw``. Nr gaussian_cdfaitchisonaitken_cdf wangryzin_cdf)rr+r ckertypeukertypeokertyper4)rr+cdf_estr0rrrcdfs   zKDEMultivariate.cdfcCsd}ttjtjtjd}|j}|j }|j}t dd|D}|| }t |j } t |D]n} t|D]<\} } || || |dd| f|| | f| dd| f<ql| j dd|} | jdd}||7}q`ttjtjtjd}t|j}d}t |j dd|j df} t|D]t\} }t|D]@\} } || || |dd| f || | f| dd| f<q(| j dd|} || jdd7}q||dd|||dS) a Returns the Integrated Mean Square Error for the unconditional KDE. Parameters ---------- bw : array_like The bandwidth parameter(s). Returns ------- CV : float The cross-validation objective function. Notes ----- See p. 27 in [1]_ for details on how to handle the multivariate estimation with mixed data types see p.6 in [2]_. The formula for the cross-validation objective function is: .. math:: CV=\frac{1}{n^{2}}\sum_{i=1}^{n}\sum_{j=1}^{N} \bar{K}_{h}(X_{i},X_{j})-\frac{2}{n(n-1)}\sum_{i=1}^{n} \sum_{j=1,j\neq i}^{N}K_{h}(X_{i},X_{j}) Where :math:`\bar{K}_{h}` is the multivariate product convolution kernel (consult [2]_ for mixed data types). References ---------- .. [1] Racine, J., Li, Q. Nonparametric econometrics: theory and practice. Princeton University Press. (2007) .. [2] Racine, J., Li, Q. "Nonparametric Estimation of Distributions with Categorical and Continuous Data." Working Paper. (2000) r)coucSsg|] }|dkqS)rBr).0rBrrr 0sz(KDEMultivariate.imse..NrZaxis)dictrZgaussian_convolutionZwang_ryzin_convolutionZaitchison_aitken_convolutionrrr rarrayprodemptyrr5r,sumZgaussianZ wang_ryzinZaitchison_aitkenr)rrFZkertypesrrr Zix_contZ_bw_cont_productZKvalr0iiZvtypeZdensZ k_bar_sumr.r/r1rrrimsesH3          zKDEMultivariate.imsecCsd}|jf}||fS)@Helper method to be able to pass needed vars to _compute_subset.r )r rZ class_typeZ class_varsrrr_get_class_vars_typeNsz$KDEMultivariate._get_class_vars_type)NN)N)N __name__ __module__ __qualname____doc__rr$r3r9rArPrSrrrrr )s;  $ $ 0Xc@sVeZdZdZdddZddZddfd d Zdd d Zdd dZddZ ddZ dS)r a Conditional multivariate kernel density estimator. Calculates ``P(Y_1,Y_2,...Y_n | X_1,X_2...X_m) = P(X_1, X_2,...X_n, Y_1, Y_2,..., Y_m)/P(X_1, X_2,..., X_m)``. The conditional density is by definition the ratio of the two densities, see [1]_. Parameters ---------- endog : list of ndarrays or 2-D ndarray The training data for the dependent variables, used to determine the bandwidth(s). If a 2-D array, should be of shape (num_observations, num_variables). If a list, each list element is a separate observation. exog : list of ndarrays or 2-D ndarray The training data for the independent variable; same shape as `endog`. dep_type : str The type of the dependent variables: c : Continuous u : Unordered (Discrete) o : Ordered (Discrete) The string should contain a type specifier for each variable, so for example ``dep_type='ccuo'``. indep_type : str The type of the independent variables; specified like `dep_type`. bw : array_like or str, optional If an array, it is a fixed user-specified bandwidth. If a string, should be one of: - normal_reference: normal reference rule of thumb (default) - cv_ml: cross validation maximum likelihood - cv_ls: cross validation least squares defaults : Instance of class EstimatorSettings The default values for the efficient bandwidth estimation Attributes ---------- bw : array_like The bandwidth parameters See Also -------- KDEMultivariate References ---------- .. [1] https://en.wikipedia.org/wiki/Conditional_probability_distribution Examples -------- >>> import statsmodels.api as sm >>> nobs = 300 >>> c1 = np.random.normal(size=(nobs,1)) >>> c2 = np.random.normal(2,1,size=(nobs,1)) >>> dens_c = sm.nonparametric.KDEMultivariateConditional(endog=[c1], ... exog=[c2], dep_type='c', indep_type='c', bw='normal_reference') >>> dens_c.bw # show computed bandwidth array([ 0.41223484, 0.40976931]) NcCs||_||_|||_t|j|_t|j|_t||j|_t||j|_t |j\|_ |_t |j|jf|_ t |j d|_|dkrtn|}|||js|||_n |||_dS)Nr)dep_type indep_typerr k_depk_indeprendogexogrrr column_stackrr rrrrrr)rr]r^rYrZrrrrrrs    z#KDEMultivariateConditional.__init__cCsd}|dt|jd7}|dt|jd7}|dt|jd7}|d|jd7}|d|jd7}|d|jd7}|S) rz$KDEMultivariateConditional instance z+Number of independent variables: k_indep = rz'Number of dependent variables: k_dep = zNumber of observations: nobs = z!Independent variable types: zDependent variable types: r )r!r\r[rrZrYr"r#rrrr$s z#KDEMultivariateConditional.__repr__cCs|Sr%rr&rrrr(r)z#KDEMultivariateConditional.c Cst|j}t|j}d}t|D]|\}}t|}t|| |j|ddf |j|jd} t||j d| |j|ddf |jd} | | } ||| 7}q$| S)a Returns the leave-one-out conditional likelihood of the data. If `func` is not equal to the default, what's calculated is a function of the leave-one-out conditional likelihood. Parameters ---------- bw : array_like The bandwidth parameter(s). func : callable, optional Function to transform the likelihood values (before summing); for the log likelihood, use ``func=np.log``. Default is ``f(x) = x``. Returns ------- L : float The value of the leave-one-out function for the data. Notes ----- Similar to ``KDE.loo_likelihood`, but substitute ``f(y|x)=f(x,y)/f(x)`` for ``f(x)``. rNr*) rrr^__iter__r,nextrrYrZr[) rrr-ZyLOOZxLOOr/r0ZY_jr1f_yxf_xr2rrrr3s  z)KDEMultivariateConditional.loo_likelihoodcCs|dkr|j}n t||j}|dkr,|j}n t||j}g}t||f}tt|dD]f}t |j |j ||ddf|j |j d}t |j |jd|j||ddf|j d}|||q\t|S)aQ Evaluate the probability density function. Parameters ---------- endog_predict : array_like, optional Evaluation data for the dependent variables. If unspecified, the training data is used. exog_predict : array_like, optional Evaluation data for the independent variables. Returns ------- pdf : array_like The value of the probability density at `endog_predict` and `exog_predict`. Notes ----- The formula for the conditional probability density is: .. math:: f(y|x)=\frac{f(x,y)}{f(x)} with .. math:: f(x)=\prod_{s=1}^{q}h_{s}^{-1}k \left(\frac{x_{is}-x_{js}}{h_{s}}\right) where :math:`k` is the appropriate kernel for each variable. Nrr*)r]rr[r^r\rr_r5rrrrrYrZr6r7)r endog_predict exog_predictr8r+r0rbrcrrrr9s&    zKDEMultivariateConditional.pdfc Cs"|dkr|j}n t||j}|dkr,|j}n t||j}t|d}t|}t|D]}t |j |jd|j||ddf|j d|j }t |}t |j d|j|j||ddf|jddddd}t |j |jd|j||ddf|j dd }||jdd } | |j |||<qX|S) a Cumulative distribution function for the conditional density. Parameters ---------- endog_predict : array_like, optional The evaluation dependent variables at which the cdf is estimated. If not specified the training dependent variables are used. exog_predict : array_like, optional The evaluation independent variables at which the cdf is estimated. If not specified the training independent variables are used. Returns ------- cdf_est : array_like The estimate of the cdf. Notes ----- For more details on the estimation see [2]_, and p.181 in [1]_. The multivariate conditional CDF for mixed data (continuous and ordered/unordered discrete) is estimated by: .. math:: F(y|x)=\frac{n^{-1}\sum_{i=1}^{n}G(\frac{y-Y_{i}}{h_{0}}) W_{h}(X_{i},x)}{\widehat{\mu}(x)} where G() is the product kernel CDF estimator for the dependent (y) variable(s) and W() is the product kernel CDF estimator for the independent variable(s). References ---------- .. [1] Racine, J., Li, Q. Nonparametric econometrics: theory and practice. Princeton University Press. (2007) .. [2] Liu, R., Yang, L. "Kernel estimation of multivariate cumulative distribution function." Journal of Nonparametric Statistics (2008) Nrr*r:r;r<F)rr+r r=r>r?tosumrr+r rfrG)r]rr[r^r\rrrLr5rrrZrr7rYrM) rrdreZN_data_predictr@r0Zmu_xZ cdf_endogZcdf_exogSrrrrAs@)     zKDEMultivariateConditional.cdfc Cst|j}d}t|j}t|jddf}t|D]t\}}|dd|jdf}|ddd|jf} t| |} t|| } t||} t||} t ||jd| |j |ddf|j dd}t ||jd| |j |ddf|j dd}t |d|j| | |j ddddd }||| |d }t || |j|ddf |j |j d |}t ||jd| |j |ddf |j d |}|||d d ||7}q4||S) a The integrated mean square error for the conditional KDE. Parameters ---------- bw : array_like The bandwidth parameter(s). Returns ------- CV : float The cross-validation objective function. Notes ----- For more details see pp. 156-166 in [1]_. For details on how to handle the mixed variable types see [2]_. The formula for the cross-validation objective function for mixed variable types is: .. math:: CV(h,\lambda)=\frac{1}{n}\sum_{l=1}^{n} \frac{G_{-l}(X_{l})}{\left[\mu_{-l}(X_{l})\right]^{2}}- \frac{2}{n}\sum_{l=1}^{n}\frac{f_{-l}(X_{l},Y_{l})}{\mu_{-l}(X_{l})} where .. math:: G_{-l}(X_{l}) = n^{-2}\sum_{i\neq l}\sum_{j\neq l} K_{X_{i},X_{l}} K_{X_{j},X_{l}}K_{Y_{i},Y_{j}}^{(2)} where :math:`K_{X_{i},X_{l}}` is the multivariate product kernel and :math:`\mu_{-l}(X_{l})` is the leave-one-out estimator of the pdf. :math:`K_{Y_{i},Y_{j}}^{(2)}` is the convolution kernel. The value of the function is minimized by the ``_cv_ls`` method of the `GenericKDE` class to return the bw estimates that minimize the distance between the estimated and "true" probability density. References ---------- .. [1] Racine, J., Li, Q. Nonparametric econometrics: theory and practice. Princeton University Press. (2007) .. [2] Racine, J., Li, Q. "Nonparametric Estimation of Distributions with Categorical and Continuous Data." Working Paper. (2000) rrNFrgZgauss_convolutionZwangryzin_convolutionZaitchisonaitken_convolution)rr+r r=r?r>rfrHr*)rrfloatrrZonesr,r[Zkronrr^rZrYrM)rrZzLOOZCVrZexpanderrOZXYZYe_LZYe_RZXe_LZXe_RZK_Xi_XlZK_Xj_XlZK2_Yi_YjGZf_X_YZm_xrrrrP[sV/       zKDEMultivariateConditional.imsecCsd}|j|j|jf}||fS)rQr )r[rYrZrRrrrrSsz/KDEMultivariateConditional._get_class_vars_type)N)NN)NNrTrrrrr UsB  ( 4 HP)rXnumpyrrZ _kernel_baserrrrr__all__r r rrrrs  .