U mdMT @s<ddlmZmZddlmZddlmZddlmZmZddl Z ddl Z erZddl mZnddZdd lmZdd lmZdd lmZdd lmZdd lmZmZddlmZmZmZmZdZe edZ!ddZ"d?ddZ#d@ddZ$dAddZ%dBddZ&dCddZ'dDddZ(e#e j)e j*e j+e j,e$e j-e(e%e'e&d Z.d d!Z/d"d#Z0dEd$d%Z1d&Z2d'Z3d(d)e3DZ4e2e5e4Z6Gd*d+d+Z7ee7j8Z9e9:d,edd-d.ge9:d/ge9:d0d1gd2gfd3gd4gfgee;e9dFd5d5d6d7ed8d9ee jfee;e?e?e@e?eeeAe@fe?e j>d: d;d<ZBGd=d>d>ZCdS)G)Appenderis_numeric_dtype)SP_LT_19)PD_LT_2)SequenceUnionNis_categorical_dtypecCs t|tjSN) isinstancepdZCategoricalDtypedtyper[/home/sam/Atlas/atlas_env/lib/python3.8/site-packages/statsmodels/stats/descriptivestats.pyr sr )stats) SimpleTable) jarque_bera)cache_readonly) Docstring Parameter) array_like bool_like float_likeint_like)  2KZ_cgY@cCs||Sr )maxmindfrrrpd_ptp sr(cCsdt|j|dS)Nraxis)npisnansum)xr*rrrnancount$sr/cCstj||dtj||dSNr))r+nanmaxnanminZarrr*rrrnanptp(sr4cCstj|d|dS)Nr))r+Znansumr3rrrnanuss,sr6cCstj|t|dSr0)r+ nanpercentile PERCENTILESr3rrrr70sr7cCstj||ddSNZomit)r*Z nan_policy)rkurtosisr3rrr nankurtosis4sr;cCstj||ddSr9)rskewr3rrr nanskewness8sr=) Zobsmeanstdr$r%Zptpvarr<Zussr: percentilescCs.zt|}Wntk r(tj}YnX|S)zi wrapper for scipy.stats.kurtosis that returns nan instead of raising Error missing options )rr: ValueErrorr+nanaresrrr _kurtosisKs  rGcCs.zt|}Wntk r(tj}YnX|S)ze wrapper for scipy.stats.skew that returns nan instead of raising Error missing options )rr<rBr+rCrDrrr_skewXs  rHcCst|}t||k}t||k}||d}ztt||||dj}Wn,tk r|tt||||d}YnX||fS)a8 Signs test Parameters ---------- samp : array_like 1d array. The sample for which you want to perform the sign test. mu0 : float See Notes for the definition of the sign test. mu0 is 0 by default, but it is common to set it to the median. Returns ------- M p-value Notes ----- The signs test returns M = (N(+) - N(-))/2 where N(+) is the number of values above `mu0`, N(-) is the number of values below. Values equal to `mu0` are discarded. The p-value for M is calculated using the binomial distribution and can be interpreted the same as for a t-test. The test-statistic is distributed Binom(min(N(+), N(-)), n_trials, .5) where n_trials equals N(+) + N(-). See Also -------- scipy.stats.wilcoxon g@?) r+asarrayr-rZ binomtestr%ZpvalueAttributeErrorZ binom_test)ZsampZmu0posnegMprrr sign_testes#  rP)nobsmissingr>std_errcir?iqr iqr_normalmad mad_normalcoef_varranger$r%r<r:rmodemedianrA)rQrRdistincttopfreqcCsg|]}|tkr|qSr)NUMERIC_STATISTICS.0statrrr srdc @seZdZdZdddgZeZeZe Z ddddde d d e e jejejfeeeeeeee eefed d d ZejejdddZeejdddZeejdddZeejdddZedddZedddZdS) Descriptiona Extended descriptive statistics for data Parameters ---------- data : array_like Data to describe. Must be convertible to a pandas DataFrame. stats : Sequence[str], optional Statistics to include. If not provided the full set of statistics is computed. This list may evolve across versions to reflect best practices. Supported options are: "nobs", "missing", "mean", "std_err", "ci", "ci", "std", "iqr", "iqr_normal", "mad", "mad_normal", "coef_var", "range", "max", "min", "skew", "kurtosis", "jarque_bera", "mode", "freq", "median", "percentiles", "distinct", "top", and "freq". See Notes for details. numeric : bool, default True Whether to include numeric columns in the descriptive statistics. categorical : bool, default True Whether to include categorical columns in the descriptive statistics. alpha : float, default 0.05 A number between 0 and 1 representing the size used to compute the confidence interval, which has coverage 1 - alpha. use_t : bool, default False Use the Student's t distribution to construct confidence intervals. percentiles : sequence[float] A distinct sequence of floating point values all between 0 and 100. The default percentiles are 1, 5, 10, 25, 50, 75, 90, 95, 99. ntop : int, default 5 The number of top categorical labels to report. Default is Attributes ---------- numeric_statistics The list of supported statistics for numeric data categorical_statistics The list of supported statistics for categorical data default_statistics The default list of statistics See Also -------- pandas.DataFrame.describe Basic descriptive statistics describe A simplified version that returns a DataFrame Notes ----- The selectable statistics include: * "nobs" - Number of observations * "missing" - Number of missing observations * "mean" - Mean * "std_err" - Standard Error of the mean assuming no correlation * "ci" - Confidence interval with coverage (1 - alpha) using the normal or t. This option creates two entries in any tables: lower_ci and upper_ci. * "std" - Standard Deviation * "iqr" - Interquartile range * "iqr_normal" - Interquartile range relative to a Normal * "mad" - Mean absolute deviation * "mad_normal" - Mean absolute deviation relative to a Normal * "coef_var" - Coefficient of variation * "range" - Range between the maximum and the minimum * "max" - The maximum * "min" - The minimum * "skew" - The skewness defined as the standardized 3rd central moment * "kurtosis" - The kurtosis defined as the standardized 4th central moment * "jarque_bera" - The Jarque-Bera test statistic for normality based on the skewness and kurtosis. This option creates two entries, jarque_bera and jarque_beta_pval. * "mode" - The mode of the data. This option creates two entries in all tables, mode and mode_freq which is the empirical frequency of the modal value. * "median" - The median of the data. * "percentiles" - The percentiles. Values included depend on the input value of ``percentiles``. * "distinct" - The number of distinct categories in a categorical. * "top" - The mode common categories. Labeled top_n for n in 1, 2, ..., ``ntop``. * "freq" - The frequency of the common categories. Labeled freq_n for n in 1, 2, ..., ``ntop``. rQrRr]NT皙?Frnumeric categoricalalphause_trAntopdatarrhrirjrkrArlc Cs|} t|tjtjfs$t|ddd} | jdkr8t|}t|d}t|d}g} d} |rh| tj d} |r| d| | dkrd nd7} | d7} |s|st d t| | |_ |j j dd krt d | d dd|j jD|_dd|j jD|_|dk r.dd|D} | r.t d| d|dkr@ttnt||_t|d|_d|jk|_d|jk|_|jr|jd krt|jkrnnt dddgddgddgddtd|jdDddtd|jdDd } | D]H}||jkr|j|}|jd|| ||j|dd|_qt|d!dd"d#|_t|j|_t|jj d |jj d krt d$t|jd%kst|jd krt d&t |d'|_!d |krdksnt d(t|d)|_"dS)*Nrnr5)maxdimrrhricategoryzand z4At least one of numeric and categorical must be Truerz Selecting z results in an empty DataFramecSsg|] }t|qSr)rrbdtrrrrd/sz(Description.__init__..cSsg|] }t|qSrrrrrrrrd0scSsg|]}|tkr|qSr)DEFAULT_STATISTICSrarrrrd5sz, z are not known statisticsrlr^r_z"top must be a non-negative integerr[ mode_frequpper_cilower_cirjarque_bera_pvalcSsg|]}d|qSZtop_rrbirrrrdHscSsg|]}d|qSZfreq_rrzrrrrdIs)r[rTrr^r_rAd)rorzpercentiles must be distinctdz.percentiles must be strictly between 0 and 100rjz&alpha must be strictly between 0 and 1rk)#r r Series DataFramerndimrappendr+numberrBZ select_dtypes_datashapeZdtypes _is_numeric _is_cat_likejoinlistrt_statsr_ntop _compute_top _compute_freqr-rZindex _percentilessortuniqueanyr_alpha_use_t)selfrnrrhrirjrkrArlZdata_arrincludeZ col_typesZundefZ replacementskeyidxrrr__init__ s            ,    $ zDescription.__init__)r'returncsjfdd|jDS)Ncsg|]}|jkr|qSrr)rbsr&rrrdcs z(Description._reorder..)locr)rr'rr&r_reorderbszDescription._reorder)rcCsT|j}|j}|jddkr|S|jddkr0|Stj||gdd}|||jjS)z Descriptive statistics for both numeric and categorical data Returns ------- DataFrame The statistics rrr))rhrirr concatrrcolumns)rrhrir'rrrframees zDescription.framec"sjjddjf}|j}|j\}}|}|}|}||}| } | j|dk|j|dkd<j rt |d djd} nt j djd} dd} || j} | jdkr^t| tjr tj| dtd } tj| dtjd }nPg} g}| jD],}| j|}| |d||dqt| } t|}ntd} }|dk}t| jdtj}|||j|||<|}zNdd l m!}| }|D]0}|||j"r||#t$%tj||<qWnt&k rYnX|jddkr.|'d |'d }n|}d d|jfddddj}| }tj|j|dk<||}tj(tj)|tjd |jd|d|jd||| || | || | ||||t*||+|,|d|d|t-t j d d g|t.dtj/|d|dtj(| |dtj(||d|0d}fdd|1D}tjt2|3|t2|4d}dj5kr~|S|jddkr|'j6d#t}ntjj6dtd}t7t8d|jd|jkrdd|jD|_nd}d}|j} |r:|d9}t8||j}t7t-|dkrd }qt8|| |d} d!t9t:|ddd"}!d#|!d$fd%d| D|_j5|j;_5._mode..) r rr+Z numpy_dtypedropnaZto_numpyrrr[ZisscalarrrrC)ZserrZser_no_missingkwargsZmode_resrrr_modesz"Description.numeric.._moder )is_extension_array_dtypeg?g?cSs,t|}|jddkr$tjfdSt|S)Nrr5)r+rJrrCr)crErrr_safe_jarque_beras  z.Description.numeric.._safe_jarque_beracst|Sr )rr)r.)rrrz%Description.numeric..expand)Z result_typer)rQrRr>rSrvrwr?rUrWrYrZr$r%r<r:rVrXrrxr[rur\cs i|]\}}|jkr||qSrrrbkvrrr s z'Description.numeric..)rrrAr~)rrcSsg|]}td|dqS)r~%)intrbrrrrrdsz'Description.numeric..TrFz0.fz{0:z}%csg|]}|qSr)formatr)outputrrrdsr))>rrrrrr?countr>abscopyrrtZppfrZnormapplyTsizer r rr+rJrint64rrZ atleast_1demptyfullrCZpandas.api.typesrrastypeobjectfillna ImportErrorZquantileronesr(r$r%diffsqrtpir\itemsrvalueskeysrrallfloorlenstrtolistrr)"rr'cols_rr?rr>rWrSqrZ mode_valuesr[Z mode_countsrrrruZ_dfrcolrUZjbZnan_meanrYresultsfinal results_dfpercZdupescalerfmtr)rrrrrhxs  $        "         " zDescription.numericcsjjddddjDfjd}j}fddDtjfddDtjd}i}i}D]}|}|jd j kr|j dj ||<t |j dd ||<qlt |j }|dgj t|7}|||<t |} | tjgj t| 7} t | ||<qld dtdj dD} tj|d | |d } ddtdj dD} tj|d | |d } tjtj|tjdjd |djd |d} fdd| D}tjt ||t |d d}jrtj|| gd d}jrtj|| gd d}|S)z Descriptive statistics for categorical data Returns ------- DataFrame The statistics of the categorical columns NcSsg|]}|qSrrrbrrrrrdsz+Description.categorical..rcsi|]}||jddqS)T) normalize)Z value_countsrr&rrrsz+Description.categorical..csi|]}||jdqS)r)rr)vcrrrsr rrcSsg|]}d|qSryrrzrrrrd*sr)rrrcSsg|]}d|qSr|rrzrrrrd,sr)rQrRr]cs i|]\}}|jkr||qSrrrrrrr6s )rrrr))rrrrrr rr+rrrrJZilocrrrCrZrrrrrrrrrr)rrrr]r^r_rZsinglerZfreq_valrZtop_dfZfreq_dfrrrr)r'rrrri sX     zDescription.categoricalc Cs|jt}|d}dd|jD}dd|jD}g}|D]\}}|dd|DqBdd}t|||dd d |d id gt |d S)z Summary table of the descriptive statistics Returns ------- SimpleTable A table instance supporting export to text, csv and LaTeX rpcSsg|] }t|qSrrrrrrrdOsz'Description.summary..cSsg|] }t|qSrrrrrrrdPscSsg|]}|qSrr)rbrrrrrdSscSs.t|tr|S|d|kr&tt|S|dS)Nrz0.4g)r rr)rrrr _formatterUs    z'Description.summary.._formatterzDescriptive StatisticsZ data_fmtsz%s)rrr)headerstubstitleZtxt_fmtZ datatypes) rrrrrrZiterrowsrrr)rr'rrrnrrowrrrrsummaryDs    zDescription.summarycCst|Sr )rrZas_textrrrr__str__eszDescription.__str__)N) __name__ __module__ __qualname____doc__Z_int_fmtr`Znumeric_statisticsCATEGORICAL_STATISTICSZcategorical_statisticsrtZdefault_statisticsr8rr+ndarrayr rrrrboolrrrrrrrhrirrrrrrrresBR  Y9!reZReturnsrzDescriptive statisticsZ AttributeszSee Also)zpandas.DataFrame.describeNzBasic descriptive statistics)reNz;Descriptive statistics class with additional output optionsTrfFrrg) rnrrhrirjrkrArlrc Cst||||||||djS)Nrg)rerrmrrrdescribe}s rc@seZdZdZddZdS)Describez Removed. cCs tddS)NzDescribe has been removed)NotImplementedError)rZdatasetrrrrszDescribe.__init__N)rrrrrrrrrrsr)r)r)r)r)r)r)r)N)DZstatsmodels.compat.pandasrrZstatsmodels.compat.scipyrrtypingrrnumpyr+Zpandasr Zpandas.core.dtypes.commonr ZscipyrZstatsmodels.iolib.tablerZstatsmodels.stats.stattoolsrZstatsmodels.tools.decoratorsrZstatsmodels.tools.docstringrrZstatsmodels.tools.validationrrrrr8arrayZ QUANTILESr(r/r4r6r7r;r=ZnanmeanZnanstdr1r2ZnanvarMISSINGrGrHrPr`rZ _additionaltuplertrerZdsZ replace_blockrrrrrrrrrrrrrs              / ;