Statistics & indices¶

Descriptive statistics, agreement measures, sample-depth signal strength, and derived indices.

stats¶

Generates summary statistics

Extended Summary

Generates summary statistics for .RWL and .CSV format files. It outputs a dataframe with 'first', 'last', 'year', 'mean', 'median', 'stdev', 'skew', 'kurtosis', 'gini', 'ar1' for each series in data file.

Parameters:

Name	Type	Description	Default
`inp`	`str or pandas dataframe`	a data file (.CSV or .RWL) or a pandas dataframe imported from dpl.readers().	required

Returns:

Name	Type	Description
`data`	`pandas dataframe`

Examples:

>>> dpl.stats(<data>)

References

.. [1] https:/opendendro.org/dplpy-man/#stats

samp_stats¶

Per-year sample-tracking statistics for a ring-width dataset.

For every year in the dataset this reports how many series are present and, across just those present series, their mean length and mean cambial age -- the num / seg / age columns of an ARSTAN "tabs" file.

Parameters:

Name	Type	Description	Default
`data`	`pandas dataframe`	ring widths, years as the index and series as columns (e.g. from dpl.readers()). Raw widths or detrended indices give the same result -- only the pattern of present/absent values is used.	required

Returns:

Name	Type	Description
`out`	`pandas dataframe indexed by year with columns`	`samp_depth` : number of series present that year (the sample depth; ARSTAN's `num`). An integer count. `seg` : mean segment length of the series present that year, where a series' segment length is its number of measured rings. `age` : mean cambial age that year, where a series' cambial age is the number of years since its first ring (`year - first_year + 1`). Years with no series present get `samp_depth` 0 and NaN for `seg` / `age`.

Notes

seg counts a series' measured rings; age counts years since its first ring. The two differ only for a series with an interior gap (an unmeasured year inside its span), for which the tree still ages but the ring is not counted.

Examples:

>>> import dplpy as dpl
>>> data = dpl.readers("../tests/data/csv/file.csv")
>>> dpl.samp_stats(data)

rwi_stats¶

Whole-record chronology signal statistics (rbar, EPS, SNR).

Extended Summary

Computes the mean interseries correlation (rbar), expressed population signal (EPS), and signal-to-noise ratio (SNR) over the entire record, as a single-row summary. This is the non-running counterpart of rwi_stats_running(): it simply calls that function with running_window=False, exactly as dplR's rwi.stats() delegates to rwi.stats.running().

See rwi_stats_running() for a full description of the parameters and the statistics computed.

Parameters:

Name	Type	Description	Default
`rwi_data`	`pandas dataframe`	detrended ring-width indices (e.g. from dpl.detrend()).	required
`ids`	`dict, pandas dataframe, or None`	tree/core structure; see rwi_stats_running().	`None`
`period`	`str`	"max" (pairwise-complete overlaps) or "common" (complete rows only).	`"max"`
`corr`	`str`	correlation type, "Spearman" or "Pearson".	`"Spearman"`
`prewhiten`	`boolean`	prewhiten each series with an AR model before computing correlations.	`False`
`min_corr_overlap`	`int or None`	minimum number of overlapping years for a series pair to contribute.	`None`
`zero_is_missing`	`boolean`	treat exact zeros as missing values.	`True`
`round_decimals`	`int or None`	decimal places to round the statistic columns to.	`3`

Returns:

Name	Type	Description
`result`	`a one-row pandas dataframe of chronology signal statistics.`

Examples:

>>> import dplpy as dpl
>>> rwi = dpl.detrend(dpl.readers("../tests/data/csv/file.csv"), plot=False)
>>> dpl.rwi_stats(rwi)

References

.. [1] https://rdrr.io/cran/dplR/man/rwi.stats.running.html .. [2] Wigley, Briffa & Jones (1984), J. Climate Appl. Meteor., 23, 201-213.

rwi_stats_running¶

Running (moving-window) chronology signal statistics (rbar, EPS, SNR).

Extended Summary

Computes, for each moving window along the chronology, the mean interseries correlation (rbar) and, from it, the expressed population signal (EPS) and signal-to-noise ratio (SNR) of Wigley et al. (1984). This is a port of dplR's rwi.stats.running(); rwi_stats() is the whole-record special case (running_window=False).

When tree/core IDs are supplied (see ids), correlations are split into within-tree (rbar_wt) and between-tree (rbar_bt) components, and the effective signal rbar_eff accounts for the averaging of multiple cores per tree (via an effective cores-per-tree factor c_eff). Without IDs, every series is treated as its own tree and rbar_eff == rbar_bt == rbar_tot.

EPS = n * rbar_eff / ((n - 1) * rbar_eff + 1), where n is the number of trees contributing to the between-tree correlations, measures how well the finite sample represents a (hypothetically infinite) population chronology. SNR = n * rbar_eff / (1 - rbar_eff) is the corresponding signal-to-noise ratio (Cook & Pederson 2011).

NOTE: no acceptance threshold (e.g. the frequently cited EPS > 0.85) is applied or implied here. As Wigley et al. (1984) state and Buras (2017) reemphasizes, that threshold was an illustrative example for SSS (see dpl.sss()), not a general criterion for EPS; whether a value is adequate is a judgement for the analyst, not the software.

Implementation notes

The within-window pairwise correlations are computed with a single pandas .corr(min_periods=...) call, which natively performs pairwise-complete correlation and drops pairs with fewer than min_corr_overlap overlapping years -- replacing dplR's hand-written per-pair correlation loops. Per-series mean normalization (which dplR applies internally) is deliberately skipped because it cannot change any correlation; AR prewhitening, which does change correlations, is applied when requested. The effective cores-per-tree factor is computed by counting cores-with-data per tree directly, rather than back-calculating it from the number of valid within-tree correlation pairs as dplR does; the two agree whenever within-tree overlap is adequate.

Parameters:

Name	Type	Description	Default
`rwi_data`	`pandas dataframe`	detrended ring-width indices (e.g. from dpl.detrend()), with years as the index and series as columns.	required
`ids`	`dict, pandas dataframe, or None`	tree/core structure used to split within- vs between-tree correlations. Either a dict mapping {series_name: tree_id}, a dataframe with a 'tree' column indexed by series name (such as produced by dpl.read_ids()), or None to treat every series as its own tree (one core per tree).	`None`
`period`	`str`	"max" uses each pair's own overlapping years (pairwise-complete); "common" restricts each window to years present in every series.	`"max"`
`corr`	`str`	correlation type, "Spearman" or "Pearson".	`"Spearman"`
`prewhiten`	`boolean`	prewhiten each series with an AR model before computing correlations.	`False`
`running_window`	`boolean`	if False, a single window spanning the whole record is used (this is what rwi_stats() invokes).	`True`
`window_length`	`int or None`	length of each moving window in years; defaults to min(50, n_years).	`None`
`window_overlap`	`int or None`	number of years successive windows overlap; defaults to window_length // 2.	`None`
`min_corr_overlap`	`int or None`	minimum number of overlapping years required for a series pair to contribute a correlation; defaults to min(30, window_length).	`None`
`zero_is_missing`	`boolean`	treat exact zeros as missing values (a zero RWI is treated as absent).	`True`
`round_decimals`	`int or None`	decimal places to round the real-valued statistic columns to; None leaves them unrounded.	`3`
`common_interval`	`str, (int, int), or None`	restrict the data to a common interval before computing statistics. 'series', 'years' or 'both' selects a complete overlap-only rectangle via dpl.common_interval() (maximising cores, years, or cells); a (start_year, end_year) pair restricts to that inclusive span of your choosing. None (default) uses the full record. This is the recommended way to get chronology statistics on unevenly-distributed data, where period='common' on the full record can be empty.	`None`

Returns:

Name	Type	Description
`result`	`pandas dataframe with one row per window (or a single row if`	running_window=False). Columns: start_year, mid_year, end_year (running only), n_cores, n_trees, n, n_tot, n_wt, n_bt, rbar_tot, rbar_wt, rbar_bt, c_eff, rbar_eff, eps, snr.

Examples:

>>> import dplpy as dpl
>>> rwi = dpl.detrend(dpl.readers("../tests/data/csv/file.csv"), plot=False)
>>> dpl.rwi_stats_running(rwi, window_length=60, window_overlap=30)

References

.. [1] https://rdrr.io/cran/dplR/man/rwi.stats.running.html .. [2] Wigley, Briffa & Jones (1984), J. Climate Appl. Meteor., 23, 201-213. .. [3] Buras (2017), Dendrochronologia, 44, 130-132.

sens1¶

Mean sensitivity (dplR sens1).

The standard measure of year-to-year variability in dendrochronology (Douglass; Eq. 1 of Biondi and Qeadan 2008), typically computed on detrended series. For a single series of n (non-missing) values,

sens1 = (2 / (n - 1)) * sum_i |x_i - x_{i-1}| / (x_i + x_{i-1})

where a 0/0 term arising from an x_i + x_{i-1} == 0 pair is skipped (matching dplR's C ISNAN guard) while n - 1 is unchanged.

Parameters:

Name	Type	Description	Default
`data`	`pandas.DataFrame, pandas.Series, 1-D array-like, or str`	A ring-width dataset (years x series), a single series, or a path/URL to a .csv/.rwl file. NA/NaN values are dropped before the calculation.	required

Returns:

Type	Description
`float or Series`	For a single series, the scalar mean sensitivity. For a DataFrame (or a file path, which is read into one), a Series of the statistic indexed by series name -- the equivalent of R's `apply(rwl, 2, sens1)`. A series with fewer than two values yields `NaN`.

Examples:

>>> dpl.sens1(data)
>>> dpl.sens1(data["CAM011"])

References

.. [1] Biondi, F. and Qeadan, F. (2008) Inequality in paleorecords. Ecology 89, 1056-1067. .. [2] https:/opendendro.org/dplpy-man/#sens1

sens2¶

Mean sensitivity for a series with a trend (dplR sens2).

Eq. 2 of Biondi and Qeadan (2008): the local two-year denominator of :func:sens1 is replaced by the series mean, so a growth trend does not inflate the statistic. For a single series of n (non-missing) values,

sens2 = sum_i |x_{i+1} - x_i| / (sum(x) - sum(x)/n)
      = sum_i |x_{i+1} - x_i| / ((n - 1) * mean(x))

Parameters:

Name	Type	Description	Default
`data`	`pandas.DataFrame, pandas.Series, 1-D array-like, or str`	A ring-width dataset (years x series), a single series, or a path/URL to a .csv/.rwl file. NA/NaN values are dropped before the calculation.	required

Returns:

Type	Description
`float or Series`	A scalar for a single series, or a Series indexed by series name for a DataFrame / file path. Fewer than two values yields `NaN`.

Examples:

>>> dpl.sens2(data)
>>> dpl.sens2(data["CAM011"])

References

.. [1] Biondi, F. and Qeadan, F. (2008) Inequality in paleorecords. Ecology 89, 1056-1067. .. [2] https:/opendendro.org/dplpy-man/#sens2

glk¶

Gleichlaeufigkeit (sign-agreement) between all pairs of series.

For every pair of series this is the proportion of shared intervals in which the two move in the same direction, counting a flat (no-change) interval as half agreement. A port of dplR's glk (Visser's implementation).

Parameters:

Name	Type	Description	Default
`data`	`DataFrame or str`	A ring-width dataset (years x series), or a path/URL to a .csv/.rwl file.	required
`overlap`	`int`	Minimum number of overlapping growth-change intervals a pair must share; pairs below this are NaN. Must be a single integer >= 3. A warning is issued for values below 50 (matches likely become statistically insignificant).	`50`
`prob`	`bool`	If True, also return a matrix of two-sided p-values.	`True`

Returns:

Type	Description
`dict of pandas.DataFrame`	`glk_mat` (the statistic, diagonal set to 1), `overlap` (the number of overlapping intervals per pair), and, when `prob` is True, `p_mat` (two-sided p-values). All are n x n, indexed by series name.

sgc¶

Synchronous and semi-synchronous growth changes between all pairs.

A decomposition of :func:glk: sgc is the proportion of shared intervals in which two series move in the same direction, and ssgc the proportion in which exactly one is flat. A port of dplR's sgc.

Parameters:

Name	Type	Description	Default
`data`	`DataFrame or str`	A ring-width dataset (years x series), or a path/URL to a .csv/.rwl file.	required
`overlap`	`int`	Minimum overlapping intervals per pair (a single integer >= 3); pairs below this are NaN. A warning is issued below 50.	`50`
`prob`	`bool`	If True, also return a matrix of two-sided p-values (computed from sgc).	`True`

Returns:

Type	Description
`dict of pandas.DataFrame`	`sgc_mat`, `ssgc_mat`, `overlap`, and, when `prob` is True, `p_mat` -- all n x n, indexed by series name. (dplPy uses the `*_mat` keys in both branches; dplR's `prob=False` list names them `sgc`/`ssgc` -- the values are identical.)

sss¶

Subsample signal strength (SSS) as a per-year series.

Extended Summary

For each year, computes how well the subsample of series present in that year represents the full N-series chronology, following Wigley et al. (1984):

SSS(t) = n(t) * (1 + (N - 1) * rbar) / (N * (1 + (n(t) - 1) * rbar))

where N is the total number of trees in the record, rbar is the effective mean interseries correlation (both taken, fixed, from a whole-record dpl.rwi_stats() call), and n(t) is the number of trees (or cores, if no tree IDs are given) present in year t. SSS rises toward 1 as n(t) approaches N.

This mirrors dplR's sss(): the correlation and full sample size are held constant and only the per-year sample depth varies. It is therefore distinct from the running EPS produced by dpl.rwi_stats_running(), which recomputes rbar within each moving window.

No acceptance threshold is applied. As Buras (2017) clarifies, SSS is the statistic Wigley et al. (1984) actually intended for assessing declining reconstruction skill back in time (the frequently cited 0.85 value was an illustrative SSS example, not a rule); whether a given SSS is adequate is a judgement for the analyst.

Parameters:

Name	Type	Description	Default
`rwi_data`	`pandas dataframe`	detrended ring-width indices (e.g. from dpl.detrend()), years as index and series as columns.	required
`ids`	`dict, pandas dataframe, or None`	tree/core structure (see dpl.rwi_stats() / dpl.read_ids()). When given, both N and the per-year sample depth are counted in trees; when None, every series is its own tree and they are counted in cores.	`None`
`corr`	`str`	correlation type used for the underlying rbar ("Spearman" or "Pearson").	`"Spearman"`
`zero_is_missing`	`boolean`	treat exact zeros as missing (applied consistently to the rbar calculation and the per-year sample-depth count).	`True`
`common_interval`	`str, (int, int), or None`	restrict to a common interval before computing SSS: 'series', 'years' or 'both' selects one via dpl.common_interval(), and a (start_year, end_year) pair restricts to that span. rbar, N, n(t) and the returned years are then all taken from that trimmed block -- equivalent to dplR's sss(common.interval(rwi)). None (default) reproduces dplR's plain sss() over the full record.	`None`

Returns:

Name	Type	Description
`result`	`pandas Series named "sss", indexed by year, with the per-year`	subsample signal strength (over the common interval if one was chosen).

Examples:

>>> import dplpy as dpl
>>> rwi = dpl.detrend(dpl.readers("../tests/data/csv/file.csv"), plot=False)
>>> dpl.sss(rwi)

References

.. [1] https://rdrr.io/cran/dplR/man/sss.html .. [2] Wigley, Briffa & Jones (1984), J. Climate Appl. Meteor., 23, 201-213. .. [3] Buras (2017), Dendrochronologia, 44, 130-132.

tree_mean¶

Average each tree's cores into a single tree-level series (dplR treeMean).

Parameters:

Name	Type	Description	Default
`rwl`	`DataFrame`	A ring-width dataset (years x cores/series).	required
`ids`	`DataFrame or dict`	The tree/core structure that says which tree each series belongs to -- the output of :func:`read_ids` (a DataFrame indexed by series name with a `tree` column), or a plain `{series_name: tree_id}` dict. Series are matched to trees by name, so `ids` need not be in column order.	required
`na_rm`	`bool`	If False (dplR's default), a tree's value for a year is NaN unless every one of its cores has a measurement that year. If True, the mean is taken over whatever cores are present (a year is NaN only when the tree has no core at all that year).	`False`

Returns:

Type	Description
`DataFrame`	A ring-width dataset (years x trees). Columns are the unique tree IDs, in order of first appearance among the input columns; the year index is preserved. Feed it straight to :func:`detrend` / :func:`chron`.

Examples:

>>> ids = dpl.read_ids(data)
>>> dpl.tree_mean(data, ids)
>>> dpl.chron(dpl.tree_mean(dpl.detrend(data), ids, na_rm=True))

References

.. [1] https:/opendendro.org/dplpy-man/#tree_mean

common_interval¶

Select a complete common interval from a set of ring-width series.

Extended Summary

Finds the largest block of years and series in which every retained series has a value for every retained year (a complete, gap-free rectangle). This is the interval over which chronology-level statistics such as rbar, EPS and SSS can be computed without missing values, and is the recommended way to handle datasets whose series are unevenly distributed in time. Ported from dplR's common.interval().

Three strategies trade off the number of series against the number of years:

"series" maximises the number of series (cores), at the highest sample depth -- typically a short, deep interval.
"years" maximises the number of years, dropping short series to extend the span -- typically a long, shallow interval.
"both" (default) maximises the number of data cells (series x years), a balance of the two. (Following dplR, this is a specific heuristic rather than the global cell maximum, which "years" can occasionally exceed.)

To use an interval of your own choosing instead, simply slice the frame (e.g. rwi.loc[1800:1980]) before passing it on.

Parameters:

Name	Type	Description	Default
`rwl`	`DataFrame`	ring-width series (raw or detrended), years as the index, series as columns.	required
`type`	`(series, years, both)`	the selection strategy described above.	`"series"`
`make_plot`	`bool`	if True, draw a coverage diagram (each series as a grey span, with the retained common interval overlaid in black). dplR draws this by default; here it is off by default so the function is quiet unless asked.	`False`

Returns:

Type	Description
`DataFrame`	the trimmed frame containing only the retained series and years (a complete rectangle). The full frame is returned unchanged if no trimming is possible/needed.

Examples:

>>> import dplpy as dpl
>>> rwi = dpl.detrend(dpl.readers("../tests/data/csv/co021.csv"), plot=False)
>>> ci = dpl.common_interval(rwi, type="both")
>>> dpl.rwi_stats(ci)                       # rbar/EPS/SNR over the interval

References

.. [1] https://rdrr.io/cran/dplR/man/common.interval.html .. [2] Bunn (2008), Dendrochronologia, 26, 115-124.

bai_out¶

Basal area increment computed from the outside in (dplR bai.out).

Parameters:

Name	Type	Description	Default
`rwl`	`DataFrame`	Year-indexed ring widths (one column per series).	required
`diam`	`DataFrame, dict, or pandas.Series`	Stem diameter for each series, in the same length units as the ring widths. A DataFrame uses a `series` column (or the first column) for the names and a `diam` column (or the second column) for the diameters; a dict/Series maps series name -> diameter. If omitted, each series' radius is taken as the sum of its ring widths (diameter = twice that).	`None`

Returns:

Type	Description
`DataFrame`	BAI in squared length units, same shape/index as `rwl` (NaN where ring width was missing).

bai_in¶

Basal area increment computed from the pith out (dplR bai.in).

Parameters:

Name	Type	Description	Default
`rwl`	`DataFrame`	Year-indexed ring widths (one column per series).	required
`d2pith`	`DataFrame, dict, or pandas.Series`	Distance from the innermost measured ring to the pith, per series, in the same length units as the ring widths. A DataFrame uses a `series` column (or the first column) for the names and a `d2pith` column (or the second column) for the offsets; a dict/Series maps series name -> offset. If omitted, every offset is 0 (the innermost ring is assumed to reach the pith).	`None`

Returns:

Type	Description
`DataFrame`	BAI in squared length units, same shape/index as `rwl` (NaN where ring width was missing).