Statistics & indices

Descriptive statistics, agreement measures, sample-depth signal strength, and derived indices.

stats

Summary statistics for each series in a ring-width dataset.

Reports, for each series, the columns 'first', 'last', 'year', 'mean', 'median', 'stdev', 'skew', 'kurtosis', 'gini', and 'ar1' (dplR's rwl.stats()).

Parameters:

Name	Type	Description	Default
inp	str or pandas dataframe	a data file (.CSV or .RWL) or a pandas dataframe imported from dpl.readers(). Series with no measured values are dropped with a warning.	required

Returns:

Name	Type	Description
data	pandas dataframe	one row per series. 'year' is the SPAN first..last in years (as dplR's rwl.stats), not the number of measured rings; the two differ for a series with interior gaps (see dpl.samp_stats() 'seg' for ring counts).

Examples:

>>> dpl.stats(data)

References

.. [1] https://opendendro.github.io/dplPy/reference/statistics/#stats

samp_stats

Per-year sample-tracking statistics for a ring-width dataset.

For every year in the dataset this reports how many series are present and, across just those present series, their mean length and mean cambial age -- the num / seg / age columns of an ARSTAN "tabs" file.

Parameters:

Name	Type	Description	Default
data	pandas dataframe	ring widths, years as the index and series as columns (e.g. from dpl.readers()). Raw widths or detrended indices give the same result -- only the pattern of present/absent values is used.	required

Returns:

Name	Type	Description
out	pandas dataframe indexed by year with columns	samp_depth : number of series present that year (the sample depth; ARSTAN's num). An integer count. seg : mean segment length of the series present that year, where a series' segment length is its number of measured rings. age : mean cambial age that year, where a series' cambial age is the number of years since its first ring (year - first_year + 1). Years with no series present get samp_depth 0 and NaN for seg / age.

Notes

seg counts a series' measured rings; age counts years since its first ring. The two differ only for a series with an interior gap (an unmeasured year inside its span), for which the tree still ages but the ring is not counted.

Examples:

>>> import dplpy as dpl
>>> data = dpl.readers("../tests/data/csv/file.csv")
>>> dpl.samp_stats(data)

rwi_stats

Whole-record chronology signal statistics (rbar, EPS, SNR).

Computes the mean interseries correlation (rbar), expressed population signal (EPS), and signal-to-noise ratio (SNR) over the entire record as a single-row summary. The non-running counterpart of rwi_stats_running(), which it calls with running_window=False; see that function for full parameter and statistic descriptions.

Parameters:

Name	Type	Description	Default
rwi_data	pandas dataframe	detrended ring-width indices (e.g. from dpl.detrend()).	required
ids	dict, pandas dataframe, or None	tree/core structure; see rwi_stats_running().	None
period	str	"max" (pairwise-complete overlaps) or "common" (complete rows only); case-insensitive.	"max"
corr	str	correlation type, "spearman" or "pearson" (case-insensitive).	"spearman"
prewhiten	boolean	prewhiten each series with an AR model before computing correlations (see rwi_stats_running()).	False
min_corr_overlap	int or None	minimum number of overlapping years for a series pair to contribute.	None
zero_is_missing	boolean	treat exact zeros as missing values.	True
round_decimals	int or None	decimal places to round the statistic columns to.	3
common_interval	str, (int, int), or None	restrict to a common interval first (see rwi_stats_running()).	None
ar_max		prewhitening controls (see rwi_stats_running()).	10
ar_method		prewhitening controls (see rwi_stats_running()).	10
first_aic_min		prewhitening controls (see rwi_stats_running()).	10

Returns:

Name	Type	Description
result	a one-row pandas dataframe of chronology signal statistics.

Examples:

>>> import dplpy as dpl
>>> rwi = dpl.detrend(dpl.readers("../tests/data/csv/file.csv"), plot=False)
>>> dpl.rwi_stats(rwi)

References

.. [1] https://rdrr.io/cran/dplR/man/rwi.stats.running.html .. [2] Wigley, Briffa & Jones (1984), J. Climate Appl. Meteor., 23, 201-213.

rwi_stats_running

Running (moving-window) chronology signal statistics (rbar, EPS, SNR).

For each moving window along the chronology, computes the mean interseries correlation (rbar) and, from it, the expressed population signal (EPS) and signal-to-noise ratio (SNR) of Wigley et al. (1984), after dplR's rwi.stats.running(); rwi_stats() is the whole-record special case (running_window=False).

When tree/core IDs are supplied (see ids), correlations are split into within-tree (rbar_wt) and between-tree (rbar_bt) components, and the effective signal rbar_eff accounts for averaging multiple cores per tree via an effective cores-per-tree factor c_eff. c_eff is ARSTAN-style running effective replication: the cores of each tree present in the window, not counting a core whose within-tree correlations all fail min_corr_overlap. This differs from dplR's back-calculation of the core count from the number of valid within-tree correlations (0.5 + sqrt(0.25 + 2 n_wt)); the two agree for the whole record when every within-tree pair is valid and trees have at most two cores, and can differ inside running windows. Without IDs, every series is its own tree and rbar_eff == rbar_bt == rbar_tot.

Windows start at the first year and advance by window_length - window_overlap; a trailing remainder shorter than a window is not covered (dplR instead chooses the start that maximises coverage).

EPS = n * rbar_eff / ((n - 1) * rbar_eff + 1) and SNR = n * rbar_eff / (1 - rbar_eff), where n is the number of trees contributing to the between-tree correlations.

NOTE: no acceptance threshold (e.g. the frequently cited EPS > 0.85) is applied or implied here; that threshold was an illustrative example for SSS (see dpl.sss()), not a general criterion for EPS.

Parameters:

Name	Type	Description	Default
rwi_data	pandas dataframe	detrended ring-width indices (e.g. from dpl.detrend()), with years as the index and series as columns.	required
ids	dict, pandas dataframe, or None	tree/core structure used to split within- vs between-tree correlations. Either a dict mapping {series_name: tree_id}, a dataframe with a 'tree' column indexed by series name (such as produced by dpl.read_ids()), or None to treat every series as its own tree (one core per tree).	None
period	str	"max" uses each pair's own overlapping years (pairwise-complete); "common" restricts each window to years present in every series that has data in the window (case-insensitive).	"max"
corr	str	correlation type, "spearman" or "pearson" (case-insensitive).	"spearman"
prewhiten	boolean	prewhiten each series with an AR model before computing correlations, using the package-standard autoreg._prewhiten() (Yule-Walker, order by the first local AIC minimum up to ar_max, ARSTAN backcast initials so no years are lost). NOTE: R's ar() / dplR take the global AIC minimum up to floor(10*log10(n)); pass first_aic_min=False for that rule.	False
running_window	boolean	if False, a single window spanning the whole record is used (this is what rwi_stats() invokes).	True
window_length	int or None	length of each moving window in years (None: min(50, n_years)).	50
window_overlap	int or None	number of years successive windows overlap; defaults to window_length // 2.	None
min_corr_overlap	int or None	minimum number of overlapping years required for a series pair to contribute a correlation; defaults to min(30, window_length).	None
zero_is_missing	boolean	treat exact zeros as missing values (a zero RWI is treated as absent).	True
round_decimals	int or None	decimal places to round the real-valued statistic columns to; None leaves them unrounded.	3
common_interval	str, (int, int), or None	restrict the data to a common interval before computing statistics. 'series', 'years' or 'both' selects a complete overlap-only rectangle via dpl.common_interval() (maximising cores, years, or cells); a (start_year, end_year) pair restricts to that inclusive span. None (default) uses the full record. Recommended for unevenly-distributed data, where period='common' on the full record can be empty.	None
ar_max	int	maximum AR order for prewhiten=True.	10
ar_method	(yw, burg, ols)	AR estimator for prewhiten=True (case-insensitive).	"yw"
first_aic_min	boolean	AR order rule for prewhiten=True: first local AIC minimum (True) or global minimum (False, R's ar()).	True

Returns:

Name	Type	Description
result	pandas dataframe with one row per window (or a single row if	running_window=False). Columns: start_year, mid_year, end_year (running only), n_cores, n_trees, n, n_tot, n_wt, n_bt, rbar_tot, rbar_wt, rbar_bt, c_eff, rbar_eff, eps, snr.

Examples:

>>> import dplpy as dpl
>>> rwi = dpl.detrend(dpl.readers("../tests/data/csv/file.csv"), plot=False)
>>> dpl.rwi_stats_running(rwi, window_length=60, window_overlap=30)

References

.. [1] https://rdrr.io/cran/dplR/man/rwi.stats.running.html .. [2] Wigley, Briffa & Jones (1984), J. Climate Appl. Meteor., 23, 201-213. .. [3] Buras (2017), Dendrochronologia, 44, 130-132. .. [4] https://opendendro.github.io/dplPy/reference/statistics/#rwi_stats_running

sens1

Mean sensitivity (dplR sens1).

The standard measure of year-to-year variability in dendrochronology (Douglass; Eq. 1 of Biondi and Qeadan 2008), typically computed on detrended series. For a single series of n (non-missing) values,

sens1 = (2 / (n - 1)) * sum_i |x_i - x_{i-1}| / (x_i + x_{i-1})

where a 0/0 term from an x_i + x_{i-1} == 0 pair is skipped while n - 1 is unchanged.

Parameters:

Name	Type	Description	Default
data	pandas.DataFrame, pandas.Series, 1-D array-like, or str	A ring-width dataset (years x series), a single series, or a path/URL to a .csv/.rwl file. NA/NaN values are dropped before the calculation.	required

Returns:

Type	Description
float or Series	For a single series, the scalar mean sensitivity. For a DataFrame (or a file path, which is read into one), a Series of the statistic indexed by series name. A series with fewer than two values yields NaN.

Examples:

>>> dpl.sens1(data)
>>> dpl.sens1(data["CAM011"])

References

.. [1] Biondi, F. and Qeadan, F. (2008) Inequality in paleorecords. Ecology 89, 1056-1067. .. [2] https://opendendro.github.io/dplPy/reference/statistics/#sens1

sens2

Mean sensitivity for a series with a trend (dplR sens2).

Eq. 2 of Biondi and Qeadan (2008): the local two-year denominator of sens1() is replaced by the series mean, so a growth trend does not inflate the statistic. For a single series of n (non-missing) values,

sens2 = sum_i |x_{i+1} - x_i| / (sum(x) - sum(x)/n)
      = sum_i |x_{i+1} - x_i| / ((n - 1) * mean(x))

A series whose values sum to zero has no defined sens2 and yields NaN.

Parameters:

Name	Type	Description	Default
data	pandas.DataFrame, pandas.Series, 1-D array-like, or str	A ring-width dataset (years x series), a single series, or a path/URL to a .csv/.rwl file. NA/NaN values are dropped before the calculation.	required

Returns:

Type	Description
float or Series	A scalar for a single series, or a Series indexed by series name for a DataFrame / file path. Fewer than two values yields NaN.

Examples:

>>> dpl.sens2(data)
>>> dpl.sens2(data["CAM011"])

References

.. [1] Biondi, F. and Qeadan, F. (2008) Inequality in paleorecords. Ecology 89, 1056-1067. .. [2] https://opendendro.github.io/dplPy/reference/statistics/#sens2

glk

Gleichlaeufigkeit (sign-agreement) between all pairs of series.

For every pair of series this is the proportion of shared intervals in which the two move in the same direction, counting a flat (no-change) interval as half agreement. A port of dplR's glk (Visser's implementation).

Parameters:

Name	Type	Description	Default
data	DataFrame or str	A ring-width dataset (years x series), or a path/URL to a .csv/.rwl file.	required
overlap	int	Minimum number of overlapping growth-change intervals a pair must share; pairs below this are NaN. Must be a single integer >= 3. A warning is issued for values below 50 (matches likely become statistically insignificant).	50
prob	bool	If True, also return a matrix of two-sided p-values.	True

Returns:

Type	Description
dict of pandas.DataFrame	glk_mat (the statistic, diagonal set to 1), overlap (the number of overlapping intervals per pair), and, when prob is True, p_mat (two-sided p-values). All are n x n, indexed by series name.

See Also

sgc : the synchronous / semi-synchronous decomposition (glk = sgc + ssgc/2).

Examples:

>>> res = dpl.glk(data)
>>> res["glk_mat"]

References

.. [1] Schweingruber, F.H. (1988) Tree Rings. Kluwer. .. [2] Visser, R.M. (2021) On the similarity of tree-ring patterns: Journal of Archaeological Science 125. .. [3] https://opendendro.github.io/dplPy/reference/statistics/#glk

sgc

Synchronous and semi-synchronous growth changes between all pairs.

A decomposition of glk(): sgc is the proportion of shared intervals in which two series move in the same direction, and ssgc the proportion in which exactly one is flat. A port of dplR's sgc.

Parameters:

Name	Type	Description	Default
data	DataFrame or str	A ring-width dataset (years x series), or a path/URL to a .csv/.rwl file.	required
overlap	int	Minimum overlapping intervals per pair (a single integer >= 3); pairs below this are NaN. A warning is issued below 50.	50
prob	bool	If True, also return a matrix of two-sided p-values (computed from sgc).	True

Returns:

Type	Description
dict of pandas.DataFrame	sgc_mat, ssgc_mat, overlap, and, when prob is True, p_mat -- all n x n, indexed by series name. (dplPy uses the *_mat keys in both branches; dplR's prob=False list names them sgc/ssgc -- the values are identical.)

See Also

glk : the combined statistic, glk = sgc + ssgc/2.

Examples:

>>> res = dpl.sgc(data)
>>> res["sgc_mat"], res["ssgc_mat"]

References

.. [1] Visser, R.M. (2021) On the similarity of tree-ring patterns: Journal of Archaeological Science 125. .. [2] https://opendendro.github.io/dplPy/reference/statistics/#sgc

sss

Subsample signal strength (SSS) as a per-year series.

For each year, computes how well the subsample of series present that year represents the full N-series chronology (Wigley et al. 1984):

SSS(t) = n(t) * (1 + (N - 1) * rbar) / (N * (1 + (n(t) - 1) * rbar))

where N is the total number of trees in the record, rbar is the effective mean interseries correlation (both fixed, from a whole-record dpl.rwi_stats() call), and n(t) is the number of trees (or cores, if no tree IDs are given) present in year t. Mirrors dplR's sss(), holding rbar and N constant while only the per-year sample depth varies -- distinct from the running EPS of dpl.rwi_stats_running(), which recomputes rbar in each window.

No acceptance threshold is applied; SSS is the statistic Wigley et al. (1984) intended for assessing declining reconstruction skill back in time (the frequently cited 0.85 value was an illustrative example, not a rule).

Parameters:

Name	Type	Description	Default
rwi_data	pandas dataframe	detrended ring-width indices (e.g. from dpl.detrend()), years as index and series as columns.	required
ids	dict, pandas dataframe, or None	tree/core structure (see dpl.rwi_stats() / dpl.read_ids()). When given, both N and the per-year sample depth are counted in trees; when None, every series is its own tree and they are counted in cores.	None
corr	str	correlation type used for the underlying rbar ("spearman" or "pearson", case-insensitive).	"spearman"
zero_is_missing	boolean	treat exact zeros as missing (applied consistently to the rbar calculation and the per-year sample-depth count).	True
common_interval	str, (int, int), or None	restrict to a common interval before computing SSS: 'series', 'years' or 'both' selects one via dpl.common_interval(), and a (start_year, end_year) pair restricts to that span. rbar, N, n(t) and the returned years are then all taken from that trimmed block. None (default) uses the full record.	None

Returns:

Name	Type	Description
result	pandas Series named "sss", indexed by year, with the per-year subsample	signal strength (over the common interval if one was chosen).

Examples:

>>> import dplpy as dpl
>>> rwi = dpl.detrend(dpl.readers("../tests/data/csv/file.csv"), plot=False)
>>> dpl.sss(rwi)

References

.. [1] https://rdrr.io/cran/dplR/man/sss.html .. [2] Wigley, Briffa & Jones (1984), J. Climate Appl. Meteor., 23, 201-213. .. [3] Buras (2017), Dendrochronologia, 44, 130-132. .. [4] https://opendendro.github.io/dplPy/reference/statistics/#sss

tree_mean

Average each tree's cores into a single tree-level series (dplR treeMean).

Parameters:

Name	Type	Description	Default
rwl	DataFrame	A ring-width dataset (years x cores/series).	required
ids	DataFrame or dict	The tree/core structure that says which tree each series belongs to -- the output of read_ids() (a DataFrame indexed by series name with a tree column), or a plain {series_name: tree_id} dict. Series are matched to trees by name, so ids need not be in column order.	required
na_rm	bool	If False (dplR's default), a tree's value for a year is NaN unless every one of its cores has a measurement that year. If True, the mean is taken over whatever cores are present (a year is NaN only when the tree has no core at all that year).	False

Returns:

Type	Description
DataFrame	A ring-width dataset (years x trees). Columns are the unique tree IDs, in order of first appearance among the input columns; the year index is preserved, and the input's provenance attrs are carried over (the output is still raw widths, kind="rwl"). Feed it straight to detrend() / chron().

Examples:

>>> ids = dpl.read_ids(data)
>>> dpl.tree_mean(data, ids)
>>> dpl.chron(dpl.tree_mean(dpl.detrend(data), ids, na_rm=True))

References

.. [1] https://opendendro.github.io/dplPy/reference/statistics/#tree_mean

common_interval

Select a complete common interval from a set of ring-width series.

Finds the largest block of years and series in which every retained series has a value for every retained year (a complete, gap-free rectangle) -- the interval over which chronology statistics such as rbar, EPS and SSS can be computed without missing values. Ported from dplR's common.interval().

Three strategies trade off the number of series against the number of years:

• "series" maximises the number of series (cores) at the highest sample depth -- typically a short, deep interval.
• "years" maximises the number of years, dropping short series to extend the span -- typically a long, shallow interval.
• "both" (default) maximises the number of data cells (series x years), a balance of the two.

To use an interval of your own choosing, slice the frame (e.g. rwi.loc[1800:1980]) before passing it on.

Parameters:

Name	Type	Description	Default
rwl	DataFrame	ring-width series (raw or detrended), years as the index, series as columns.	required
type	(series, years, both)	the selection strategy described above (case-insensitive).	"series"
plot	bool	if True, draw a coverage diagram (each series as a grey span, with the retained common interval overlaid in black).	False

Returns:

Type	Description
DataFrame	the trimmed frame containing only the retained series and years (a complete rectangle), carrying the input's attrs. The full frame is returned unchanged if no trimming is possible/needed.

Examples:

>>> import dplpy as dpl
>>> rwi = dpl.detrend(dpl.readers("../tests/data/csv/co021.csv"), plot=False)
>>> ci = dpl.common_interval(rwi, type="both")
>>> dpl.rwi_stats(ci)

References

.. [1] https://rdrr.io/cran/dplR/man/common.interval.html .. [2] Bunn (2008), Dendrochronologia, 26, 115-124.

bai_out

Basal area increment computed from the outside in (dplR bai.out).

Parameters:

Name	Type	Description	Default
rwl	DataFrame	Year-indexed ring widths (one column per series).	required
diam	DataFrame, dict, or pandas.Series	Stem diameter for each series, in the same length units as the ring widths. A DataFrame uses a series column (or the first column) for the names and a diam column (or the second column) for the diameters; a dict/Series maps series name -> diameter. If omitted, each series' radius is taken as the sum of its ring widths (diameter = twice that).	None

Returns:

Type	Description
DataFrame	BAI in squared length units, same shape/index as rwl (NaN where ring width was missing), tagged attrs["kind"] = "bai". A warning names any series whose supplied diam is smaller than twice its summed ring widths: the radius then goes negative inside the stem and the innermost BAI values are negative (dplR behaves the same, silently).

bai_in

Basal area increment computed from the pith out (dplR bai.in).

Parameters:

Name	Type	Description	Default
rwl	DataFrame	Year-indexed ring widths (one column per series).	required
d2pith	DataFrame, dict, or pandas.Series	Distance from the innermost measured ring to the pith, per series, in the same length units as the ring widths. A DataFrame uses a series column (or the first column) for the names and a d2pith column (or the second column) for the offsets; a dict/Series maps series name -> offset. If omitted, every offset is 0 (the innermost ring is assumed to reach the pith).	None

Returns:

Type	Description
DataFrame	BAI in squared length units, same shape/index as rwl (NaN where ring width was missing), tagged attrs["kind"] = "bai".