Chronology

Averaging detrended series into a site chronology, with optional autoregressive modelling and variance stabilization. See the Building chronologies guide.

chron

Creates a mean value chronology for a dataset of tree-ring widths.

Extended Summary

Creates a mean value chronology for a dataset, typically the ring width indices of a detrended series. Determines whether to find means using Tukey's bi-weight robust mean (default 'True'), whether to prewhiten data by fitting to an AR model (default 'False'), and whether to plot the results of the chronology (default 'True'). When prewhitening, the AR-model order can be controlled with max_lag and aic (mirroring dplR's chron(), which forwards these to ar() as order.max and aic).

Parameters:

Name	Type	Description	Default
rwi_data	pandas dataframe	a pandas dataframe imported from dpl.readers()	required
biweight	boolean	use Tukey's bi-weight robust mean	True
prewhiten	boolean	run pre-whitening on the time series before building chronology	False
plot	boolean	plot the results	True
max_lag	int	when prewhiten is True, the maximum AR order considered. The default of 10 follows ARSTAN's convention (and matches chron_ars). dplR's own chron() instead forwards to ar() whose default order.max is floor(10*log10(n)) (~27 for a 500-yr series), so to reproduce dplR's chron() exactly, pass max_lag=floor(10*log10(n)).	10
aic	boolean	when prewhiten is True, select the AR order by AIC up to max_lag (dplR's ar() aic=TRUE). If False, fit a fixed AR of order max_lag.	True
ar_method	(yw, ols)	AR estimator used for prewhitening. "yw" (Yule-Walker) matches R's ar() default and reproduces dplR's residual chronology at the same order; "ols" (statsmodels AutoReg, conditional least squares) matches R's ar(method="ols"). chron_ars() also uses Yule-Walker.	"yw"
first_aic_min	boolean	AR order-selection rule when aic is True. False takes the global AIC minimum (R's ar() behaviour, so chron() reproduces dplR's chron()); True takes the first local AIC minimum -- ARSTAN's rule, shared with chron_ars() -- which is more parsimonious (it rarely selects an order above ~7, so max_lag seldom binds) at the cost of a little AIC.	False
stabilize	(None, rbar, spline, both)	If not None, also variance-stabilize each chronology and return the stabilized versions in extra *_vsc columns (see Notes). This is the ARSTAN workflow, where variance stabilization (menu option isb) is applied to the finished chronologies. The method families are as in :func:stabilize_chron: "rbar" (Osborn/Frank N_eff scaling), "spline" (ARSTAN stabit spline), or "both" (rbar then a 67% spline; ARSTAN isb=2).	None
stabilize_kwargs	dict	Extra keyword arguments forwarded to :func:stabilize_chron when stabilize is set (e.g. {"rbar_mode": "constant"}, {"win_length": 30}, {"spline_period": 100}). The chronology, column, source RWI matrix, and sample depth are supplied automatically and may not be overridden here. Defaults (running rbar, 67% spline) match Frank (2006) / ARSTAN.	None

Returns:

Name	Type	Description
data	pandas dataframe	a mean-value chronology indexed by year. Columns: std (the standard chronology), res (the residual/AR-prewhitened chronology; only when prewhiten is True), and samp_depth (the sample depth). These names match chron_ars() and mirror dplR's std/res/samp.depth. When stabilize is set, a stabilized std_vsc (and res_vsc when prewhiten is True) column is added next to each base chronology.

Notes

Variance stabilization (stabilize). Following ARSTAN's per-chronology rbar convention, the standard chronology is stabilized with the rbar of the standard (detrended) RWI matrix, and the residual chronology with the rbar of the residual (prewhitened) matrix. Both restandardize to the input chronology's full period (see :func:stabilize_chron). For the ARSTAN (re-reddened) chronology, use :func:chron_ars with stabilize=.

Examples:

>>> import dplpy as dpl 
>>> data = dpl.readers("../tests/data/csv/file.csv")
>>> rwi_data = dpl.detrend(data)
>>> dpl.chron(rwi_data)
>>> dpl.chron(rwi_data, prewhiten=True)
>>> chron_data = dpl.chron(rwi_data, biweight=False, plot=False)

References

.. [1] https:/opendendro.org/dplpy-man/#chron

chron_stabilized

Variance Stabilization functions

Extended Summary

Builds a variance stabilized mean-value chronology from a pandas dataframe of detrended ring widths, by multiplying the chronology with the square root of the effective independent sample size, Neff, defined as

Neff = n(t) / 1+(n(t)-1)rbar(t)

where n(t) is the number of series at time t, and rbar(t) is the
mean pairwise correlation between all series (not "interseries
correlation" -- see dpl.interseries_corr() -- which correlates each
series against a composite chronology of the others, rather than
against each other series individually).

In the limiting cases, when the rbar is zero or unity, Neff obtains values of the true sample size and unity, respectively. Neff is calculated over different segments of the data of length win_length, and only series with at least min_seg_ratio of valid values in the segment are considered.

This is a port of dplR's chron.stabilized() (see reference below); the intent is to match that implementation's behavior as closely as possible, including its two independent window-length recommendations and its use of an unfiltered overall rbar constant (as opposed to the windowed rbar, which does apply the min_seg_ratio overlap filter).

Parameters:

Name	Type	Description	Default
rwi_data	DataFrame	a Pandas dataset representing detrended tree rings/widths.	required
win_length	int	an integer for specifying the window lengths where rbar values will be calculated.	50
min_seg_ratio	float	the minimum ratio of non-NA values to the window length for a series to be considered in an Neff calculation. Defaults to exactly ⅓ to match dplR's chron.stabilized(), which drops overlaps below win_length/3.	1/3
biweight	boolean	flag indicating whether or not to use Tukey's bi-weight robust mean when calculating the mean-value chronology	True
running_rbar	boolean	flag indicating whether or not to return the running rbar values as part of chronology output (only for method='running_rbar').	False
method	str	the variance-stabilization method: 'running_rbar' : the running-window-rbar effective-signal adjustment (Frank et al. 2006 "RUNNINGr"), a port of dplR's chron.stabilized(). The default; uses win_length and min_seg_ratio. 'briffa' (alias 'mean_rbar') : the same effective-sample-size adjustment but with a single, time-constant rbar computed over all pairwise series overlaps longer than 20 years (Osborn et al. 1997 constant-rbar; Frank et al. 2006 "MEANr"; ARSTAN's Briffa option). Corrects for changing sample size ONLY. 'spline' : ARSTAN's ad-hoc spline stabilization. Removes any time trend in the ABSOLUTE departures of the chronology with a smoothing spline, so it can stabilize variance even when the heteroscedasticity is NOT a sample-size effect. NOTE (per ARSTAN and Osborn et al. 1997): this is strictly ad hoc and can remove real low-frequency variance -- use with care. Stiffness set by spline_nyrs.	'running_rbar'
spline_nyrs	(int, float or None)	stiffness for method='spline'. An int is a fixed spline wavelength in years (ARSTAN's fixed-n cutoff); a float in (0, 1) is a fraction of the chronology length (ARSTAN's %-n cutoff). None uses 0.5 (half the length). Ignored by the rbar methods.	None

Returns:

Name	Type	Description
stabilized_chron	a pandas dataframe of a variance-stabilized mean-value	chronology, indexed by year. Columns: vsc (the variance-stabilized chronology, matching dplR's naming), Running rbar (only when running_rbar=True), and samp_depth.

Examples:

>>> import dplpy as dpl 
>>> data = dpl.readers("../tests/data/csv/file.csv")
>>> dpl.chron_stabilized(data, win_length=60, min_seg_ratio=0.4) -> returns mean
                                        value chronology with stabilized
                                        variance.
References
----------
.. [1] https://rdrr.io/cran/dplR/man/chron.stabilized.html

stabilize_chron

Variance-stabilize an already-built chronology.

Unlike :func:chron_stabilized (a port of dplR's chron.stabilized(), which takes the RWI matrix and builds its own standard mean-value chronology internally), this operates on a chronology you already built with :func:chron or :func:chron_ars -- standard, residual, or ARSTAN -- which is how ARSTAN itself is organized (variance stabilization is a post- construction step, menu option isb).

Parameters:

Name	Type	Description	Default
chron	Series or DataFrame	The chronology to stabilize. A Series is used directly; a DataFrame (e.g. the output of chron()/chron_ars()) uses column and, if present, its samp_depth column.	required
method	(rbar, spline, both)	Required, explicit (the families do different things): "rbar" -- Osborn (1997) / Frank (2006) N_eff scaling; corrects variance driven by changing sample size (and, with running rbar, changing interseries correlation). Needs sample depth + rbar (see rwi / the precomputed inputs). "spline" -- ARSTAN's ad-hoc absolute-departure spline (stabit): flattens all time-varying variance whatever the cause; needs only the chronology. Can remove real low-frequency variance (Osborn 1997). "both" -- ARSTAN isb=2 / Frank: "rbar" then a 67% "spline", in sequence.	"rbar"
column	str	Which column to stabilize when chron is a DataFrame.	"std"
rwi	DataFrame	The source ring-width-index matrix for the chronology (for the "rbar" family). When given, rbar, the running rbar, and sample depth are derived from it. For a residual or ARSTAN chronology this should be the residual (prewhitened) matrix -- following ARSTAN, which uses the residual rbar for the ARSTAN chronology.	None
rbar	float	Precomputed scalar rbar (for rbar_mode="constant" without rwi).	None
running_rbar	array - like	Precomputed per-year running rbar (for rbar_mode="running" without rwi).	None
samp_depth	array - like	Per-year sample depth. Taken from the DataFrame's samp_depth column or from rwi when not given.	None
rbar_mode	(running, constant)	Running rbar (Frank RUNNINGr; the widely used default) or a single time-constant rbar (Osborn/ARSTAN Briffa MEANr).	"running"
win_length	(int, float)	Running-rbar window length and minimum overlap ratio (as in chron_stabilized).	50
min_seg_ratio	(int, float)	Running-rbar window length and minimum overlap ratio (as in chron_stabilized).	50
spline_period	(int, float or None)	Spline stiffness for the "spline"/"both" families. None = the 67% spline (floor(0.67*n) years, ARSTAN's default for the combined method); an int > 1 is a fixed wavelength in years; a float in (0, 1) is a fraction of length; a negative value is a percent spline.	None
f	float	Spline frequency response (50% amplitude cutoff).	0.5
clip_negative	bool	Clamp the stabilized chronology at >= 0 (ARSTAN behavior).	True
return_info	bool	Also return a dict of diagnostics (rbar used, N_eff, sample depth).	False

Returns:

Type	Description
DataFrame	Indexed by year, with the stabilized chronology in column vsc and, when known, samp_depth. If return_info is True, returns (DataFrame, info_dict).

Notes

Both families restandardize to the input chronology's full-period mean and SD. This differs from ARSTAN's stabbm, which rescales to a user-chosen reference period; the full-period convention keeps the stabilized chronology on the same overall scale as the input while removing the time-varying variance.

For the ARSTAN chronology, ARSTAN stabilizes the residual chronology and then re-reddens, so faithful ARSTAN stabilization of the ARSTAN chronology is produced by chron_ars(stabilize=...) (which does it in that order), not by calling this function on an already re-reddened series.

References

Osborn, Briffa & Jones (1997) Dendrochronologia 15, 89-99. Frank, Esper & Cook (2006) TRACE 4, 56-66.

chron_ars

Build ARSTAN standard, residual, and re-reddened chronologies.

Extended Summary

Produces the three ARSTAN chronologies of Cook (1985) from a set of detrended ring-width indices:

• std: the standard chronology -- the (robust) mean of the RWI series, with no autoregressive modeling.
• res: the residual chronology -- each series is prewhitened with a pooled AR(p) model, the prewhitened series are averaged, and that mean is prewhitened once more to order p, yielding a near-white chronology. With backcast=True (the default) each prewhitening step backcasts its initial values as ARSTAN does, so no years are lost to the AR model (see Notes).
• ars: the ARSTAN chronology -- the common red-noise persistence (the pooled AR model) is reintroduced by "re-reddening" (postAR). How this is done is controlled by ars_method (see Notes): the default "arstan" re-reddens the mean of the prewhitened series (average then re-redden, as in Ed Cook's ARSTAN); "dplr" re-reddens each series and then averages (matching dplR's chron.ars).

The AR order p is chosen from a pooled autocovariance accumulated across all series and lags (Cook's pooled-AR approach), converted to AR coefficients (Durbin-Levinson) and selected by AIC.

This is a port of dplR's chron.ars(). The pooled-AR accumulation -- the step the dplR authors flagged as a "time killer" -- is vectorized with numpy here (a per-series-pair loop of vectorized lag dot-products, replacing dplR's triple loop with per-iteration cbind/rowSums), while following dplR/FORTRAN's behavior, including its practice of compressing each series pair to its common overlap and then lagging by position. The re-reddening (postAR) forward filter is applied with scipy.signal.lfilter.

Parameters:

Name	Type	Description	Default
rwi_data	pandas dataframe	detrended ring-width indices (e.g. from dpl.detrend()), with years as the index and series as columns.	required
biweight	boolean	if True, aggregate series with Tukey's biweight robust mean (tbrm); if False, use the arithmetic mean.	True
max_lag	int	maximum AR lag considered when selecting the pooled AR order.	10
first_aic_min	boolean	if True, select the order at the first local AIC minimum (dplR's default); if False, select the global AIC minimum.	True
verbose	boolean	if True, print the pooled ACF, AR coefficients, AIC, and selected order.	True
prewhiten_method	str	"ar.yw" (Yule-Walker, the default and the numerically exact match to dplR) or "arima.CSS-ML" (matches dplR within tolerance).	"ar.yw"
ars_method	str	how the ARSTAN (re-reddened) chronology is constructed (see Notes): "arstan" re-reddens the mean of the prewhitened series (average then re-redden -- Ed Cook's ARSTAN order); "dplr" re-reddens each series and then averages (dplR's chron.ars order). The two agree only when every series spans the full period; under staggered sample depth they differ.	"arstan"
backcast	boolean	if True, backcast the initial values of each Yule-Walker prewhitening step (as ARSTAN's bckcst/albino do), so residuals are defined for every year and no leading years are lost to the AR model. If False, the first p residuals of each series (and of the re-prewhitened mean) are set to NaN, reproducing dplR's chron.ars. See Notes. Applies to the "ar.yw" path; the "arima.CSS-ML" path already returns full-length residuals from its state-space filter.	True
stabilize	(None, rbar, spline, both)	If not None, also variance-stabilize each chronology and return the stabilized versions in extra *_vsc columns (see Notes). This mirrors ARSTAN, where variance stabilization (menu option isb) is applied to the finished chronologies. The method families are as in :func:stabilize_chron: "rbar" (Osborn/Frank N_eff scaling), "spline" (ARSTAN stabit spline), or "both" (rbar then a 67% spline; ARSTAN isb=2).	None
stabilize_kwargs	dict	Extra keyword arguments forwarded to :func:stabilize_chron when stabilize is set (e.g. {"rbar_mode": "constant"}). The chronology, column, source matrix, and sample depth are supplied automatically and may not be overridden here.	None

Returns:

Name	Type	Description
out	pandas dataframe indexed by year with columns 'std', 'res', 'ars',	and 'samp_depth'. When stabilize is set, a stabilized std_vsc, res_vsc, and ars_vsc column is added next to each base chronology.

Notes

ARSTAN chronology construction (ars_method). The re-reddening and averaging steps do not commute when sample depth changes through time, so there are two distinct ways to build the ARSTAN chronology:

• "arstan" (default): ars = postAR(mean(prewhitened series)) -- average the prewhitened series first, then re-redden that single mean series once. This reproduces Ed Cook's ARSTAN FORTRAN, which re-reddens the residual chronology directly (commie1 applied to the residual chronology). dplPy defaults to this to be faithful to ARSTAN, from which chron_ars takes its DNA.
• "dplr": ars = mean(postAR(prewhitened series_i)) -- re-redden each prewhitened series and then average. This matches dplR's chron.ars and was dplPy's behavior before this option existed.

The std and res chronologies, the pooled AR order, and the common-pooled-order prewhitening are identical for both settings; only the ars column changes. Use ars_method="dplr" to reproduce dplR exactly.

Backcasting (backcast). An AR(p) filter has no data for its p lagged terms at the start of a series, so ordinarily the first p residuals are undefined. ARSTAN avoids this by backcasting: it synthesizes p pre-sample values by running the AR model in reverse (subroutine bckcst, used by albino when prewhitening and commie when re-reddening), so the filter can produce output for every year. dplPy does the same at both prewhitening stages -- the per-series prewhitening and the re-prewhitening of the mean -- when backcast=True (the default), so the residual and ARSTAN chronologies are full length (no leading NaN) and match ARSTAN's construction more closely: a young series' first p residuals are now included in the robust mean rather than dropped, exactly as ARSTAN includes them. With backcast=False the first p residuals are set to NaN, reproducing dplR's chron.ars. The re-reddening step (postAR) always backcasts, as in ARSTAN. The backcast values are model-based estimates, so treat the first p years as slightly less certain than the interior.

Variance stabilization (stabilize). Following ARSTAN's per-chronology rbar convention, std_vsc uses the rbar of the standard (detrended) RWI matrix, and both res_vsc and the ARSTAN chronology use the rbar of the residual (prewhitened) matrix. The ARSTAN column is stabilized in the ARSTAN order: with ars_method="arstan" the residual-style mean of the prewhitened series is stabilized before re-reddening (ARSTAN stabilizes the residual chronology, then commie1), so ars_vsc stays consistent with the unstabilized ars. With ars_method="dplr" the ARSTAN chronology is already built by re-reddening each series, so ars_vsc is stabilized post-hoc on the reddened chronology (not the ARSTAN order). All columns restandardize to their own full period (see :func:stabilize_chron).

Examples:

>>> import dplpy as dpl
>>> rwi = dpl.detrend(dpl.readers("../tests/data/csv/file.csv"), plot=False)
>>> dpl.chron_ars(rwi)

References

.. [1] https://rdrr.io/cran/dplR/man/chron.ars.html .. [2] Cook, E.R. (1985) A time series analysis approach to tree-ring standardization. PhD dissertation, University of Arizona.

autoreg

Auto Regressive (AR) functions

Extended Summary

Selects the best AR model with a specified maximum order for the given data, and returns the parameters for the model. The best model is selected based on AIC value.

Parameters:

Name	Type	Description	Default
data	Series	an individual (Pandas) series representing tree rings/widths.	required
max_lag	int	max lag to consider when selecting the AR model.	5

Returns:

Name	Type	Description
params	array containing the parameters of best-fit AR model in order.

Examples:

>>> import dplpy as dpl 
>>> data = dpl.readers("../tests/data/csv/file.csv")
>>> dpl.autoreg(data['series name']) -> returns parameters of best fit AR model
                                        with maxlag of 5 (default) or other 
                                        specified number
References
----------
.. [1] https:/opendendro.org/dplpy-man/#autoreg

ar_func

Auto Regressive (AR) functions

Extended Summary

Fits a given data to an the best-fit autoregressive model, returns the residuals of AR fit relative to the original data + the mean of the original data.

Parameters:

Name	Type	Description	Default
data	DataFrame | Series	a pandas dataframe imported from dpl.readers() or a series extracted from such a dataframe.	required
max_lag		max lag to consider when selecting the AR model.	5

Returns:

Name	Type	Description
res	pandas dataframe or series of AR-modeled data, depending on which was given as input.

Examples:

>>> import dplpy as dpl 
>>> data = dpl.readers("../tests/data/csv/file.csv")
>>> dpl.ar_func(data['series name']) -> returns residuals plus mean of best fit 
                                        from AR models with max lag of either 5 
                                        (default) or specified number

References

.. [1] https:/opendendro.org/dplpy-man/#ar_func