Detrending & standardization¶
Removing the biological growth trend to produce ring-width index (RWI) series. See the Detrending & standardization guide for when to use each method.
detrend¶
Detrend a series or dataframe to ring-width indices.
Fit a growth curve to each series (a smoothing spline by default) and form the
ring-width index as the ratio (method="ratio", division, the default) or the
difference (method="difference", subtraction) of the data to the curve.
fit chooses the CURVE and method chooses the ARITHMETIC. This differs
from dplR, where method selects the curve; passing a curve name to
method raises a helpful error.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
data
|
pandas dataframe or series
|
ring widths from |
required |
fit
|
str or list
|
the growth curve to fit (case-insensitive; dplR's names are canonical):
|
"Spline"
|
method
|
(ratio, difference)
|
how the index is formed: |
"ratio"
|
plot
|
bool or sequential
|
|
False
|
spline_period
|
int or float
|
smoothing wavelength for |
None
|
nyrs0
|
int
|
initial spline stiffness for |
50
|
pos_slope
|
boolean
|
allow a positive slope at the old-age end: for |
False
|
f
|
float
|
spline frequency-response amplitude at |
0.5
|
verbose
|
boolean
|
print, per series, the curve fitted and the arithmetic used. |
False
|
return_info
|
boolean
|
if True, return a dict (like dplR's |
False
|
Returns:
| Name | Type | Description |
|---|---|---|
data |
pandas dataframe or series
|
the detrended data, tagged |
Examples:
>>> import dplpy as dpl
>>> data = dpl.readers("../tests/data/csv/file.csv")
>>> dpl.detrend(data) # all series in a dataframe
>>> dpl.detrend(data["SeriesA"], fit="ModNegExp", method="difference")
Notes
Interior NaN are unmeasured rings. They are left out of the fit and stay
NaN in the index, but they still count as rings: the position-based fits
(every curve except Spline and Mean) are fitted on the true ring
position, so a gap advances cambial age rather than compressing the axis
after it (AgeDepSpline carries the previous ring's value across the gap
for its fit). Spline is fit over calendar years and Mean is
order-invariant. To fill gaps instead, use dpl.fill_internal first.
A series too short for the requested curve (fewer than 2 to 4 rings, depending on the fit) is detrended by its mean with a warning -- dplR's "dirty dog".
References
.. [1] https://opendendro.github.io/dplPy/reference/detrending/#detrend
rcs¶
Regional Curve Standardization of a set of ring-width series.
Align every series on a common cambial-age axis using its pith offset, average
to form the regional curve, smooth it, and divide (ratio) or subtract
(difference) the curve value at each ring's cambial age. This preserves the
low-frequency growth signal that per-series detrending removes. Shares the
fit / method / spline_period vocabulary of detrend. Analog of
dplR's rcs.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
rwl
|
DataFrame
|
ring-width series, years as the index and series as columns. |
required |
po
|
(DataFrame, dict, Series or None)
|
pith offsets: a table with |
None
|
fit
|
str or None
|
curve fitted to the age-aligned mean, from the |
"AgeDepSpline"
|
method
|
(ratio, difference)
|
how each index is formed from the ring width and the regional-curve value at
its cambial age: |
"ratio"
|
spline_period
|
int or float
|
smoothing wavelength for |
0.1
|
nyrs0
|
int
|
initial stiffness for |
50
|
f
|
float
|
frequency-response amplitude for |
0.5
|
pos_slope
|
bool
|
whether the regional curve may rise at the old-age end. False keeps a
non-increasing end, avoiding an artefactual old-age upturn. Applies to
|
False
|
biweight
|
bool
|
build the regional curve with a Tukey biweight robust mean (else arithmetic). |
True
|
min_n
|
int or None
|
truncate the regional-curve tail where by-age sample depth falls below this.
Raises if no cambial age reaches |
None
|
preset
|
(None, crust)
|
None follows the parameters above. "crust" builds the regional curve the CRUST
way (Melvin & Briffa 2014): an age-dependent spline smoothed only where sample
depth >= 4 with the no-rise tail rule, gaps infilled, a 0.02 mm floor, and flat
tail extension. When set, |
None
|
rescale_difference
|
bool
|
used only when |
False
|
rc_out
|
bool
|
if True, return a dict with both the detrended |
False
|
plot
|
bool
|
draw the regional-curve figure (series and curve vs cambial age). The former
name |
True
|
Returns:
| Type | Description |
|---|---|
DataFrame or dict
|
the detrended |
Notes
An interior NaN is an unmeasured ring: it advances the series' cambial age and its index stays NaN, so the years after a gap keep their true age (as in CRUST and dplR). Where no series reaches a cambial age inside the curve's span, the parametric fits use the true age axis and the splines carry the previous age's mean forward (with a warning).
References
.. [1] https://rdrr.io/cran/dplR/man/rcs.html .. [2] Melvin, T. M. & Briffa, K. R. (2014) CRUST: Software for the implementation of Regional Chronology Standardisation: Part 1, Signal-Free RCS. Dendrochronologia, 32, 7-20. .. [3] Melvin, T. M. & Briffa, K. R. (2014) CRUST: Software for the implementation of Regional Chronology Standardisation: Part 2, Further RCS options and recommendations. Dendrochronologia, 32, 343-356.
sfrcs¶
Signal-Free Regional Curve Standardization of a set of ring-width series.
Ordinary RCS lets the single regional curve absorb some of the common climate
signal when the sample's age structure varies over time, biasing the low
frequencies. Signal-Free RCS breaks that feedback: it iterates, each pass
rebuilding the regional curve from measurements with the current chronology
divided out, while still forming the tree indices from the original
measurements. Iteration stops when the chronology stops changing (tol). Port
of CRUST's single-curve signal-free RCS (Melvin & Briffa 2014). The regional
curve is built exactly as rcs builds it, so sfrcs(max_iterations=1)
equals rcs (with the same curve settings) up to the signal-free
normalisation of the chronology to mean 1.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
rwl
|
DataFrame
|
ring-width series, years as the index and series as columns. |
required |
po
|
(DataFrame, dict, Series or None)
|
pith offsets (true offset, 0 = pith present), as in |
None
|
fit
|
regional-curve settings, as in |
'AgeDepSpline'
|
|
spline_period
|
regional-curve settings, as in |
'AgeDepSpline'
|
|
nyrs0
|
regional-curve settings, as in |
'AgeDepSpline'
|
|
f
|
regional-curve settings, as in |
'AgeDepSpline'
|
|
pos_slope
|
regional-curve settings, as in |
'AgeDepSpline'
|
|
min_n
|
regional-curve settings, as in |
'AgeDepSpline'
|
|
method
|
(ratio, difference)
|
how indices are formed: "ratio" divides, "difference" subtracts. |
"ratio"
|
biweight
|
bool
|
use a Tukey biweight robust mean both for the regional curve by cambial age
and for the yearly chronology. Default False (arithmetic means) matches
CRUST. The former pair |
False
|
preset
|
(None, crust)
|
"crust" (the default, unlike |
None
|
ss
|
int
|
CRUST's age-dependent spline stiffness offset ( |
10
|
rise
|
bool or None
|
allow the CRUST curve to rise in its final third. None uses CRUST's rule (allowed only for curves longer than 1500 years). |
None
|
max_iterations
|
int
|
maximum signal-free iterations. |
40
|
tol
|
float
|
convergence threshold on the maximum chronology change between iterations. |
1e-3
|
rescale_difference
|
bool
|
|
False
|
plot
|
bool
|
draw the regional-curve figure (series and final curve vs cambial age). The
former name |
True
|
return_info
|
bool
|
if True, return a dict of diagnostics (see Returns) instead of just |
False
|
verbose
|
bool
|
print a convergence line. |
True
|
Returns:
| Type | Description |
|---|---|
DataFrame or dict
|
the detrended |
Notes
Single-curve case (CRUST trc=1, src=1); multiple-curve signal-free RCS is
sfmrcs. There is no dplR reference for this method: the port follows CRUST's
stand.f90 and its spline smoother is validated against CRUST's compiled
kernel, but the full iteration is not checked against a gold standard, so treat
results as CRUST-faithful rather than machine-verified. An interior NaN is an
unmeasured ring: it advances cambial age and stays NaN in the output.
References
.. [1] Melvin, T. M. & Briffa, K. R. (2014) CRUST: Software for the implementation of Regional Chronology Standardisation: Part 1, Signal-Free RCS. Dendrochronologia, 32, 7-20. .. [2] Melvin, T. M. & Briffa, K. R. (2014) CRUST: Software for the implementation of Regional Chronology Standardisation: Part 2, Further RCS options and recommendations. Dendrochronologia, 32, 343-356.
mrcs¶
Multiple-curve Regional Curve Standardisation.
Sort trees by relative growth rate, split them into n_curves equal-count
classes, build a regional curve for each class, and standardise each tree by its
own class curve. This removes the modern-sample-bias slope that a single
regional curve injects when the sample's growth-rate (or age) mix changes over
time. Each class curve is built by rcs(), so mrcs shares its
fit/method/spline_period/nyrs0/f/pos_slope/preset
vocabulary, and mrcs(rwl, n_curves=1) is identical to rcs(rwl). Port of
CRUST's multiple RCS.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
rwl
|
DataFrame
|
ring-width series, years as the index and series as columns. |
required |
po
|
(DataFrame, dict, Series or None)
|
pith offsets ( |
None
|
n_curves
|
int
|
number of regional curves. |
2
|
sort
|
(growth, age, diameter, none)
|
how trees are ranked before the equal-count split: "growth" by relative growth rate, "age" by cambial age, "diameter" by total radius, "none" keeps input order. Case-insensitive. |
"growth"
|
fit
|
|
'AgeDepSpline'
|
|
method
|
|
'AgeDepSpline'
|
|
spline_period
|
|
'AgeDepSpline'
|
|
nyrs0
|
|
'AgeDepSpline'
|
|
f
|
|
'AgeDepSpline'
|
|
pos_slope
|
|
'AgeDepSpline'
|
|
biweight
|
|
'AgeDepSpline'
|
|
min_n
|
|
'AgeDepSpline'
|
|
preset
|
|
'AgeDepSpline'
|
|
rescale_difference
|
passed through to |
False
|
|
reset_means
|
bool
|
if True (ratios only), rescale each tree's indices to the mean it would have
had under one-curve RCS, recovering the low-frequency variance multiple RCS
removes. Skipped with a warning for |
True
|
rc_out
|
bool
|
if True, return a dict |
False
|
plot
|
bool
|
draw the per-class regional-curve figure. |
True
|
Returns:
| Type | Description |
|---|---|
DataFrame or dict
|
the detrended |
References
.. [1] Melvin, T. M. & Briffa, K. R. (2014) CRUST: Software for the implementation of Regional Chronology Standardisation: Part 2, Further RCS options and recommendations. Dendrochronologia, 32, 343-356.
sfmrcs¶
Signal-Free multiple-curve Regional Curve Standardisation.
Combine sfrcs() (signal-free iteration) with mrcs() (multiple
curves): sort trees by growth rate into n_curves equal-count classes, then
iteratively rebuild each class's regional curve from measurements that have had
the current chronology divided out, while forming the tree indices from the
original measurements, until the chronology converges. reset_means (ratios
only) restores each tree's one-curve mean to recover the low-frequency variance
multiple RCS removes. sfmrcs(rwl, n_curves=1) is identical to
sfrcs(rwl). Port of CRUST's signal-free multiple RCS.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
rwl
|
DataFrame
|
ring-width series, years as the index and series as columns. |
required |
po
|
(DataFrame, dict, Series or None)
|
pith offsets (true offset, 0 = pith present), as in |
None
|
n_curves
|
int
|
number of regional curves. |
2
|
sort
|
(growth, age, diameter, none)
|
how trees are ranked before the equal-count split (see |
"growth"
|
fit
|
the regional-curve settings, identical to |
'AgeDepSpline'
|
|
spline_period
|
the regional-curve settings, identical to |
'AgeDepSpline'
|
|
nyrs0
|
the regional-curve settings, identical to |
'AgeDepSpline'
|
|
f
|
the regional-curve settings, identical to |
'AgeDepSpline'
|
|
pos_slope
|
the regional-curve settings, identical to |
'AgeDepSpline'
|
|
min_n
|
the regional-curve settings, identical to |
'AgeDepSpline'
|
|
preset
|
the regional-curve settings, identical to |
'AgeDepSpline'
|
|
ss
|
the regional-curve settings, identical to |
'AgeDepSpline'
|
|
rise
|
the regional-curve settings, identical to |
'AgeDepSpline'
|
|
method
|
(ratio, difference)
|
ratio (division) or difference (subtraction) indices. |
"ratio"
|
biweight
|
the signal-free settings, identical to |
False
|
|
max_iterations
|
the signal-free settings, identical to |
False
|
|
tol
|
the signal-free settings, identical to |
False
|
|
verbose
|
the signal-free settings, identical to |
False
|
|
reset_means
|
bool
|
restore each tree's one-curve mean (ratios only; skipped with a warning
for |
True
|
rescale_difference
|
bool
|
|
False
|
plot
|
bool
|
draw the per-class regional-curve figure ( |
True
|
return_info
|
bool
|
if True, return a dict of diagnostics (see Returns) instead of |
False
|
Returns:
| Type | Description |
|---|---|
DataFrame or dict
|
the detrended |
References
.. [1] Melvin, T. M. & Briffa, K. R. (2014) CRUST: Software for the implementation of Regional Chronology Standardisation: Part 2, Further RCS options and recommendations. Dendrochronologia, 32, 343-356.
ssf¶
Simple signal-free chronology (dplR's ssf()).
Build a signal-free chronology (Melvin & Briffa 2008): form an initial
chronology, divide every series by it to strip the common signal, rescale and
re-detrend the resulting signal-free measurements, and build a new chronology,
repeating until the high-frequency chronology stops changing (the sample-depth-
weighted median absolute difference falls below tol) or max_iterations
is reached. Reproduces dplR's ssf(); for data with many absent rings that the
basic method cannot standardise, use preset="crust" (see the note below).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
rwl
|
DataFrame
|
ring-width series (raw), years as the index and series as columns. Series with no measured values are dropped with a warning. |
required |
fit
|
(Spline, AgeDepSpline)
|
detrending curve used at each iteration (the |
"Spline"
|
method
|
(ratio, difference)
|
how indices are formed: |
"ratio"
|
spline_period
|
int or float or None
|
stiffness of the curve. For |
None
|
max_iterations
|
int
|
maximum signal-free iterations before giving up. Under |
25
|
tol
|
float
|
convergence threshold on the median absolute difference of the
high-frequency chronology between successive iterations (dplR's
|
5e-4
|
recode_zeros
|
bool
|
recode zero ring-widths to 0.001 before processing (avoids div-by-zero
from literal zero rings). The input frame is not modified. Note this does
not help a chronology that is low but positive in a severe pointer year
(the usual cause of a non-positive signal-free curve); use |
False
|
return_info
|
bool
|
if True, return a dict of the full iteration history (chronologies, signal-free measurements and curves, MAD vector, ...) instead of just the final chronology. |
False
|
verbose
|
bool
|
print progress and the per-iteration convergence diagnostics. |
True
|
preset
|
(None, crust)
|
None reproduces the basic signal-free method (dplR's ssf; Melvin & Briffa 2008). "crust" adds CRUST's robustness guards (Melvin & Briffa 2014): multiplicative rescaling of the signal-free measurements, the raw measurement kept in any year where the chronology is near zero (< 0.01), the fitted curves floored at 0.02 mm, and up to 40 iterations. These make ssf robust to sensitive sites with many absent rings (e.g. co021) that the basic method cannot standardise, and keep every series in the chronology. Case-insensitive. |
None
|
Returns:
| Type | Description |
|---|---|
DataFrame
|
the signal-free chronology with columns |
Notes
The basic method stops with an error when a signal-free curve goes to zero or
below -- typically in a year dominated by absent (zero-width) rings, which
drives the chronology near zero so that dividing by it inflates the signal-free
measurements and the refitted spline dips below zero. This afflicts the whole
collection at that pointer year rather than a few individual series (on co021,
for example, trying to drop the offending series would cascade to nearly all of
them), so ssf does not drop series; it reports the cause and directs you to
preset="crust", which floors the curve and standardises such data with
every series retained. dplPy's error is more specific than dplR's but stops in
the same place.
References
.. [1] https://rdrr.io/cran/dplR/man/ssf.html .. [2] Melvin, T. M. & Briffa, K. R. (2008) A "signal-free" approach to dendroclimatic standardisation. Dendrochronologia, 26(2), 71-86. .. [3] Melvin, T. M. & Briffa, K. R. (2014) CRUST: Software for the implementation of Regional Chronology Standardisation: Part 1, Signal-Free RCS. Dendrochronologia, 32, 7-20. .. [4] Melvin, T. M. & Briffa, K. R. (2014) CRUST: Software for the implementation of Regional Chronology Standardisation: Part 2, Further RCS options and recommendations. Dendrochronologia, 32, 343-356.
ads¶
Age-dependent smoothing spline of a single series (dplR's ads()).
Fit a smoothing spline whose stiffness increases with cambial age: the i-th ring is fit with an (nyrs0 + i - 1)-year spline, so young rings are smoothed least and the curve grows stiffer with age. Ports dplR's ads().
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
y
|
array - like
|
a single ring-width series (no missing values, at least 4 rings), oldest ring first. |
required |
nyrs0
|
int
|
initial spline stiffness (wavelength, in years) at the first ring. |
50
|
pos_slope
|
bool
|
if False (the default, as in dplR's detrend() and every dplPy detrender), once the spline stops decreasing its tail is held flat and the spline is refit, preventing an artefactual upturn at the end of the series. True lets the curve rise (dplR's standalone ads() default). |
False
|
Returns:
| Type | Description |
|---|---|
ndarray
|
the fitted spline curve, same length as |
Examples:
>>> import dplpy as dpl
>>> rwl = dpl.readers("../tests/data/csv/ca533.csv")
>>> curve = dpl.ads(rwl["CAM011"].dropna().to_numpy(), nyrs0=50)
References
.. [1] https://rdrr.io/cran/dplR/man/ads.html .. [2] Melvin, T. M. (2004) Historical Growth Rates and Changing Climatic Sensitivity of Boreal Conifers. PhD Thesis, Climatic Research Unit, University of East Anglia. .. [3] Melvin, T. M., Briffa, K. R., Nicolussi, K. & Grabner, M. (2007) Time-varying-response smoothing. Dendrochronologia, 25(2), 65-69. .. [4] Cook, E. R. & Peters, K. (1981) The Smoothing Spline: A New Approach to Standardizing Forest Interior Tree-Ring Width Series for Dendroclimatic Studies. Tree-Ring Bulletin, 41, 45-53.
powt¶
Adaptive power transformation of ring-width series (variance stabilising).
Stabilise the variance of ring-width series by raising each value to a data-driven power p. For adjacent rings, level M = (x[t] + x[t-1]) / 2 and spread S = |x[t] - x[t-1]|; regressing log(S) on log(M) gives a slope b, and p = |1 - b| removes the level-spread dependence. Port of dplR's powt (Cook & Peters 1997).
Two methods are offered:
"cook"(default) estimates p independently for each series."universal"estimates a single p for the whole data set with a linear mixed-effects model (random intercept per year), matching dplR's default. Requires statsmodels; a log transform is used if the estimate is p <= 0.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
rwl
|
DataFrame or Series
|
ring-width series (raw, non-negative), years as the index. A Series is allowed only with method='cook'. Series with no measured values are dropped with a warning. |
required |
method
|
(cook, universal)
|
per-series ('cook') or single mixed-model power ('universal'). Case-insensitive. |
"cook"
|
rescale
|
bool
|
if True, rescale each transformed series back to its original mean and standard deviation. |
False
|
return_power
|
bool
|
if True, return a (transformed, power) tuple; |
False
|
plot
|
bool
|
if True, draw the data-transform-statistics diagnostic: per-series skew and spread-vs-level correlation, before and after the transform. A successful transform drives the after values toward zero. |
False
|
return_stats
|
bool
|
if True, also return a per-series diagnostics DataFrame with columns
|
False
|
Returns:
| Type | Description |
|---|---|
transformed data of the same type as ``rwl`` (tagged ``attrs["kind"] =
|
|
"rwl_powt"`` and carrying the input's provenance attrs). With
|
|
``return_power`` and/or ``return_stats`` the return becomes a tuple:
|
|
``(data, power)``, ``(data, stats)``, or ``(data, power, stats)`` (power
|
|
first, stats last).
|
|
Notes
dplPy reproduces dplR's power exactly. A given series' power can differ slightly
from ARSTAN's, which uses the trailing ring as the level and clamps p to
[0, 1].
Examples:
>>> import dplpy as dpl
>>> rwl = dpl.readers("../tests/data/csv/ca533.csv")
>>> pt = dpl.powt(rwl) # per-series (cook)
>>> pt, p = dpl.powt(rwl, return_power=True)
>>> rwi = dpl.detrend(pt, plot=False) # transform, then detrend
References
.. [1] https://rdrr.io/cran/dplR/man/powt.html .. [2] Cook & Peters (1997), The Holocene, 7, 361-370.