\documentclass[10pt]{article}

\usepackage{fullpage}
\usepackage{setspace}
\usepackage{parskip}
\usepackage{titlesec}
\usepackage[section]{placeins}
\usepackage{xcolor}
\usepackage{breakcites}
\usepackage{lineno}
\usepackage{hyphenat}





\PassOptionsToPackage{hyphens}{url}
\usepackage[colorlinks = true,
            linkcolor = blue,
            urlcolor  = blue,
            citecolor = blue,
            anchorcolor = blue]{hyperref}
\usepackage{etoolbox}
\makeatletter
\patchcmd\@combinedblfloats{\box\@outputbox}{\unvbox\@outputbox}{}{%
  \errmessage{\noexpand\@combinedblfloats could not be patched}%
}%
\makeatother


\usepackage{natbib}




\renewenvironment{abstract}
  {{\bfseries\noindent{\abstractname}\par\nobreak}\footnotesize}
  {\bigskip}

\titlespacing{\section}{0pt}{*3}{*1}
\titlespacing{\subsection}{0pt}{*2}{*0.5}
\titlespacing{\subsubsection}{0pt}{*1.5}{0pt}


\usepackage{authblk}


\usepackage{graphicx}
\usepackage[space]{grffile}
\usepackage{latexsym}
\usepackage{textcomp}
\usepackage{longtable}
\usepackage{tabulary}
\usepackage{booktabs,array,multirow}
\usepackage{amsfonts,amsmath,amssymb}
\providecommand\citet{\cite}
\providecommand\citep{\cite}
\providecommand\citealt{\cite}
% You can conditionalize code for latexml or normal latex using this.
\newif\iflatexml\latexmlfalse
\providecommand{\tightlist}{\setlength{\itemsep}{0pt}\setlength{\parskip}{0pt}}%

\AtBeginDocument{\DeclareGraphicsExtensions{.pdf,.PDF,.eps,.EPS,.png,.PNG,.tif,.TIF,.jpg,.JPG,.jpeg,.JPEG}}

\usepackage[utf8]{inputenc}
\usepackage[T2A]{fontenc}
\usepackage[greek,ngerman,polish,english]{babel}



\usepackage{float}








\begin{document}

\title{A mismatch between community assembly and abundance-based diversity
indices}



\author[1]{Matthew Edmonds}%
\author[2]{Jennifer Bufford}%
\author[3]{William Godsoe}%
\affil[1]{University of Canterbury }%
\affil[2]{Manaaki Whenua - Landcare Research}%
\affil[3]{University of Canterbury}%


\vspace{-1em}



  
  \date{May 20, 2024}


\begingroup
\let\center\flushleft
\let\endcenter\endflushleft
\maketitle
\endgroup





\selectlanguage{english}
\begin{abstract}
Long-term ecological studies have consistently reported slower than
expected changes in biodiversity over time. One explanation for this
phenomenon is that commonly used diversity measurements such as species
richness are too coarse to detect mechanisms shaping community assembly.
It remains unclear whether abundance based diversity measurements are
susceptible to the same problem. To test this, we study temporal changes
in abundance based diversity indices across 3341 observations from 880
plots from 15 long-term vegetation plot studies. We then partition
diversity change into mechanisms of interest to ecologists: selection,
drift, and immigration. We show that these mechanisms are an imperfect
predictor of temporal diversity change, creating a mismatch between
changes in species abundances and changes in diversity, particularly
when shifts in diversity are rapid. To resolve this mismatch, we
quantify a less studied mechanism ``rarity shifts'', which measure how
an individual's contributions to diversity changes over time. We found
rarity shifts are an important component of diversity change across many
studies. Furthermore, rarity shifts tend to oppose other mechanisms,
particularly selection. Therefore, rarity shifts obscure changes in
relative abundance from abundance based diversity measurements,
revealing why diversity changes appear slower than expected. Ultimately,
understanding rarity shifts can lead to a more accurate understanding of
the rate and nature of temporal diversity change in ecology and
conservation.%
\end{abstract}%



\sloppy


\subsection*{A mismatch between community assembly and abundance-based
diversity
indices}

{\label{a-mismatch-between-community-assembly-and-abundance-based-diversity-indices}}

\section*{Abstract}

{\label{abstract}}

Long-term ecological studies have consistently reported slower than
expected changes in biodiversity over time. One explanation for this
phenomenon is that commonly used diversity measurements such as species
richness are too coarse to detect mechanisms shaping community assembly.
It remains unclear whether abundance based diversity measurements are
susceptible to the same problem. To test this, we study temporal changes
in abundance based diversity indices across 3341 observations from 880
plots from 15 long-term vegetation plot studies. We then partition
diversity change into mechanisms of interest to ecologists: selection,
drift, and immigration. We show that these mechanisms are an imperfect
predictor of temporal diversity change, creating a mismatch between
changes in species abundances and changes in diversity, particularly
when shifts in diversity are rapid. To resolve this mismatch, we
quantify a less studied mechanism ``rarity shifts'', which measure how
an individual's contributions to diversity changes over time. We found
rarity shifts are an important component of diversity change across many
studies. Furthermore, rarity shifts tend to oppose other mechanisms,
particularly selection. Therefore, rarity shifts obscure changes in
relative abundance from abundance based diversity measurements,
revealing why diversity changes appear slower than expected. Ultimately,
understanding rarity shifts can lead to a more accurate understanding of
the rate and nature of temporal diversity change in ecology and
conservation.

Key Words: Biodiversity, species richness, mechanistic models,
partition, selection, immigration, Shannon entropy, Gini-Simpsons

Introduction

A central goal of ecology is to understand biodiversity change over time
(MacArthur 1965, McGill et al. 2015, Dornelas et al. 2018, Chase et al.
2019, Dornelas et al. 2019). Biodiversity represents the variety of
living organisms found in a given place. Biodiversity is a key concept
in conservation, used to assess ecosystem health in response to
management or anthropogenic stressors and to prioritize areas for
protection (McGill et al. 2015, Hillebrand et al. 2018). Despite
disruptive anthropogenic environmental changes, many long-term
ecological studies show surprisingly little change in biodiversity
within locations (Sax et al. 2002, Vellend et al. 2013, Vellend et al.
2017, Hillebrand et al. 2018). For example, analyses of vegetation plots
show no overall decline in local scale biodiversity over time (Vellend
et al. 2013). Surprisingly, this apparent stasis concealed important
trends in community assembly: the rate of extinctions increased over
time, but this effect was obscured by an increase in the rate of
colonisations (Dornelas et al. 2019). Given the need to predict shifts
in biodiversity (Urban et al. 2016), there is a need to understand the
extent of this mismatch.

Species richness, the most widely studied diversity measurement, may
give an overly coarse picture of diversity change (Hillebrand et al.
2018). This is because richness provides no explicit information about
species abundances, and many of the mechanisms that operate in
communities act on abundances (MacArthur 1965, Urban et al. 2016, Godsoe
et al. 2023). The logical alternative is abundance-based diversity
indices such as Shannon entropy and Gini-Simpson's (Jost 2006). These
metrics incorporate information on the evenness of species' relative
abundances in addition to richness.

When studying changes in abundance based diversity indices, it makes
sense to focus on mechanisms that change relative abundances such as
selection and drift at the species level (Vellend 2016). A major
advantage of this approach is that selection describes changes in
relative abundances and diversity summarizes information on relative
abundances (Jost 2006). In contrast, species interactions such as
competition or predation are characterized by their effects on absolute
abundances. Competition, for example, decreases the absolute abundance
of some species at the expense of others, while predation increases the
absolute abundance of predators and decreases the absolute abundance of
prey (Holland and DeAngelis 2009). By focusing on selection and drift,
we are implicitly capturing some of the consequences of species
interactions (Vellend 2010), but multiple species interactions can have
indistinguishable effects on diversity (Godsoe et al. 2023).

To quantify the effects of mechanisms that alter diversity, Godsoe et
al. (2021) proposed partitioning diversity change for abundance-based
diversity measurements. This approach starts by measuring each
individual's contribution to a diversity index, with individuals
belonging to rare species contributing more than individuals belonging
to common species (Patil and Taillie 1982, Roswell et al. 2021). Overall
diversity change is then decomposed into changes in individual
contributions. For example, species-level selection occurs when
individuals belonging to rare species have higher fitness than
individuals belonging to common species. Species-level drift occurs when
chance events lead to differences in the number of descendants produced
by individuals belonging to rare species relative to common species.
Note that, in small, observational studies drift and selection can
produce similar effects (i.e. a change in the relative abundances of
individuals from species that were already in the community). For this
reason, we describe drift under the term ``selection'', but see Godsoe
et al. (2022), for a simulation-based test for the effects of drift in
larger plots.

Under some circumstances, partitioning emphasizes the role of mechanisms
familiar to ecologists. For example, Figure 1 illustrates changes in
relative abundances in observations of forest plots in Brazil (Farah et
al. 2014) from the BioTIME database (Dornelas et al. 2018). In this
case, the relative abundance of a rare species \emph{Psychotria
vauthieri} slightly increased, while the relative abundance of a common
species \emph{Ixora gardneriana} (Figure 1A) slightly decreased between
1994 and 1999. This leads to a small increase in diversity (Figure 1C).
There was no immigration in this community, and the total change in
diversity is more or less equal to the increase in selection from rare
species becoming more common in the plot (Figure 1E). In contrast, rapid
changes in relative abundances can lead to a mismatch between total
change in diversity and mechanisms changing relative abundances. For
example, the right column in Figure 1 illustrates forest plots with far
more dramatic changes in relative abundances. In 1994 \emph{Plinia
cauliflora} is the rarer species while \emph{Croton floribundus} is more
common. However, \emph{P. cauliflora} increases in relative abundance to
become more common than \emph{C. floribundus} in 1999. This dramatic
flip in relative abundances produces a small change in diversity (Figure
1 D) which is similar to the diversity change recorded in the first
forest plot (Figure 1C). This is because the success of rare species,
which is captured by the selection term, increasing diversity, is
counteracted by the rapid changes in rarity exhibited. The opposing
mechanisms of rarity shifts and selection in this example explained why
diversity changes were smaller in magnitude than expected; highlighting
how a mismatch between relative abundances and diversity change may
arise (Figure 1 F).\selectlanguage{english}
\begin{figure}[H]
\begin{center}
\includegraphics[width=0.70\columnwidth]{figures/image1/image1}

\end{center}
\end{figure}

Figure 1: Gradual changes in diversity may obscure strong changes in
relative abundance, as shown between 1994 and 1999 (indicated by grey
bands) in Santa Genebra Forest, Brazil (Farah et al. 2014). A)
illustrates gradual changes in relative abundances for
species\emph{Ixora gardneriana} (orange) and \emph{Psychotria
vauthieri}(yellow). B) illustrates comparatively rapid changes in
relative abundances for species \emph{Croton floribundus} (orange)
and\emph{Plinia cauliflora} (yellow). C) The gradual changes in
abundances in A lead to relatively small changes in diversity. Here, the
arrow indicates the change in the community with the diversity change
highlighted in grey. D) The rapid changes in abundances in B also lead
to a surprisingly weak effect on diversity change. E) Partitions total
change in diversity into an effect of selection, and rarity shifts
(blue, but barely visible). This highlights the primary role of
selection in the example presented in A. F) In contrast, partitioning of
change in example B shows selection increases diversity, while rarity
shifts simultaneously decrease diversity. This interplay of selection
and rarity shifts leads to a modest change in diversity.

To better understand the importance of rarity shifts we analyse 15
terrestrial vascular plant studies. Vegetation plots are ideal for our
purposes because plants are sessile, meaning new arrivals are unlikely
to be confused with descendants (Table 1). We use partitioning to
decompose total diversity change into selection, immigration, and rarity
shifts. We then address three questions: 1) Are rarity shifts a large
component of diversity change? We address this by comparing the
distribution of measured rarity shifts to overall diversity change
across all observations. 2) Are rarity shifts equally common across
studies? We address this by contrasting the distribution of rarity
shifts across each study. 3) We then asked how the strength of rarity
shifts changes with the strength of the other mechanisms? We address
this by using a generalized additive model to predict when rarity shifts
are most common.

\section*{Material and Methods}

{\label{material-and-methods}}

\subsection*{The model}

{\label{the-model}}

To analyse change within a plot we use the partitioning approach from
Godsoe et al. (2021). Within a given plot the absolute abundance of each
species is \(n_{i}\) and the relative abundance of each species
is\(p_{i}=n_{i}/\sum_{i}n_{i}\). For many diversity indices, the contribution that
each individual makes to diversity can be defined as z\textsubscript{i},
the rarity of the species to which it belongs (Equation 1). For Shannon
entropy, the measure of rarity is
z\textsubscript{i}=--log(p\textsubscript{i}). Because we are interested
in change between one period and another we can define rarity scores for
Gini-Simpson's diversity as z\textsubscript{i} =
-p\textsubscript{i}(Godsoe et al. 2021). Diversity is an average
measurement of rarity across all individuals in a plot (Patil and
Taillie 1982); For Shannon entropy, the arithmetic mean of rarity scores
are weighted by the relative abundance of each species (Jost 2006, 2007,
Cover and Thomas 2012).

\(D=\sum_{i}{p_{i}z_{i}}\).

Total change in diversity over time is the difference between a
measurement of diversity in the present and diversity in the past
(Equation 2). Here and elsewhere, describes change over time and the
prime superscript denotes present measurements (Frank 2012).

\(D=D^{{}^{\prime}}-D\) .

To understand the effects of immigrants on diversity change, we further
divide the present community is into two components:

\(D^{{}^{\prime}}=\selectlanguage{greek}\text{\ ω}\selectlanguage{english}\sum_{i}{p_{i}^{\prime}z_{i}^{\prime}}+\mu\sum_{j}{a_{j}z_{j}^{*}}\).

The first term represents what we will call ``descendants'', individuals
that are not immigrants. They may represent individuals that were in the
community in the first survey, or their offspring. Here
\(\omega\) is the proportion of individuals in the present
community that descended from the past community. Among descendants the
proportion belonging to species \emph{i} is
p\textsubscript{i}\({}^{\prime}\), and the rarity score of species
\emph{i} is \emph{z\textsubscript{i}} \({}^{\prime}\) (note this is
rarity as a proportion of total individuals in both time steps). The
second term describes the contribution of immigrants, where
\(\mu\) is the proportion of individuals in the present
community which immigrated to the community since the initial
measurement. Among immigrants, the proportion belonging to species
\emph{i} is a\textsubscript{j}\({}^{\prime}\), and the rarity score
of species j is z\textsubscript{j}* (note this is rarity as a proportion
of all individuals, not just immigrants).

Overall diversity change in equation 2 can then be partitioned using
extensions of the Price equation (Price 1970, Kerr and Godfrey-Smith
2009). This leads to explicit definitions for the effects of selection,
immigration and rarity shifts on diversity (Equation 4).

(4) \(D=\underset{\par
\begin{matrix}\text{Selection}\\
(+\ drift)\\
\end{matrix}}{}+\underset{\text{immigration}}{}+\underset{\par
\begin{matrix}\text{rarity\ }\\
\text{shifts}\\
\end{matrix}}{}\)

In Equation 4, the first term describes the effect of selection, where
the tendency of species \emph{i} to leave more descendants increases its
frequency relative to other descendants (\(p_{i}=p_{i}^{{}^{\prime}}-p_{i})\). This term
implicitly includes effects of drift because both mechanisms change
species' relative abundances (Rice 2004). In this framework, selection
on species identity emerges when one species tends to leave more
descendants than another either by leaving more offspring or having
higher survival (Vellend 2016). Drift emerges when one species increases
in relative abundance due to stochastic sampling. The second term
describes immigration, which changes diversity when diversity among
immigrants \(\sum_{j}{a_{j}z_{j}^{*}}\) is different from the diversity among
resident ancestors. The final term denotes rarity shifts where the
rarity score for descendants is different from the rarity scores of
ancestors \(z_{i}\), as in (Figure 1 C).

We have presented diversity indices that are familiar and easy to
partition on a linear scale. More complexities emerge when analysing so
called ``Hill numbers'', a re-scaled version of diversity indices
expressed in common units. Both Shannon entropy and Simpsons diversity
indices can be converted into common units the equivalent number of
uniformly distributed species needed to produce the observed diversity
index (Jost 2006, 2007). While not used in this paper, the numbers
equivalent Shannon entropy can be partitioned by exponentiation Equation
4 (Godsoe et al. 2022). This conversion changes the scale of measurement
from additive to multiplicative but preserves many of the qualitative
patterns. At present, techniques exist to other Hill numbers into
selection and transmission terms, in the absence of immigration (Frank
and Godsoe 2020), but this condition is unrealistic for most
observations in our study.

\subsection*{The Data}

{\label{the-data}}

We analysed the causes of biodiversity change from plots in 15 studies
compiled in the BioTIME database (Dornelas et al. 2018). As of December
2021, the BioTIME database contained 361 studies, 201 of which were
terrestrial surveys. We selected vegetation studies with multiple plots
of area greater than 1 m\textsuperscript{2}, including observations from
3 or more time periods, 5 or more species, and counts of individuals (as
opposed to biomass, presence or vegetation cover). This resulted in a
database of 3341 observations of diversity change, with each observation
representing the difference in diversity between one sample and a
previous sample.

To compare the effect of rarity shifts with other mechanisms, we
calculated the change in Shannon entropy and Gini-Simpson's diversity
within each plot across each time step using Equations 1 and 2. This
change in diversity was partitioned using Equation 4. We used histograms
to compare the prevalence of rarity shifts relative to other sources of
diversity change.

To determine how the strength of rarity shifts changes with other
mechanisms, we used a generalized additive model (GAM). This was done
using the mgcv package (Wood and Wood 2015) where the strength of rarity
shifts is modelled as a smoothed non-linear function of selection and
immigration, with study and plot treated as random effects. Increasing
effective degrees of freedom (edf) indicate greater model complexity,
where edf = 1 indicates a straight line. The model was fit using
Restricted Maximum Likelihood (REML). Rarity shifts had an extremely
long tail and so to improve the interpretability of Figure 4,
\textasciitilde{}3\% of the data with the lowest rarity shifts were
removed from the analysis of Shannon entropy. We also removed one
unusually large estimate of selection.

\section*{Results}

{\label{results}}

All mechanisms contributed substantially to the 3341 observations of
diversity change examined. For Shannon entropy, selection had a
substantial effect on diversity change (Figure 2 A). Selection could
either increase or decrease diversity (median: 0, inter quartile range:
-0.025 - 0.061). Immigration tended to increase diversity (median:
0.024, inter quartile range: 0 - 0.085) (Figure 2 B), Intuitively,
immigration increases diversity because the number of species increases
as new species enter a community. Immigration occasionally decreased
diversity when highly diverse communities were replaced by a new
community of immigrants with lower diversity, or when the new immigrant
species became the most dominant species. Rarity shifts commonly
decreased diversity (median: -0.011, inter quartile range: 0.072 - 0)
(Figure 2 C). Total diversity change was centered on zero (median: 0,
inter quartile range: -0.065 - 0.124) (Figure 2 D). Gini-Simpsons
diversity showed similar trends (Supporting information).\selectlanguage{english}
\begin{figure}[H]
\begin{center}
\includegraphics[width=0.70\columnwidth]{figures/image2/image2}

\end{center}
\end{figure}

Figure 2: Histograms plot 3341 observation periods for Shannon entropy
change including A) selection, B) immigration, C) rarity shifts and D)
total diversity change across all plots and from the 15 studies. Each
histogram represents the number of observations where a given partition
was of a particular magnitude. To improve readability, we set the x-axis
from -1 to 1, excluding a small number (3.7\%) of values (see supporting
information for further details). Boxplots show median values,
interquartile range and values exceeding 1.5 times past the
interquartile range.\selectlanguage{english}
\begin{figure}[H]
\begin{center}
\includegraphics[width=0.70\columnwidth]{figures/image3/image3}

\end{center}
\end{figure}

Patterns of diversity varied across the 15 studies (Figure 3). Many
studies showed little diversity change, particularly studies of
long-term forest plots (Bradford et al. 2014). In these studies, total
diversity change is typically zero and the contribution of each
mechanism is negligible. Other studies are far more dynamic such as
communities experiencing disturbance regimes (Webb and Scanga 2001,
Thorn et al. 2016) or regeneration (Venturoli et al. 2011) experienced
strong rarity shifts. In these studies, all mechanisms could make a
substantial contribution to diversity change. Ernest et al. (2009)
studies seasonal flowering xeric shrubland communities in the deserts of
Arizona each summer and winter. Immigration had an unusually strong
strongly influenced diversity. Immigration increased diversity strongly
in some plots and decreased diversity strongly in others. The high
immigration rate reflected the high turnover in species, with the
dominant species frequently being replaced, largely due to the dramatic
fluctuations in precipitation (Baldwin et al. 2002). A similar
experiment examines flowering plant communities in the desert of Arizona
within plots where small mammal have been excluded (Ernest et al. 2009).
Immigration tends to increase diversity, rarity shifts tend to decrease
diversity and selection appeared to take a wide range of positive and
negative values. In this community, the most abundant species is the
Creosote bush, \emph{Larrea tridentata} . This species is perennial and
its abundance changes little over the course of the study. Many of the
rarer species are annuals whose abundances fluctuate dramatically from
one season to another. Figure 3: Partitions of Shannon diversity change
across all plots and time periods for each of the 15 studies. Diversity
change (purple) is partitioned into selection (red), immigration (green)
and rarity shifts (blue) as is described in Equation 4. The dotted line
represents zero diversity change. To improve readability, we set the
x-axis from -.5 to .5, excluding (3.7\%) of values. See supporting
information for more details and Gini-Simpsons diversity comparison.

Generalized Additive Models indicate that there is a non-linear
relationship between the strength of rarity shifts related and the
strength of other mechanisms. Rarity shifts were close to zero when
selection was zero, but tended to become negative when selection was
non-zero. Rarity shifts were close to zero when immigration was close to
zero and was slightly positive when immigration was non-zero (Figure 4).
Each term in each model required a relatively complex spline with many
effective degrees of freedom (selection edf = 7.8, immigration edf =
8.8). Only a modest amount of variation was explained by the model
(R\textsuperscript{2}\textsubscript{adjusted} = 0.345). Supporting
information shows similar trends in Gini-Simpsons diversity.\selectlanguage{english}
\begin{figure}[H]
\begin{center}
\includegraphics[width=0.70\columnwidth]{figures/image4/image4}

\end{center}
\end{figure}

Figure 4: Predictions of the strength of rarity shifts for Shannon
entropy derived from Generalized Additive Models. A) When selection is
close to zero, rarity shifts are close to zero. Selection values further
away from zero lead to negative rarity shifts. Blue lines indicate
predictions, dashes indicate 95 \% confidence intervals, and grey
vertical bars indicate observations. B) When immigration's effects on
diversity are close to zero rarity shifts are close to zero. When
immigration's effects on diversity are more extreme, rarity shifts tend
to be slightly positive.

\section*{Discussion}

{\label{discussion}}

Our work highlights how abundance-based diversity indices can obscure
changes in community assembly. We quantified this effect by measuring
rarity shifts, and showing how this effect can counteract more familiar
sources of diversity change such as selection (Figure 1). This insight
extends the observation that richness based observations of diversity
change can obscure changes in communities (Chase et al. 2019, Dornelas
et al. 2019); (Urban 2015, Urban et al. 2016). Our results indicate that
strong changes in relative abundances can lead to large shifts in
species' rarity. These shifts produce the illusion of negligible
diversity change in dynamic communities.

Rarity shifts tended to oppose increases in selection and decreases in
immigration (Figure 4). Increases in diversity from selection and
decreases in diversity from rarity shifts oppose overall diversity
change (Figure 1). This mismatch is commonplace across all studies
(Figure 2). We have shown that rarity shifts are commonplace across many
datasets (Figure 3), particularly in studies where changes in relative
abundances are high studies such as studies experiencing environmental
disturbances (Webb and Scanga 2001, Thorn et al. 2016) or regeneration
(Venturoli et al. 2011). Additionally, studies of annual plant
communities in the Arizonan desert by (Ernest et al. 2009) and
(Lightfoot 2011) experienced strong rarity shifts because measurements
matched or exceeded the seasonal lifespan of the community, meaning
diversity measures reflected the high mortality and rapid emergence of
plant species. Rarity shifts were uncommon in long-term forest plots
where changes in community composition and diversity are slow (Bradford
et al. 2014). Nevertheless, we elect to analyse the data using the
observations, as they were collected, even if, in retrospect, the
sampling periods used may have been comparatively brief. This choice is
likely conservative, as picking longer intervals would likely increase
shifts in relative abundances increasing the importance of rarity
shifts. By analysing the sampling intervals in the original data, our
results illustrate how gradual changes in diversity sometimes imply
rarity shifts and sometimes indicate negligible overall change.

The observations in the previous paragraph highlight the benefits of
focusing on diversity change in plants. Plants are sessile which lowers
the chance that descendants of one species will be confused with
immigrants of the same species. Long-term vegetation plots are available
in many model systems. As a result, data in our study represents 5
continents - North America, South America, Asia and Australia and
Europe. It is of course possible that other sources of change dominate
in other taxa. We suspect for example that immigration will be more
important in other systems.

Our results are one of several recent applications of diversity
partitions. These include analyses of why empirical observations show
seemingly slow shifts in diversity over time (Dornelas et al. 2019), how
to quantify the effects of biotic homogenization (Tatsumi et al. 2020,
Tatsumi et al. 2021, Godsoe et al. 2022), and how to relate biotic
interactions such as competition to shifts in diversity (Godsoe et al.
2021). We have added to this literature by showing that rarity shifts, a
mechanism derived from partitioning (Godsoe et al. 2021), is an
important source of diversity change.

For generations, ecologists have sought to understand the relationship
between changes in diversity, ecosystem functioning, and the fates of
individual species (MacArthur 1965, Blowes et al. 2019). This debate has
become more urgent as we have confronted our limited ability to explain
shifts in diversity in nature (McGill et al. 2015, Socolar et al. 2016,
Urban et al. 2016, Vellend et al. 2017). To help resolve this problem,
we have shown how abundance-based diversity indices can be influenced by
shifts in species rarity. We have provided examples from individual
plots that highlight how these rarity shifts can obscure selection
(Figure 1). We have shown rarity shifts are comparable in magnitude to
selection, and immigration across the 15 studies (Figure 2 and Figure
3). Finally, we have shown rarity shifts can oppose selection and
immigration (Figure 4). Our results highlight how abundance based
diversity measurements can give a disconcertingly slow impression of
diversity change, even in communities with rapid shifts in relative
abundances. Therefore, reliable predictions of diversity change will
require a more nuanced understanding of the role of rarity shifts

Table 1: Summary of the 15 long-term vegetation plots analysed from the
BioTIME database.\selectlanguage{english}
\begin{longtable}[]{@{}lllllll@{}}
\toprule
\begin{minipage}[b]{0.14\columnwidth}\raggedright\strut
Citation\strut
\end{minipage} & \begin{minipage}[b]{0.14\columnwidth}\raggedright\strut
Number in Biotime\strut
\end{minipage} & \begin{minipage}[b]{0.14\columnwidth}\raggedright\strut
Location\strut
\end{minipage} & \begin{minipage}[b]{0.14\columnwidth}\raggedright\strut
Description\strut
\end{minipage} & \begin{minipage}[b]{0.14\columnwidth}\raggedright\strut
Years between samples\strut
\end{minipage} & \begin{minipage}[b]{0.14\columnwidth}\raggedright\strut
Plot size (m\textsuperscript{2})\strut
\end{minipage} & \begin{minipage}[b]{0.14\columnwidth}\raggedright\strut
Plant density (m\textsuperscript{-2})\strut
\end{minipage}\tabularnewline
\midrule
\endhead
\begin{minipage}[t]{0.14\columnwidth}\raggedright\strut
Webb and Scanga (2001)\strut
\end{minipage} & \begin{minipage}[t]{0.14\columnwidth}\raggedright\strut
10\strut
\end{minipage} & \begin{minipage}[t]{0.14\columnwidth}\raggedright\strut
Minnesota USA\strut
\end{minipage} & \begin{minipage}[t]{0.14\columnwidth}\raggedright\strut
Windthrow on mixed hardwood-evergreen forest\strut
\end{minipage} & \begin{minipage}[t]{0.14\columnwidth}\raggedright\strut
3\strut
\end{minipage} & \begin{minipage}[t]{0.14\columnwidth}\raggedright\strut
4\strut
\end{minipage} & \begin{minipage}[t]{0.14\columnwidth}\raggedright\strut
6.025\strut
\end{minipage}\tabularnewline
\begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
Zachmann et al. (2010)\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
18\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
Idaho USA\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
Old, natural temperate sagebrush scrub\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
1, 2, 3\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
1\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
141.5\strut
\end{minipage}\tabularnewline
\begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
Barreto (2016)\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
302\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
Sao Paulo Brazil\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
Semi-deciduous subtropical broadleaf forest\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
3, 6\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
400\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
0.115\strut
\end{minipage}\tabularnewline
\begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
Salami et al. (2014)\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
303\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
Lages Brazil\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
Subtropical, broadleaf forest.\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
4\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
200\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
0.184\strut
\end{minipage}\tabularnewline
\begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
Farah et al. (2014)\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
322\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
Sao Paulo Brazil\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
Semi-deciduous, subtropical, forest structure\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
5, 11\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
100\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
0.122\strut
\end{minipage}\tabularnewline
\begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
Venturoli et al. (2011)\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
324\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
Brazil\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
Regenerating semi-deciduous saplings\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
1\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
16\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
0.2875\strut
\end{minipage}\tabularnewline
\begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
Venturoli et al. (2011)\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
325\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
Brazil\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
Regenerating semi-deciduous forest\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
1\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
25\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
0.556\strut
\end{minipage}\tabularnewline
\begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
Pelissier et al. (2011)\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
329\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
India\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
Evergreen dipterocarp mature forest\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
3, 4, 5\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
3600\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
0.1172\strut
\end{minipage}\tabularnewline
\begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
Ernest et al. (2009)\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
336\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
Arizona USA\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
Flowering desert shrubs - seasonal\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
0.5\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
2500\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
1.3478\strut
\end{minipage}\tabularnewline
\begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
Lightfoot (2011)\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
340\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
Arizona USA\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
Flowering desert shrubs\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
1\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
1\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
264.6\strut
\end{minipage}\tabularnewline
\begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
Sanquetta (2008)\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
346\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
Sao Paulo Brazil\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
Subtropical grass or shrubland\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
3, 6\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
400\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
0.2062\strut
\end{minipage}\tabularnewline
\begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
Myster (2007)\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
352\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
Peru\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
flooded rainforest\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
1\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
25\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
1.436\strut
\end{minipage}\tabularnewline
\begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
Bradford et al. (2014)\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
356\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
Queensland Australia\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
mature rainforest\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
varies\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
5000\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
0.0092\strut
\end{minipage}\tabularnewline
\begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
Thorn et al. (2016)\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
465\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
Bavaria Germany\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
Vascular plants after windfall\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
1\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
200\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
0.078\strut
\end{minipage}\tabularnewline
\begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
Thorn et al. (2016)\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
548\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
Bavaria Germany\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
Vascular plants after windfall\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
1\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
200\strut
\end{minipage} & \begin{minipage}[t]{0.12\columnwidth}\raggedright\strut
0.0705\strut
\end{minipage}\tabularnewline
\bottomrule
\end{longtable}

\section*{References}

{\label{references}}

Baldwin, B. G., S. Boyd, B. J. Ertter, R. W. Patterson, T. J. Rosatti,
and D. Wilken. 2002. The Jepson desert manual: vascular plants of south
eastern California. University of California Press, Berkeley, Los
Angeles, London.

Blowes, S. A., S. R. Supp, L. H. Ant\selectlanguage{ngerman}ão, A. Bates, H. Bruelheide, J. M.
Chase, F. Moyes, A. Magurran, B. McGill, and I. H. Myers-Smith. 2019.
The geography of biodiversity change in marine and terrestrial
assemblages. Science \textbf{366} :339-345.

Bradford, M. G., H. T. Murphy, A. J. Ford, D. L. Hogan, and D. J.
Metcalfe. 2014. Long\selectlanguage{english}-term stem inventory data from tropical rain forest
plots in Australia: Ecological Archives E095\selectlanguage{english}-209. Ecology\textbf{95}
:2362-2362.

Chase, J. M., B. J. McGill, P. L. Thompson, L. H. Antao, A. E. Bates, S.
A. Blowes, M. Dornelas, A. Gonzalez, A. E. Magurran, and S. R. Supp.
2019. Species richness change across spatial scales. Oikos\textbf{128}
:1079-1091.

Cover, T. M., and J. A. Thomas. 2012. Elements of information theory.
John Wiley \& Sons.

Dornelas, M., L. H. Antao, F. Moyes, A. E. Bates, A. E. Magurran, D.
Adam, A. A. Akhmetzhanova, W. Appeltans, J. M. Arcos, and H. Arnold.
2018. BioTIME: A database of biodiversity time series for the
Anthropocene. Global Ecology and Biogeography \textbf{27} :760-786.

Dornelas, M., N. J. Gotelli, H. Shimadzu, F. Moyes, A. E. Magurran, and
B. J. McGill. 2019. A balance of winners and losers in the Anthropocene.
Ecology Letters \textbf{22} :847-854.

Ernest, S. M., T. J. Valone, and J. H. Brown. 2009. Long\selectlanguage{english}-term monitoring
and experimental manipulation of a Chihuahuan Desert ecosystem near
Portal, Arizona, USA: Ecological Archives E090\selectlanguage{english}-118. Ecology\textbf{90}
:1708-1708.

Farah, F., R. Rodrigues, F. Santos, J. Tamashiro, G. Shepherd, T.
Siqueira, J. Batista, and B. Manly. 2014. Forest destructuring as
revealed by the temporal dynamics of fundamental species--case study of
Santa Genebra Forest in Brazil. Ecological Indicators \textbf{37}
:40-44.

Frank, S. A. 2012. Natural selection. IV. The Price equation. Journal of
evolutionary biology \textbf{25} :1002-1019.

Frank, S. A., and W. Godsoe. 2020. The generalized Price equation:
forces that change population statistics. Frontiers in Ecology and
Evolution \textbf{8} :240.

Godsoe, W., E. K. Eisen, D. Stanton, and K. M. Sirianni. 2021. Selection
and biodiversity change. Theoretical Ecology:367--379.

Godsoe, W., E. Moltchanova, and P. J. Bellingham. 2022. Disentangling
niche theory from biodiversity change. The American
Naturalist\textbf{199} :510-522.

Godsoe, W., R. Murray, and R. Iritani. 2023. Species interactions and
diversity: a unified framework using Hill numbers. Oikos\textbf{2023}
:e09282.

Hillebrand, H., B. Blasius, E. T. Borer, J. M. Chase, J. A. Downing, B.
K. Eriksson, C. T. Filstrup, W. S. Harpole, D. Hodapp, and S. Larsen.
2018. Biodiversity change is uncoupled from species richness trends:
Consequences for conservation and monitoring. Journal of Applied
Ecology\textbf{55} :169-184.

Holland, J. N., and D. L. DeAngelis. 2009. Consumer-resource theory
predicts dynamic transitions between outcomes of interspecific
interactions. Ecology Letters \textbf{12} :1357-1366.

Jost, L. 2006. Entropy and diversity. Oikos \textbf{113} :363-375.

Jost, L. 2007. Partitioning diversity into independent alpha and beta
components. Ecology \textbf{88} :2427-2439.

Kerr, B., and P. Godfrey\selectlanguage{english}-Smith. 2009. Generalization of the Price
equation for evolutionary change. Evolution \textbf{63} :531-536.

Lightfoot, D. 2011. Small Mammal Exclosure Study (SMES) Vegetation Data
from the Chihuahuan Desert Grassland and Shrubland at the Sevilleta
National Wildlife Refuge, New Mexico (2006-2009).

MacArthur, R. H. 1965. Patterns of species diversity. Biological
Reviews\textbf{40} :510-533.

McGill, B. J., M. Dornelas, N. J. Gotelli, and A. E. Magurran. 2015.
Fifteen forms of biodiversity trend in the Anthropocene. Trends in
ecology \& evolution \textbf{30} :104-113.

Patil, G., and C. Taillie. 1982. Diversity as a concept and its
measurement. Journal of the American statistical Association\textbf{77}
:548-561.

Price, G. R. 1970. Selection and covariance. Nature\textbf{227}
:520-521.

Rice, S. H. 2004. Evolutionary theory: mathematical and conceptual
foundations. Sinauer Associates Sunderland.

Roswell, M., J. Dushoff, and R. Winfree. 2021. A conceptual guide to
measuring species diversity. Oikos \textbf{130} :321-338.

Sax, D. F., S. D. Gaines, and J. H. Brown. 2002. Species invasions
exceed extinctions on islands worldwide: a comparative study of plants
and birds. The American Naturalist \textbf{160} :766-783.

Socolar, J. B., J. J. Gilroy, W. E. Kunin, and D. P. Edwards. 2016. How
should beta-diversity inform biodiversity conservation? Trends in
ecology \& evolution \textbf{31} :67-80.

Tatsumi, S., R. Iritani, and M. W. Cadotte. 2021. Temporal changes in
spatial variation: partitioning the extinction and colonisation
components of beta diversity. Ecology Letters \textbf{24} :1063-1072.

Tatsumi, S., J. Strengbom, M. \selectlanguage{polish}Č\selectlanguage{english}ugunovs, and J. Kouki. 2020. Partitioning
the colonization and extinction components of beta diversity across
disturbance gradients. Ecology \textbf{101} :e03183.

Thorn, S., C. B\selectlanguage{ngerman}ässler, M. Bernhardt\selectlanguage{english}-Romermann, M. Cadotte, C. Heibl, H.
Schafer, S. Seibold, and J. Muller. 2016. Changes in the dominant
assembly mechanism drive species loss caused by declining resources.
Ecology Letters \textbf{19} :163-170.

Urban, M. C. 2015. Accelerating extinction risk from climate change.
Science \textbf{348} :571-573.

Urban, M. C., G. Bocedi, A. P. Hendry, J.-B. Mihoub, G. Pe'er, A.
Singer, J. Bridle, L. Crozier, L. De Meester, W. Godsoe, A. Gonzalez, J.
J. Hellmann, R. D. Holt, A. Huth, K. Johst, C. B. Krug, P. W. Leadley,
S. C. F. Palmer, J. H. Pantel, A. Schmitz, P. A. Zollner, and J. M. J.
Travis. 2016. Improving the forecast for biodiversity under climate
change. Science \textbf{353} :aad8466.

Vellend, M. 2010. Conceptual synthesis in community ecology. The
Quarterly review of biology \textbf{85} :183-206.

Vellend, M. 2016. The theory of ecological communities (MPB-57).
Princeton University Press.

Vellend, M., L. Baeten, A. Becker-Scarpitta, V. Boucher-Lalonde, J. L.
McCune, J. Messier, I. H. Myers-Smith, and D. F. Sax. 2017. Plant
biodiversity change across scales during the Anthropocene. Annual Review
of Plant Biology \textbf{68} .

Vellend, M., L. Baeten, I. H. Myers-Smith, S. C. Elmendorf, R.
Beausejour, C. D. Brown, P. De Frenne, K. Verheyen, and S. Wipf. 2013.
Global meta-analysis reveals no net change in local-scale plant
biodiversity over time. Proceedings of the National Academy of
Sciences\textbf{110} :19456-19459.

Venturoli, F., J. M. Felfili, and C. W. Fagg. 2011. Temporal evaluation
of natural regeneration in a semideciduous secondary forest in
Pirenopolis, Goias, Brazil. Revista Arvore \textbf{35} :473-483.

Webb, S. L., and S. E. Scanga. 2001. Windstorm disturbance without patch
dynamics: twelve years of change in a Minnesota forest.
Ecology\textbf{82} :893-897.

Wood, S., and M. S. Wood. 2015. Package `mgcv'. R package
version\textbf{1} :29.

\selectlanguage{english}
\FloatBarrier
\end{document}

