{'fraction_non_alphanumeric': 0.028288656489161356, 'fraction_numerical': 0.009937091797892374, 'mean_word_length': 4.901631591392764, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 1, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
A scale-dependent measure of system dimensionality

Summary A fundamental problem in science is uncovering the effective number of degrees of freedom in a complex system: its dimensionality. A system’s dimensionality depends on its spatiotemporal scale. Here, we introduce a scale-dependent generalization of a classic enumeration of latent variables, the participation ratio. We demonstrate how the scale-dependent participation ratio identifies the appropriate dimension at local, intermediate, and global scales in several systems such as the Lorenz attractor, hidden Markov models, and switching linear dynamical systems. We show analytically how, at different limiting scales, the scale-dependent participation ratio relates to well-established measures of dimensionality. This measure applied in neural population recordings across multiple brain areas and brain states shows fundamental trends in the dimensionality of neural activity—for example, in behaviorally engaged versus spontaneous states. Our novel method unifies widely used measures of dimensionality and applies broadly to multivariate data across several fields of science.


In brief
We present a scale-dependent dimensionality analysis that reveals the effective degrees of freedom in a complex system at all scales. We illustrate how, when applied to various types of data, the scale-dependent participation ratio exposes interesting geometrical patterns. Our method incorporates widely known dimensionality measures and is applicable to multivariate data in a wide range of scientific areas.

INTRODUCTION
In many branches of science, complex systems are characterized by simultaneous values of many observables evolving over time. For example, the operational state of a living cell may be summarized by the expression levels of myriad pro-teins. Likewise, the instantaneous activity levels of the many neurons in a brain region summarize its state. 1 The dynamics of these systems can be much lower dimensional. For example, at the coarsest scale, the overall dynamics of a brain area may be described just by slow fluctuations in the mean neural firing rate, i.e., a single dynamical variable. 2,3 THE BIGGER PICTURE Data mining is based on the discovery of structure within data. However, such a structure is often complex. The fact that the properties of data distributions vary depending on the scale at which they are examined is a fundamental component of this complexity. For example, a manifold may appear smooth at small scales but jagged or even fractal at larger scales. This scale dependence is critical, yet it is commonly overlooked. We introduce a fundamental approach for analyzing the properties of data distributions at all scales. This single scale-dependent description enables simultaneous examination of how characteristics vary across all scales, offering insight into the structure of the data distribution. This will help us gain a better grasp of data structures and pave the way for future theoretical advances in data science.
At an intermediate scale, the same dynamics could consist of several characteristic firing patterns evolving smoothly on a fixed d-dimensional manifold embedded in the state space. If we knew the underlying dynamical system, we could derive the relevant manifold at each scale from first principles. But in many of the most exciting complex systems becoming accessible to experimental study, our goal is to discover the dynamical system, a task that starts by determining the number of effective latent variables, i.e., the dimensionality of the system.
One approach to this problem has roots in point-set topology: a manifold is d dimensional if the number of uniformly sampled points in a region of characteristic length L scales as L d . This fact leads to definition of the capacity dimension D 0 4 in terms of the number n of Euclidean boxes of side length e needed to cover the system's trajectory in its embedding space: log n logð1=eÞ . This intuitive quantity is difficult to compute in more than three dimensions, 5 while sampling of dynamical systems is often too coarse to directly estimate the e/ 0 limit. [6][7][8] A variation on this idea, easier to estimate in high dimensions, is the correlation dimension D corr , 9 determined from the scaling of the number of pairs of data points with separations less than r, in the r/0 limit. In this way, the capacity dimension and the correlation dimension both give local, fine-scale measures of dimension. 9,10 A second class of approaches starts with the correlation matrix between observations. For example, techniques related to principal-component analysis define the effective dimension as the number of eigenmodes of the correlation matrix that capture most of the variance. Substantial literature analyzes how to choose the threshold that defines ''most,'' 4,11 but the resulting arbitrariness makes it challenging to define the dimension associated to different scales of observation. 12,13 A dimension can be defined more naturally from the participation ratio (PR), 1,14 which counts the effective dimensions along which data are spread as a ratio of the square of the first moment and the second moment of the eigenvalue probability density function. 15 Here, we generalize this notion to a measure of effective dimension at different observation scales and show that it interpolates between correlation dimension at small scales and PR dimension globally. We show that the new quantity has intuitive meanings when applied to dynamical systems including the Lorenz attractor and clustered systems like hidden Markov models. We then apply our method to elucidate the structure of neural population activity in different brain areas and states.

RESULTS
Generalizing the notion of dimension to all scales Consider T observations of N observables xðiÞ, i = 1; .T sampled from a data distribution M. For example, xðiÞ could be generated by a dynamical process xðt + tÞ = F ðxðtÞÞ with specified initial conditions. The empirical covariance matrix S over xðiÞ is S = 1 T À 1 P T i = 1 ðxðiÞ À xÞðxðiÞ À xÞ u , an N3 N matrix with x = 1 T P T i xðiÞ. The eigenvalues of S are l i = 1::N , and the associated spectral density is rðlÞ. The PR dimension, D PR , is defined as the ratio between the second and the first moments of the spectral density: 15 D PR measures the concentration of the eigenvalue distribution and quantifies how many eigenmodes are needed to substantially capture the data distribution, a similar notion to counting eigenmodes (or principal components) that capture most of the variance.
To extend D PR to a scale-dependent measure of dimension, we first consider a ball of radius r, B i ðrÞ, around a point xðiÞ.
The local covariance matrix of points within this ball is an effective dimension up to scale r. Evaluating D PR ðrÞ for the Lorenz attractor and the noisy two-dimensional (2D) spiral 4 illustrates its scale-dependent properties. For the Lorenz attractor, D PR ðrÞ is roughly 2 across scales ( Figure 1B), reflecting dense coverage of a 2-manifold by this chaotic system and agreeing at small scales with the Lyapunov dimension arising from its dynamics. 16 For the spiral, D PR ðrÞ starts from 2 for small r, reflecting the spread of local noise, and then dips to approach $1, reflecting the line making the spiral, and returns to 2 at large scales r, reflecting the overall embedding of the spiral ( Figure 1E).
Alternatively, the correlation dimension can be generalized across scales. 17,18 Let d ij = x i À x j be distances between samples, and define the correlation integral at distance r as Here, H is the Heaviside step function, and PðxÞ is the distribution of pairwise distances. We expect that CðrÞfr d , for small r, where d is the dimension of the manifold supporting the data M. Then, D corr is defined as D corr = lim r/0 log CðrÞ log r . Although D corr is defined as r/0, it is sometimes extended to general r as the 4 To overcome sampling constraints as r/0, one seeks a plateau in D corr ðrÞ 17 at small but finite r, a valid approach if the dimension is relatively stable across a range of scales. This method effectively treats low-dimensional strange attractors, 16 but sampling remains a challenge for high-dimensional manifolds, and the problem is compounded if the effective dimension varies with scale. 6,7,19 Likewise, for any bounded dataset, D corr /0 at large r simply because the data look point-like at large scales. (Technically, the correlation integral CðrÞ = R r 0 PðxÞdx/1 by construction at large r, so its derivative vanishes.) Indeed, applying D corr ðrÞ to the Lorenz attractor, we recover the expected dimension just bigger than 2 at small scales, but the value declines to zero at large scales at which the compact attractor is effectively point-like ( Figure 1C). By contrast, D PR ðrÞ ( Figure 1B) remains roughly constant across scales, as it performs a kind of adaptive rescaling (via the denominator in Equation 2.1). Similarly, D corr ðrÞ declines to zero at large scales for the noisy spiral ( Figure 1F), while the PR dimension ( Figure 1E) stably captures the essentially 2D large-scale structure. Note that D corr ðrÞ also produces difficult-to-interpret oscillations at intermediate scales.
At small scales, we can derive a universal expression for the PR D PR ðr /0Þ via a tangent space approximation to the data manifold that enables a new link between the PR and the correlation dimension. Approximating the local distribution as a Gaussian, MzM Gauss = ð2pÞ À N=2 Sj À 1=2 e À 1 2 ðx À mÞS À 1 ðx À mÞ u , where m is the mean and S is the covariance. Rotating and centering the data so that mh0 and S is diagonal, each component of the sampled vectors will be normally distributed: x a $ N ð0; l a Þ, where l a is an eigenvalue of S. Thus, in this limit, In the same limit, the correlation dimension is determined by the distribution of squared Euclidean distances between sampled points. For any pair, the distance squared is a sum of squared separations in each coordinate. Since the coordinates are Gaussian distributed, the squared separations are Gamma distributed: Gðk; q a ÞðxÞ = x k À 1 e À x=qa q k a GðkÞ , with x > 0 where k = 1=2 and q a = 2l a for the a th coordinate direction. The (E) The sdPR dimension is 2 at small scales reflecting spread due to noise, approaches 1 at intermediate scales reflecting the line, and 2 at large scales reflecting the coarse-grained spiral. (F) The scale-dependent correlation dimension interpolates non-montonically between 2 and 0. Details of the sdPR are in Section S1.
convolution of independent Gamma distributions for each coordinate gives the overall distribution of squared Euclidean distances between sampled points as an approximate Gamma distribution with parameters ðk s ; q s Þ given by the Welch-Satterthwaite equation [20][21][22][23] simplifying here a qa . Exploiting this formula to compute the correlation integral in the r/0 limit (cf. Section S3) yields Thus, the sdPR coincides with the correlation dimension, a well-accepted local notion of dimension, at small scales. In Section S2 and Figure S1, we also relate sdPR to the Renyi dimension for small scales. We will next show in several tractable models that the sdPR-unlike the correlation and Renyi dimensions-generalizes naturally across scales.

Dimensionality across scales
We numerically computed D corr ðrÞ and D PR ðrÞ for isotropic multidimensional Gaussians. As shown in Figures 2A and  S2A, these examples illustrate the relationship between sdPR and the correlation dimension. While D corr decreases at larger scales, D PR remains constant. Furthermore, we see how, limiting the sampled statistics to 50,000 points, it is not possible to achieve a plateau in D corr at small scales, even for d = 10-this means that capturing the dimensionality of the system from the correlation dimension in Figure 2A (left) is difficult, if not impossible. In the case of 2D Gaussians with increased elongation along one of the coordinate axes ( Figures 2B and S2B), the correlation dimension accurately quantifies the local two dimensions in the regime r/0 but fails to quantify the distribution's skewness, which effectively makes it 1D at large scales. In comparison, D PR quantifies both the local two dimensions and the skewness, which results in a reduced dimension at larger scales. The correlation dimension captures well the local two dimensions in the regime r/0 but fails to quantify the skewness of the distribution, which makes it effectively 1D at large scales. By contrast, D PR quantifies both the local two dimensions and the skewness, which induces a lower dimension at larger scales.
A key further test of the sdPR is to understand how it behaves in scale-free systems. Thus, we considered multidimensional Gaussians whose covariance matrices have power-law-distributed eigenvalues, resembling structures near criticality. In these cases, sdPR equals the full rank of the covariance matrix at small scales and declines at large scales ( Figures 2C and S2C). The global PR (Equation 2.1), at large scales, can be quantitatively captured; with a scale-free spectrum l a = ba À a , we find ; where zðaÞ is the Riemann zeta function. This global dimension declines monotonically with the scaling exponent a: larger a creates a more skewed distribution that is effectively lower dimensional when viewed at large scales ( Figure 2C, right, inset). 24 In contrast to this consistent behavior of the sdPR, the correlation dimension declines to zero at large scales, failing to capture the data dimension (Figure 2C, left). Section S3 of the supplemental experimental procedures contains another scale-free example, as well as an example based on receptive field maps of neural populations (see also Figure S3).

Limitations of the study
The comparison between sdPR and correlation dimension highlighted advantages of sdPR over the correlation dimension. However, there also some limitations in evaluating sdPR, especially concerning its computational complexity.
The primary advantage of sdPR over correlation dimension is that it gives an estimate of the dimensionality across all scales. The disadvantages stem from the fact that D PR ðrÞ is a local point-wise dimensionality estimator: D PR ðrÞ is calculated as the average of multiple data points surrounding each ball of radius r (see Equation 2.2). Both the sdPR and correlation dimensions require the computation of second order statistics: the covariance matrix S for the sdPR and the matrix of squared distances for the correlation dimension. These two quantities have the same computational cost since they are related: D ij = S ii + S jj À 2S ij . Their computational cost scales as OðNT 2 Þ, where T is the number of data points and N is the dimensionality of the data. Thus, the increased computational cost of sdPR coming from the average over points is the main difference between the costs of the correlation dimension's and sdPR's.
However, the computational cost of sdPR, arising from the point-wise average in its definition, can be limited in practice, as the estimation around independent points need not be carried out over all points but just on a relatively small subset of repeats (we use N repeats = 100). An analysis of the bias over independent repetitions yields a fast decay of the estimation bias as 1=N repeats (cf. Section S3.4 and Dahmen et al. 25 ). Given this rapid convergence, it is enough to select 1 ( N repeats ( T when T is in the order of thousands or higher.

OPEN ACCESS
Article A second consequence of sdPR arising from a local point-wise estimator is that the value of the minimum scale accessible to sdPR is higher than for the correlation dimension. The minimum distance for which the correlation dimension can be computed is r corr min = min xi;xj˛M ðd ij Þ. Meanwhile, D PR ðrÞ computes the distance only within a ball of radius r around a subset of initial points and then averages over the initial points. Therefore, the minimum scale for sdPR is given by r sdPR min = max xk˛M ðmin xi ;xj˛Bk ðrÞ ðd ij ÞÞ. Thus, the correlation dimension is more granular than the sdPR dimension, r corr min % r sdPR min . Despite this apparent limitation, however, in practice, sdPR often appears to have a faster convergence to the true dimensionality at small scales, especially for higherdimensional manifolds (cf. Figure 2A).

A measure of the local and global dimension
We next show how the sdPR can separate dimensionality driven by local dynamics that are partly influenced by noise and global dynamics that are dominated by the latent structure of a dynamical system. To illustrate, we considered data generated by an underlying hidden Markov model (HMM) with dynamics hopping between latent states (Figures 3A and S4; Section S4). The system is observed through measurements of noisy ''emissions'' distributed around well-separated means associated with each latent state. Locally, the dimension should be determined by the statistical spread of the data and hence equal the dimension of the emission noise in the observation space. Globally, the dimension should be related to the number of latent states and the separation of their emissions, relative to the magnitude and structure of the noise. The sdPR uncovers precisely this structure (example in Figure 3B; further details and examples in Section S4) and indicates a characteristic scale for transitioning between local and global dimensions, set by separability of the emission distributions.
For many applications, we must contend with the challenge that a ''state'' of the system may itself involve a characteristic dynamical trajectory with changing internal variables. Thus, we studied the sdPR of switching linear dynamical systems (SLDSs) (Figures 3C and S5; Section S4), in which the latent states of an HMM describe dynamical trajectories following an ordinary differential equation (ODE). We found that, for intermediate scales r, D PR ðrÞ identifies the system as 1D, which reflects the fact that trajectories in each state follow deterministic lowdimensional dynamics. Figure 3D demonstrates this for the case of switching rotating dynamics; the increase in dimensionality at the very smallest scales reflects the addition of emission noise. Thus, in this case, similar to Figures 1D and 1E, the local dimension is driven by stochastic observations, the global dimension is driven by the overall geometry of the dynamical system, and at intermediate scales, the structure of dynamical trajectories is revealed.
The dimensionality of neuronal data We next applied our sdPR method to recordings of the simultaneous activity of thousands of neurons, 26 made possible by recent technological advances. 27,28 Others have examined the global dimension of the underlying systems; 1,24,29 here, we use sdPR to reveal differences in latent dynamics across scales, in different brain areas and states. Thus, we analyzed 37 Neuropixels recordings in mice from two sensory areas (visual thalamus and cortex) and two decision areas (frontal cortex and midbrain). 26,30,31 Animals were either in a ''spontaneous'' state (awake, no task) or in an ''engaged'' state (performing two alternative forced-choice tasks; Section S5). 26 We binned neural spike counts in 100 ms windows and subdivided neurons into groups of 100 to compare across sessions sampling variable numbers of units. Intriguingly, we first found that the sdPR depended systematically on scale in the visual cortex, midbrain, and frontal cortex-decreasing at larger scales (D PR ðr min Þ=D PR ðr max Þ = 1.63 ± 0.05 mean ± SEM)-but was roughly scale invariant in the thalamus ( Figure 4A). Here, distances between neural response vectors quantify scale in the functional space of neural activity, rather than physically on the cortical sheet, suggesting that activity in deeper areas is structured in groups of similar patterns spread broadly over lowerdimensional functional manifolds, like our models with skewed or scale-invariant data covariance. The thalamus, a peripheral sensory area directly reflecting visual input, instead shows scaleinvariant dimensionality, like our models with isotropic data covariance, and recalling the spatial scale invariance of natural images. 32 Moreover, the closeness of the thalamus to the input may explain its significantly higher dimension than the visual cortex at every scale (see Section S5 and Figures S6 and S7), despite massive expansion in the number of neurons involved in cortical as opposed to thalamic representation. 33 Interestingly, the frontal cortex, a key player in planning and executive control, had higher dimensionality than the midbrain in engaged, but not spontaneous, conditions-prominently at smaller scales accessible to our analysis despite limited data, as shown in Figure 4B.
To assess dimensionality changes within brain areas but across behavioral states, we computed differences DD AB PR = D A PR À D B PR between engaged and spontaneous conditions ( Figures 4C, S6, and S7), finding significant deviation in the frontal cortex and midbrain, which are involved in decision making, but not in sensory areas. Specifically, task-driven neural activity was lower dimensional, especially at small and intermediate scales, recalling Mazzucato et al. 34 We also considered a ''passive'' condition in which decision task stimuli were presented in randomized order without eliciting behaviors. The difference between spontaneous and passive was not significantly different from zero ( Figure 4C, green), demonstrating that task engagement influences response dimension.
Interpreting our results on neural data (Figure 4) in light of the dynamical model results in Figure 3 leads to hypotheses about the underlying system dynamics. For example, in the engaged condition, neurons in the frontal cortex and midbrain display reduced dimensionality at intermediate scales. This suggests reduced complexity of the underlying latent dynamics (number of states or latent space dimensionality in HMM and SLDS (C) Difference in dimensionality between engaged and spontaneous (green) and between spontaneous and passive conditions (yellow). Dimensionality modulation D AB between conditions. Shading = 95% confidence interval (CI). Extended analysis in Figures S6 and S7. models) in task-driven conditions, aligning with intuitive notions of focus on a prescribed sensorimotor behavior.

DISCUSSION
Many fields employ analogs of the global PR as a measure of effective intrinsic dimension. In physics, this ratio was first introduced in atomic spectroscopy 35,36 and then used as a measure of localization in condensed matter. 15 In quantum information, a similar quantity is called ''purity'' and measures the degree of mixedness of states. In economics and sociology, the Herfindahl-Hirschman Index measures market concentration of an industrial sector. 37,38 In sociology, the related Simpson Index quantifies diversity, 39 while in politics, it is a measure of the effective number of parties. 40 In machine learning, the same quantity serves as a metric of expressivity for learning kernels, 41 and in neuroscience, it measures the global dimension of neural activity. 1,25,34 Thus, the PR is used to quantify dimension in a remarkably wide variety of disciplines. However, complex systems behave differently at different scales-and thus their dimension is not necessarily characterized by a single number. Our D PR ðrÞ measures a ''running dimension,'' capturing the effective number of latent degrees of freedom required to summarize observables at different scales. Importantly, it also approaches the wellknown correlation dimension at the smallest scales. Moreover, the sdPR dimension can be computed as a simple, exact functional of a system's second-order statistics, and can be derived analytically in many cases (e.g., Dahmen et al. 25 and Hu and Sompolinsky 42 ). Overall, our approach may be used to analyze multivariate data in a range of domains, and we expect that it will reveal new geometrical aspects of data across many fields.

EXPERIMENTAL PROCEDURES
Resource availability Lead contact Further information and requests for resources should be directed to and will be fulfilled by the lead contact, Stefano Recanatesi (stefano.recanatesi@ gmail.com).

Materials availability
This study did not generate new unique reagents or materials. Data and code availability Original code has been deposited at https://github.com/ubcbraincircuits/ accelnet-jupyternotebooks-dimensionality and is publicly available as of the date of publication. Any additional information required to reanalyze the data reported in this paper is available from the lead contact upon request. This paper analyzes existing, publicly available electrophysiology data deposited in Neurodata Without Borders (NWB) format at https://figshare.com/articles/ steinmetz/9598406.