{'fraction_non_alphanumeric': 0.044536192524450166, 'fraction_numerical': 0.007896265364750946, 'mean_word_length': 4.308057661861511, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 24, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 19, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Channel Capacity under Sub-Nyquist Nonuniform Sampling

This paper investigates the effect of sub-Nyquist sampling upon the capacity of an analog channel. The channel is assumed to be a linear time-invariant Gaussian channel, where perfect channel knowledge is available at both the transmitter and the receiver. We consider a general class of right-invertible time-preserving sampling methods which include irregular nonuniform sampling, and characterize in closed form the channel capacity achievable by this class of sampling methods, under a sampling rate and power constraint. Our results indicate that the optimal sampling structures extract out the set of frequencies that exhibits the highest signal-to-noise ratio among all spectral sets of measure equal to the sampling rate. This can be attained through filterbank sampling with uniform sampling at each branch with possibly different rates, or through a single branch of modulation and filtering followed by uniform sampling. These results reveal that for a large class of channels, employing irregular nonuniform sampling sets, while typically complicated to realize, does not provide capacity gain over uniform sampling sets with appropriate preprocessing. Our findings demonstrate that aliasing or scrambling of spectral components does not provide capacity gain, which is in contrast to the benefits obtained from random mixing in spectrum-blind compressive sampling schemes.


I. INTRODUCTION
The capacity of analog Gaussian channels and their capacity-achieving transmission strategies were pioneered by Shannon [2], which has provided fundamental insights for modern communication system design. Shannon's work focused on capacity of analog channels sampled at or above twice the channel bandwidth. However, these results do not explicitly account for sub-Nyquist sampling rate constraints that may be imposed by hardware limitations. This motivates exploration of the effects of sub-Nyquist sampling upon the capacity of an analog Gaussian channel, and the fundamental capacity limits that result when considering general sampling methods that include irregular nonuniform sampling.

A. Related Work and Motivation
Shannon introduced and derived the information theoretic metric of channel capacity for time-invariant analog waveform channels [2], which established the optimality of water-filling power allocation based on signal-to-noise ratio (SNR) over the spectral domain [3], [4]. A key idea in determining the analog channel capacity is to convert the continuous-time channel into a set of parallel discrete-time channels based on the Shannon-Nyquist sampling theorem [5]. This paradigm was employed, for example, by Medard et. al. to bound the maximum mutual information in time-varying channels [6], [7], and was used by Forney et. al. to investigate coding and modulation for Gaussian channels [8]. Most of these results focus on the analog channel capacity commensurate with uniform sampling at or above the Nyquist rate associated with the channel bandwidth. There is another line of work that characterizes the effects upon information rates of oversampling with quantization [9], [10]. In practice, however, hardware and power limitations may preclude sampling at the Nyquist rate for a wideband communication system.
More general irregular sampling methods beyond pointwise uniform sampling have been extensively studied in the sampling literature, e.g. [11]- [13]. One example is sampling on non-periodic quasi-crystal sets, which has been shown to be stable for bandlimited signals [14], [15]. These sampling approaches are of interest in some realistic situations where signals are only sampled at a nonuniformly spaced sampling set due to constraints imposed by data acquisition devices. Many sophisticated reconstruction algorithms have been developed for the class of bandlimited signals or, more generally, the class of shift-invariant signals [11], [16], [17]. For all these nonuniform sampling methods, the Nyquist sampling rate is necessary for perfect recovery of bandlimited signals [11], [18], [19].
However, for signals with certain structure, the Nyquist sampling rate may exceed that required for perfect signal reconstruction from the samples [20], [21]. For example, consider multiband signals, whose spectral contents reside within several subbands over a wide spectrum. If the spectral support is known, then the necessary sampling rate for the multiband signals is their spectral occupancy, termed the Landau rate [22]. Such signals admit perfect recovery when sampled at rates approaching the Landau rate, provided that the sampling sets are appropriately chosen (e.g. [23], [24]). One type of sampling mechanism that can reconstruct multiband signals sampled at the Landau rate is a filter bank followed by sampling, studied in [25]- [27]. Inspired by recent "compressive sensing" [28]- [30] ideas, spectrum-blind sub-Nyquist arXiv:1204.6049v5 [cs.IT] 22 Mar 2014 sampling for multiband signals with random modulation has been developed [31] as well.
Although sub-Nyquist nonuniform sampling methods have been extensively explored in the sampling literature, they are typically investigated either under a noiseless setting, or based on statistical reconstruction measures (e.g. mean squared error (MSE)) instead of information theoretic measures. Gastpar et al [32] studied the necessary sampling density for nonuniform sampling. Recent work by Wu and Verdu [33] investigated the tradeoff between the number of samples and the reconstruction fidelity through information theoretic measures. However, these works did not explicitly consider the capacity metric for an analog channel. The most relevant capacity result to our work was by Berger et al [34], who related MSE-based optimal sampling with capacity for several special types of channels. But they did not derive the sub-Nyquist sampled channel capacity for more general channels, nor did they consider nonuniformly spaced sampling. Our recent work [35] established a new framework that characterizes sampled capacity for a broad class of sampling methods, including filter and modulation bank sampling [31], [36], [37]. For these sampling methods, we determined optimal sampling structures based on capacity as a metric, illuminated intriguing connections between MIMO channel capacity and capacity of undersampled channels, as well as a new connection between capacity and MSE. However, this prior work did not investigate analog channel capacity using more general nonuniform sampling under a sub-Nyquist sampling rate constraint.
One interesting fact discovered in [35] is the nonmonotonicity of capacity with sampling rate under filter-and modulation-bank sampling, assuming an equal sampling rate per branch for a given number of branches. This indicates that more sophisticated sampling techniques, adaptive to the channel response and the sampling rate, are needed to maximize capacity under sub-Nyquist rate constraints, including both uniform and nonuniform sampling. However, none of the aforementioned work has investigated the question as to which sampling method can best exploit channel structure, thereby maximizing sampled capacity under a given sampling rate constraint. Although several classes of sampling methods were shown in [35] to have closed-form capacity solutions, the capacity limits might not even exist for general sampling methods. This raises the question as to whether there exists a capacity upper bound over a general class of sub-Nyquist sampling systems beyond the classes we discussed in [35] and, if so, when the bound is achievable. That is the question we investigate herein.

B. Contributions and Organization
Our main contribution is to derive the capacity of sub-Nyquist sampled analog channels for a general class of rightinvertible time-preserving nonuniform sampling methods, under a sub-Nyquist sampling rate constraint. The channel is assumed to be a linear time-invariant (LTI) Gaussian channel, where perfect channel knowledge is available at both the transmitter and the receiver. The class of sampling systems we consider subsumes sampling structures employing irregular nonuniform sampling grids.
We first develop in Theorem 2 an upper bound on the sampled channel capacity, which corresponds to the capacity of a channel whose spectral occupancy is no larger than the sampling rate f s . As a key step in the analysis framework for Theorem 2, we characterize in closed form the sampled channel capacity for any specific periodic sampling system, formally defined in Definition 8 (Lemma 1). We demonstrate that this fundamental capacity limit can be achieved by filterbank sampling with varied sampling rates at different branches, or by a single branch of modulation and filtering followed by a uniform sampling set (Theorems 3-4). In particular, the optimal sampler extracts out a spectral set of size f s with the highest SNR, and suppresses all signal and noise components outside this spectral set.
Our results indicate that irregular nonuniform sampling sets, while typically complicated to realize in hardware, do not increase channel capacity relative to analog preprocessing with regular uniform sampling sets. We also show that when optimal filterbank or modulation sampling is employed, a mild perturbation of the optimal sampling grid does not change the capacity. Our findings demonstrate that aliasing or scrambling of spectral contents does not provide capacity gain. This is in contrast to the benefits obtained from random mixing of frequency components in many sub-Nyquist sampling schemes with unknown signal support (e.g. [31]).
The main innovation of this paper compared to our previous sub-sampled channel capacity results in [35] is as follows.
• While [35] characterizes the capacity under two types of sampling mechanisms that are widely used in practice (filter-bank sampling and modulation-bank sampling), the focus of this paper is instead to develop capacity results over a much more general class of sampling methods. • While all results of [35] hold only under uniform sampling, our analysis herein accommodates irregular nonuniform sampling. Our results in turn corroborate the optimality of uniform sampling in achieving sampled capacity, assuming that the analog channel output is appropriately pre-processed.
The remainder of the paper is organized as follows. In Section II, we introduce our system model of sampled analog channels, and provide formal definitions of time-preserving systems, sampling rates, and sampled channel capacity. We then develop, in Section III-A, an upper bound on the sampled channel capacity ranging over all right-invertible timepreserving sampling methods, along with an approximate analysis highlighting insights into the result. The achievability of this upper bound is derived in Section III-B. The proof of Theorem 2 is provided in Appendix A. The implications of our main results are summarized in Section IV.
Before continuing, we introduce some notation that will be used throughout. We use µ (·) to represent the Lebesgue measure, and denote by F and F −1 the Fourier and inverse Fourier transform, respectively. We let [x] + ∆ = max (x, 0), and use card (A) to denote the cardinality of a set A. These and other notation in the paper are summarized in Table I.
impulse response and frequency response of the ith (post-modulation) filter p(t), P (f ) impulse response and frequency response of the pre-modulation filter Sη(f ), sη(t) power spectral density of the noise η(t) and sη (t) := F −1 Sη (f ) fs, Ts aggregate sampling rate and the corresponding sampling interval (Ts = 1/fs) q(t, τ ) impulse response of the sampling system, i.e. the output seen at time t due to an impulse in the input at time τ . Tq, fq period of the modulating sequence q(t) such that Fourier transform and inverse Fourier transform

A. System Model
We consider an analog waveform channel, which is modeled as an LTI filter with impulse response h(t) and frequency re- . With x(t) denoting the transmitted signal, the analog channel output is given by where the noise process η(t) is assumed to be an additive stationary zero-mean Gaussian process with power spectral density S η (f ). We also define s η (t) := F −1 S η (f ) . Unless otherwise specified, we assume throughout that perfect channel state information (i.e. the knowledge of both H(f ) and S η (f )) is available at both the transmitter and the receiver.
The analog channel output r(t) is passed through M (1 ≤ M ≤ ∞) branches of linear preprocessing systems, each followed by a pointwise sampler, as illustrated in Fig.  1. The preprocessed output y i (t) at the ith branch is obtained by applying a linear bounded operator T i to the channel output r(t): where q i (t, τ ) denotes the impulse response of the timevarying system represented by T i , i.e. the output seen at time t due to an impulse in the input at time τ . Note that the linear operator T i can be time-varying, which subsumes filtering and modulation as special cases. For example, a modulation system T i (x(t)) = p(t)x(t) for some given modulation sequence p(t) has an impulse response q(t, τ ) = p(τ )δ (t − τ ). A cascade combination of two systems T 1 and T 2 has an impulse response q(t, τ ) =´∞ −∞ q 2 (t, τ 1 )q 1 (τ 1 , τ )dτ 1 , with q 1 (·, ·) and q 2 (·, ·) denoting respectively the impulse responses of T 1 and T 2 [38].
When an operator T is LTI, we use q(τ ) := q(t, t − τ ) as shorthand to represent its impulse response.
The pointwise sampler following the preprocessor can be uniform or irregular [11]. Specifically, the preprocessed output y i (t) (at the ith branch) is sampled at times t i,n (n ∈ Z), yielding a sample sequence y i [n] = y i (t i,n ). Here, we define the sampling set Λ i at the ith branch as In particular, if t i,n = nT i,s , then the sampling set at the ith branch is said to be uniform with period T i,s .
The input x(t) is passed through the analog channel and contaminated by noise η(t). The analog channel output r(t) is then passed At the ith branch, the preprocessed output y i (t) is sampled on the sampling set Λ i = {t i,n | n ∈ Z}.

B. Sampling Rate Definition
Our metric of interest is the sampled channel capacity under a sampling rate constraint. We first formally define sampling rate for general nonuniform sampling mechanisms.
In general, the sampling set Λ = {t n | n ∈ Z} may be irregular and hence aperiodic, which calls for a generalized definition of sampling rate. One notion commonly used in sampling theory is the Beurling density introduced by Beurling [18] and Landau [22], as defined below [11].
When the sampling set is uniform with period T s , the Beurling density is D(Λ) = 1/T s , which coincides with the conventional definition of sampling rate. The notion of Beurling density allows the Shannon-Nyquist sampling theorem to be extended to nonuniform sampling. Moreover, we will use Beurling density to define sampling rate for a large class of sampling mechanisms with preprocessing.
Under a nonuniform sampling set Λ, the set of exponential functions {exp (j2πt n f ) | n ∈ Z} forms a nonharmonic Fourier series [12]. Whether the class of original signals are recoverable from the nonuniform sampled sequence is determined by the completeness of the associated nonharmonic set. In particular, when Λ is uniform, the set {exp (j2πt n f ) | n ∈ Z, t n = n/f s } with D(Λ) = f s forms a Riesz basis [39] of L 2 (−f s /2, f s /2) by the Shannon-Nyquist sampling theorem. For the class of sampling systems without preprocessing, a fundamental rate limit necessary for perfect reconstruction of bandlimited signals has been characterized by Landau using the definition of Beurling density, as stated in the following theorem.
Theorem 1 (Landau Rate [22]). Consider the set B Ω of all signals whose spectral contents are supported on the frequency set Ω. Suppose that pointwise sampling without preprocessing is employed with a sampling set Λ. If all signals f (t) ∈ B Ω can be uniquely determined by the samples {f (t n ) | t n ∈ Λ}, then one must have D − (Λ) ≥ µ (Ω). The value µ (Ω) is termed the Landau rate.
Theorem 1 characterizes the fundamental sampling rate requirement for perfect signal reconstruction under pointwise sampling without preprocessing. In particular, when Ω = [−B/2, B/2], Theorem 1 reduces to the Shannon-Nyquist theorem.
We will thereby use Beurling density to characterize the sampling rate for a general sampling set. However, since the preprocessor might distort the time scale of the input, the resulting "sampling rate" might not characterize the true sampling rate applied to the original signal, as illustrated in the following example.
Example 1 (Compressor). Consider a preprocessing system defined by the relation y(t) = T (r(t)) = r (Lt) with L ≥ 2 being a positive integer. If we apply a uniform sampling set Λ = {t n : t n = n/f s } on the preprocessed output y(t), the sampled sequence at a "sampling rate" f s is given by y[n] = y (n/f s ) = r (nL/f s ) , which corresponds to sampling the system input r(t) at rate f s /L. The compressor effectively time-warps the signal, thus resulting in a mismatch of the time scales between the input and output.
The compressor example illustrates that the notion of sampling rate may be misleading for systems that experience time warping. Hence, this paper will focus only on sampling that preserves time scales. One class of linear systems that preserves time scales are modulation operators (y(t) = p(t)x(t)), which perform pointwise scaling of the input, and hence do not change the time scale. Another class is the periodic system which includes LTI filtering, defined as follows.
Definition 2 (Periodic System). A linear preprocessing system is said to be periodic with period T q if its impulse response q(t, τ ) satisfies q(t, τ ) = q(t + T q , τ + T q ), ∀t, τ ∈ R.
A more general class of systems that preserve the time scale can be generated through modulation and periodic subsystems.
Specifically, we can define a general time-preserving system by connecting a set of modulation or periodic operators in parallel or in serial. This leads to the following definition.
Definition 3 (Time-preserving System). Given an index set I, a preprocessing system T : x(t) → {y k (t), k ∈ I} is said to be time-preserving if (1) The system input is passed through |I| (possibly countably many) branches of linear preprocessors, yielding a set of analog outputs {y k (t) | k ∈ I}.
(2) In each branch, the preprocessor comprises a set of periodic or modulation operators connected in serial. With a preprocessing system that preserves the time scale, we can now define the aggregate sampling rate through the Beurling density.
Definition 4 (Sampling Rate for Time-preserving Systems).
A sampling system is said to be time-preserving with sampling rate f s if (1) Its preprocessing system T is time-preserving.
(2) The preprocessed output y k (t) is sampled by a sampling set Λ k = {t l,k | l ∈ Z} with a uniform Beurling density f k,s , which satisfies 1 k∈I f k,s = f s . We note that the class of time-preserving sampling structures does not preclude random sampling schemes. For example, the preprocessing system can be a random modulator and the sampling set can be randomly spaced. Our definition also includes multibranch sampling methods. In fact, each multibranch sampling can be converted to an equivalent single branch sampling as follows.
Proposition 1. Suppose that a multibranch sampling system has sampling rate f s . Then there exists a single branch sampling system with sampling rate f s that yields the same set of sampled output values as the original system. This holds simultaneously for all input signals.
Proof: Suppose that the impulse response for the kth branch is given by q k (t, τ ) with sampling set Λ k = {t k,n | n ∈ Z}. Without loss of generality 2 , suppose that Λ k ∩ Λ k = ∅ for any k = k . By ordering all sample times in ∪ k∈I Λ k and renaming them to be t l | l ∈ Z such that t l <t l+1 for all l, we can construct an equivalent single branch sampling system such that ift l corresponds tot k,n in the original sampling set. The sampling ratef s of the new system is given byf s = k∈I D(Λ k ) = f s . 1 We note that the sampling system may comprise countably many branches, each with non-zero sampling rate. For instance, if the kth branch is sampled at a rate f k,s = k −2 f 0 , we have an aggregate rate fs we can introduce certain delay to the preprocessed output and shift the sampling set correspondingly. Apparently, this new sampling structure leads to the same collection of sample outputs.
The samples obtained through this new single branch system preserve all information we can obtain from the samples of the original multibranch system. As will be seen, this proposition allows us to simplify the analysis.

C. Capacity Definition
There are two capacity definitions that are of interest in sub-Nyquist sampled channels: (1) the sampled capacity for a given sampling system; (2) the capacity for a large class of sampling systems under a sampling rate constraint. We now detail these definitions.
Suppose that the transmit signal x(t) is constrained to the time interval [−T, T ], and the received signal y(t) is sampled with sampling rate f s and observed over the time interval [−T, T ]. For a given sampling system P that consists of a preprocessor T and a sampling set Λ, and for a given time duration T , we define the information metric C P T (P ) to be where the supremum is over all input distributions subject to a power constraint The capacity of the undersampled channel under a given sampling system can then be studied by taking the limit as T → ∞. It was shown in [35] that lim T →∞ C P T (P ) exists for a broad class of sampling methods, including sampling via filter banks and sampling via periodic modulation. We caution, however, that the existence of the limit is not guaranteed for all sampling methods, e.g. the limit might not exist for irregular sampling. We therefore define the capacity for a given sampling system as follows.
Definition 5. C P is said to be the information capacity of a given sampled analog channel (or sampled channel capacity) if lim T →∞ C P T (P ) exists and C P (P ) = lim Note that any sampled analog channel can be converted to a set of independent discrete channels via a Karhunen Loeve decomposition. The metric C P (P ) then quantifies asymptotically the maximum mutual information between the input and output of these discrete channels, or equivalently, the maximum data rate that can be conveyed reliably through these channels.
The above capacity is defined for a given sampling mechanism. Another metric of interest is the maximum data rate achievable by all sampling schemes within a general class. This motivates us to define the sub-Nyquist sampled channel capacity for a class of sampling systems as follows.
Definition 6 (Sampled Capacity under A Class of Sampling Systems). C A (f s , P ) is said to be the capacity of an analog channel over all a class A of sampling systems under a given sampling rate f s if The above definition of sub-sampled channel capacity characterizes the capacity limit of an analog channel over a large set of sampling mechanisms subject to a sampling rate constraint. This gives rise to the natural problem of jointly optimizing the input and the sampling architecture to maximize capacity, a goal we address in the next section.

III. MAIN RESULTS
This section characterizes in closed form the sampled channel capacity for a very general class of sampling systems, under a sampling rate constraint. Specifically, we are concerned with the sampled channel capacity C A (f s , P ), where Here, the right-invertibility represents some mild regularity constraints that ensure each sample contains innovation information, as will be formally defined later. Unless otherwise specified, all sampling systems mentioned in this section are assumed to be right-invertible time-preserving linear systems.
Before proceeding, we shall assume, throughout, that for any frequency f , the following holds A. An Upper Bound on Sampled Channel Capacity 1) The Converse: A time-preserving sampling system preserves the time scale of the signal, and hence does not compress or expand the frequency scale. We now determine an upper limit on the sampled channel capacity for this class of general nonuniform sampling systems. Proposition 1 implies that any multibranch sampling system can be converted into a single branch sampling system without loss of information. Therefore, we restrict our analysis in this section to the class of single branch sampling systems, which provides exactly the same upper bound as the one accounting for multibranch systems. In addition, we constrain our attention to sampling methods that are right-invertible, as defined below 3 .
Definition 7 (Right-Invertible Sampling System). A sampling system with sampling set Λ and impulse response 1) for any k ∈ Z, the frequency response F (q k (τ )) is bounded; 2) for any frequency f and any T with its associated sampling subset the least singular value of the N T × ∞-dimensional matrix F T (f ) is uniformly bounded away from 0.
The right invertibility of the sampling system essentially implies that for each subset of the impulse response {q(t i , τ ) | i ∈ I}, the Fourier matrix associated with any sampled response is right-invertible. This typically implies that the set of samples is a linearly independent family -each sample provides a sufficient amount of innovative information. Our main theorem is now stated as follows.
Theorem 2 (Converse). Assume that there exists a small Under any time-preserving right-invertible sampling system P w.r.t. S η (f ) with sampling rate f s , the sampled channel capacity is upper bounded by where ν is given parametrically bŷ Remark 1. Note that C u (f s , P ) is monotonically nondecreasing in f s and P . In fact, when the sampling rate is increased from f s to f s + δ, C u (f s , P ) corresponds to the optimal value when considering all spectral sets of support size f s + δ.
Since we are still allowed to employ (suboptimal) strategies to allocate power over smaller spectral sets with size f s , we are optimizing over a larger set of transmission / power allocation strategies than the situation with sampling rate f s . Therefore, It can be easily shown that the upper limit C u (f s , P ) is equivalent to the maximum capacity of a channel whose spectral occupancy is no larger than f s . The above result basically implies that even if we allow for more complex irregular sampling sets, the sampled capacity cannot exceed the one commensurate with the analog capacity when constraining all transmit signals to the interval of bandwidth f s that experience the highest SNR. Accordingly, the optimal input distribution will lie in this maximizing frequency set. This theorem also demonstrates that the capacity is attained when aliasing is suppressed by the sampling structure, as will be seen later in our capacity-achieving scheme. When the optimal frequency interval B m is selected, a water filling power allocation strategy is performed over the spectral domain with some water level ν determined by (8).
2) Approximate Analysis: To give some intuition into the results, we provide an approximate (but non-rigorous) argument relying on "noise whitening" and "orthonormal projections".
Suppose that the Fourier transform of the analog channel output r(t) is given by and N (f ) denote, respectively, the frequency responses of x(t) and η(t). When the sampled sequence does not collapse information, we can characterize the sampling process through a linear injective mapping R from the space of linear This way the noise component R (N ) can be treated as additive sampled noise in the frequency domain. We note, however, that this Gaussian noise R (N ) is not necessarily independent over the spectral support [−f s /2, f s /2]. This motivates us to whiten it first without loss of information.
Denote by W the whitening operator and suppose that R (N ) is bounded away from 0. Then the prewhitening process is performed as such that the noise component W (R (N )) is independent across the frequency domain. If we setR(·) ∆ = W (R (·)), then we can rewrite the input-output relation as φ (·) =R (HX) +Ñ , withÑ being white over [−f s /2, f s /2] andR being an orthonormal operator onto L 2 (−f s /2, f s /2). That said, the operatorR effectively projects all spectral components H(f )X(f ) + N (f ) onto a subspace L 2 (−f s /2, f s /2). Instead of scrambling the spectral contents, the optimal projection that maximizes the SNR extracts out a spectral set B m of support size f s that contains the frequency components with the highest SNR. This leads to the capacity upper bound (7). As illustrated in Fig. 2, scrambling the spectral contents does not in general improve capacity. This will be formally proved in Appendix A.
3) Proof Sketch: We now outline the key steps underlying the proof of Theorem 2 for the white-noise scenario. i) We start by analyzing the class of periodic sampling systems: a special type of sampling methods that allow closedform capacity expressions. We then demonstrate that the capacity under any periodic sampling system with sampling rate f s and transmit power P is bounded above by C u (f s , P ).
ii) The upper bound is then derived by relating general (possibly aperiodic) sampled channels with periodic sampled channels through a finite-duration approximation argument. In fact, instead of studying the true sampled channel response directly, we truncate the channel response so that its impulse response is nonzero only for a finite duration. The capacity bound for the resulting truncated channel is then bounded by the capacity of a new periodized channel we construct. As we show, the capacity of the truncated channel can be made arbitrarily close to the capacity of the true sampled channel. The most technically involved step is Step ii), which proceeds by considering two cases as follows.
(a) Consider any given right-invertible timepreserving sampling system P with impulse response q(t, τ ), and suppose that the input x(t) is time constrained to the interval [−T, T ]. Construct a periodic channel with period 2(T + L 0 ) based on q(t, τ ). Let C P p (P ) denote the capacity of the periodized channel, whose sampling rate is bounded above by f s + for some arbitrarily small > 0.
(b) Show that C P T (P ) ≤ T +L0 T C P p T T +L0 P holds uniformly for all P. Since we know that C P p (P ) ≤ C u (f s + , P ) for any periodized channel (or, equivalently, any channel followed by a periodic sampling system), this establishes the capacity upper bound for this class of finite-duration channels, provided that T is sufficiently large. 2) Infinite-duration h(t). We next extend the results to channels for which h(t) is non-zero over infinite duration.
(a) Construct a truncated channel such that for some sufficiently large L 1 . The capacity upper bound holds for the truncated channel, as shown in Step 1).
(b) For any given sampling system P and any time interval [−T, T ], compare the capacity of the original channel (denoted by C P T (P )) with the capacity of the truncated channel (denoted byC P T (P )), which can be completed by investigating the spectrum of the operators associated with both sampled channels. It can be shown that C P T (P ) can be upper bounded byC P T (P + ξ) + ξ for some arbitrarily small constant ξ > 0, which holds uniformly over all sampling systems P. Combining this with results shown in Step 1), we demonstrate that C P T (P ) (and henceC P T (P )) is bounded arbitrarily close by C u (f s , P ) , which establishes the claim for the whole class of infinite-duration channels.

B. Achievability
It turns out that for most scenarios of interest, the capacity upper bound given in Theorem 2 can be attained through filterbank sampling, as stated in the following theorem.
Theorem 3 (Achievability -Sampling with a Filter Bank). Suppose that the maximizing frequency set B m introduced in Theorem 2 exists and is piecewise continuous or, more precisely, where X is an index set, and B i 's are some non-overlapping continuous intervals. Consider the following filterbank sampling mechanism P FB : in the kth branch, the frequency response of the filter is given by and each filter is followed by an ideal uniform sampler with sampling rate µ (B k ). Then where C u (f s , P ) is the upper bound given by (7).
Proof: The spectral components in B i can be perfectly reconstructed from the sequence that is obtained by first extracting out a subinterval B i and then uniformly sampling the filtered output with sampling rate f s,i . The capacity under P FB is commensurate to the analog capacity when constraining the transmit signal to ∪ i B i , which is equivalent to C u (f s , P ).
Note that the bandwidth of B i may be irrational and the system may require an infinite number of filters. Theorem 3 indicates that filterbank sampling with varied sampling rates in different branches maximizes capacity.
The optimality of filterbank sampling immediately leads to another optimal sampling structure under mild conditions. As we have shown in [35], filterbank sampling with equal rates on different branches is equivalent to a single branch of modulation, as illustrated in Fig. 3. This approach attains the sampled capacity achievable by filterbank sampling if the SNRs of the analog channel are piecewise constant in frequency. Although the filterbank sampling we derive in (9) does not employ equal rates on different branches, for most channels of physical interest we can simply divide each branch further into a number of sub-branches to allow the rates at different branches to be reasonably close to each other. Therefore, for most channels of physical interest (say, the channels whose SNRs in frequency are Riemann-integrable), the capacity achievable through filterbank sampling can be approached arbitrarily closely by a single branch of sampling with modulation. This achievability result is formally stated in the following theorem.
Theorem 4 (Achievability -A Single Branch of Sampling with Modulation and Filtering). Under the assumptions of Theorem 3, suppose further that |H(f )| 2 /S η (f ) is constant within each set B i . Then for any > 0, there exists a timepreserving sampling system P MF with sampling rate f s using a single branch of sampling with modulation and filtering such that C PMF (P ) ≥ C u (f s , P ) − , where C u (f s , P ) is defined in (7).
Proof: It is straightforward to see that there exists a set of non-overlapping intervals B k each with equal measure that can approximate the original sets {B l | 1 ≤ l ≤ n} arbitrarily well. Employing the sampling method described in [35, Section V.D] achieves a sampled capacity arbitrarily close to C u (f s , P ).
A channel of physical interest can often be approximated as piecewise constant over frequency in this way. Given the maximizing frequency set B m , the sampling structure suggested in [35, Section V.D] first suppresses the frequency components outside B m using an optimal LTI prefilter. A modulation module is then applied to scramble all frequency components within B m . The aliasing effect can be significantly mitigated by appropriate choices of modulation weights for different spectral subbands. We then employ another band-pass filter to suppress out-of-band signals, and sample the output using a pointwise uniform sampler. Compared with filterbank sampling, a single branch of modulation and filtering only requires the design of a lowpass filter, a band-pass filter, and a multiplication module, which might be of lower complexity to implement than a filter bank.

IV. DISCUSSION
Some properties of the capacity and capacity-achieving strategies are now discussed.
• Monotonicity. It can be seen from Theorem 2 that increasing the sampling rate from f s tof s results in another frequency setB m of support sizef s that has the highest SNRs. By definition, the original frequency set B m must be a subset ofB m . Therefore, the sampled capacity with ratef s is no lower than the sampled capacity with rate f s . • Irregular sampling set. Sampling with irregular nonuniform sampling sets, while requiring complicated reconstruction and interpolation techniques [11], does not outperform filterbank or modulation bank sampling with regular uniform sampling sets in maximizing capacity for the channels considered herein. • Alias suppression. We have seen that aliasing does not allow a higher capacity to be achieved when perfect channel state information is known at both the transmitter and the receiver. The optimal sampling method corresponds to the optimal alias-suppression strategy. This is in contrast to the benefits obtained through random mixing of spectral components in many sub-Nyquist sampling schemes with unknown signal supports. When we are allowed to jointly optimize over both input and sampling schemes with perfect channel state information, scrambling of spectral contents does not in general maximize capacity. • Perturbation of the sampling set. If optimal filterbank or modulation sampling is employed, then mild perturbation of post-filtering uniform sampling sets does not degrade the sampled capacity. One surprisingly general example was proved by Kadec [40]. Suppose that a sampling ratef s is used in any branch and the sampling set satisfies t n − n/f s ≤f s /4. Then exp j2πt n f | n ∈ Z also forms a Riesz basis of L 2 (−f s /2,f s /2), thereby preserving information integrity. These nonuniform sampling and reconstruction schemes, while generally complicated to implement in practice, significantly broaden the class of sampling mechanisms that allow perfect reconstruction of bandlimited signals, and indicate stability and robustness of the sampling sets. Kadec's result immediately implies that the sampled capacity is invariant under mild perturbation of the sampling sets. • Hardware implementation. When the sampling rate is increased from f s1 to f s2 , we need only to insert an additional filter bank of overall sampling rate f s2 − f s1 to extract out another set of spectral components with bandwidth f s2 −f s1 . Thus, the adjustment of the sampling hardware system for filterbank sampling is incremental with no need to rebuild the whole system from scratch. • Spectrum Blind Sampling. This paper focuses on the scenario with perfect channel state information known at the transmitter, the receiver, and the sampler. This is different from the setting of compressed sensing, where the signal spectrum is unknown to the sampler and the decoder. In fact, the alias-suppressing sampler requires knowledge of the channel. If this knowledge is not available, then alias-suppressing samplers might result in low capacity. When the sampler is spectrum blind and the channel realization is uncertain, random sampling that scrambles the spectral contents [31], [41] outperforms alias-suppressing sampling in minimizing the rate loss due to channel-independent sampling design. We investigate the capacity of sub-Nyquist sampled channels with unknown CSI in our companion paper [42].

V. CONCLUDING REMARKS
We developed the maximum achievable information rate for a general class of right-invertible time-preserving nonuniform sampling methods under a sampling rate constraint. It is shown that nonuniformly spaced sampling sets, while requiring fairly complicated reconstruction / approximation algorithms, do not provide any capacity gain. Encouragingly, filterbank sampling with varied sampling rates on different branches, or a single branch of sampling with modulation and filtering, are sufficient to achieve the sampled channel capacity. In addition, both strategies suppress aliasing effects. In terms of maximizing capacity, there is no need to employ irregular sampling sets that are more complicated to implement in practical hardware systems. The resulting sampled capacity is shown to be monotonically increasing in sampling rate.
Our results in this paper are based on the assumption that perfect channel state information is known at both the transmitter and the receiver. It remains to be seen what sampling strategies can optimize information rates when only partial channel state information is known. It is unclear whether antialiasing methods are still optimal in maximizing capacity. Moreover, when it comes to the multi-user information theory setting, anti-aliasing methods might not outperform other spectral-mixing approaches in the entire capacity region. It would be interesting to see how to optimize the sampling schemes in multi-user channels, for example, joint sampling schemes in sampled multiple access analog channels.

ACKNOWLEDGMENTS
The authors would like to thank Prof. Young-han Kim and the anonymous reviewers for extensive and constructive comments that greatly improved the paper.

APPENDIX A PROOF OF THEOREM 2
For simplicity of presentation, we assume throughout that the noise is white, i.e. S η (f ) ≡ 1. In fact, under the assumption (6), we can always split the channel filter H (f ) into two parts with respective frequency response H (f ) / S η (f ) and S η (f ). Since the colored noise is equivalent to a white Gaussian noise passed through a filter with transfer function S η (f ), the original system can be redrawn as in Fig. 4. The filter with frequency response S η (f ) can then [n y new preprocessor be incorporated into the preprocessing system to generate a new time-preserving preprocessor.
Recall that Proposition 1 indicates that any multibranch sampling system can be converted into a single branch sampling system without loss of information. As a result, we restrict our proof to the class of single branch sampling systems.

A. Capacity under Periodic Sampling Systems
Recall that for a time varying system, the impulse response q(t, τ ) is defined as the output seen at time t due to an impulse in the input at time τ . The sampling system may not be timeinvariant, but a broad class of sampling mechanisms applied in practice exhibit block-wise time invariance properties. Specifically, we introduce the notion of periodic sampling systems as follows.
Definition 8 (Periodic Sampling). Consider a sampling system with a preprocessing system of impulse response q(t, τ ) followed by a sampling set Λ = {t k | k ∈ Z}. A linear sampling system is said to be periodic with period T q and sampling rate f s (f s T q ∈ Z) if the preprocessing system is periodic with period T q and the sampling set satisfies In short, a periodic sampling system consists of a periodic preprocessor followed by a pointwise sampler with a periodic sampling set, as illustrated in Fig. 5. Since the impulse response can be arbitrary within a period, this allows us to model multibranch sampling methods with each branch using the same sampling rate. Periodic sampling schemes subsume as special cases a broad class of sampling techniques, e.g. sampling via filter banks, sampling via periodic modulation, and recurrent nonuniform sampling [13], [43].
The periodicity of the sampling system renders the linear operator associated with the whole system to be block Toeplitz. The asymptotic spectral properties of block Toeplitz operators (e.g. [44]) guarantee the existence of lim T →∞ C P T (f s , P ) for a given periodic sampling system P, and allows a capacity expression to be obtained in terms of the Fourier representation. Denote by Q k (f ) the Fourier transform of the impulse response q(t k , t k − t) of the sampling system, i.e. Q k (f ) :=´∞ −∞ q(t k , t k − t) exp(−j2πf t)dt. We further introduce an f s T q × ∞ dimensional Fourier series matrix F q (f ) associated with the sampling system, and another infinite diagonal square matrix F h (f ) associated with the channel response. For all m, l ∈ Z and 1 ≤ k ≤ f s T q , we .

(11)
We can then express the sampled analog capacity for a given periodic system P in closed form as follows 4 .
Lemma 1. Suppose the sampling system P is periodic with period T q and sampling rate f s , where f s T q ∈ Z.
We also assume that the smallest singular value of F q F * q (f ) is uniformly bounded away from 0 over all (1) The sampled channel capacity under optimal power allocation is given by where ν satisfieŝ Here, λ i denotes the ith largest eigenvalue of the matrix (2) Suppose further that H(f ) = 0 for any f / ∈ [0, W ], and that the transmitter employs equal power allocation over [0, W ]. Then the sampled channel capacity is given by Proof: See Appendix B. In Lemma 1, ν is the water-level with respect to the optimal water-filling power allocation strategy over the eigenvalues of The capacity expression (12) admits a simple upper bound, as stated below. 4 We note that a periodic sampling system can be equivalently converted to a filterbank sampling system, and hence [35,Theorem 4] immediately leads to (12). While the analysis framework in [35] relies on a discretization argument, we provide here a more general approach based on operator analysis that no longer requires discretization. More importantly, this approach allows to accommodate any sampled channel with integrable |H(f )Q k (f )| 2 /Sη(f ), which subsumes channels under band-limited filterbank sampling. In comparison, [35] requires F −1 |H(f )Q k (f )| / Sη(f ) to lie in L 1 , which is far more restrictive (e.g. it does not subsume band-limited filterbank sampling as the sinc function is not absolutely integrable). Corollary 1. (a) Consider the setup and assumptions in Lemma 1. Under all periodic sampling systems with period T q and sampling rate f s , the sampled channel capacity can be bounded above by Then where C u (f s , P ) is given by (7).
Proof: (a) Following the same steps as in [35, Proposition 1], we can see that the ith largest eigenvalue satisfies which immediately leads to (14).
(b) For any given f q , the upper bound (14) is obtained by extracting out a certain frequency set B that has measure µ(B) = f s and suppressing all spectral components outside B. By our definition of B m , any choice of B with spectral size f s will not outperform B m . Hence, choosing B = B m leads to a universal upper bound.
Corollary 1 reveals that the capacity under any periodic sampling system, no matter what its period is, cannot exceed the upper bound C u (f s , P ) in Theorem 2. Our remaining proof is then established by observing that any aperiodic sampling system can be related to a periodic sampling system by truncation and periodization, as elaborated in the next two subsections.

B. General Upper Bound: Finite-duration h(t)
In this subsection, we focus on the channel whose impulse response is of finite duration 2L 0 , i.e. h(t) = 0, ∀t (|t| > L 0 ).
Our goal is to prove that the capacity upper bound (7) holds for this type of channel.
For any transmission block of duration 2T , we call the transmit signal x(t) over this block a codeword (or symbol) of code length 2T . The information conveyed through such finiteduration codewords can be bounded via certain analog channel capacity, as long as we can preclude inter-symbol interference.
The key idea here is to separate consecutive codewords with a guard zone with sufficient length and then use capacityachieving strategies separately for each codeword. When the code length 2T is sufficiently large, the transmission time wasted on the guard zones becomes negligible, which in turn allows us to approach the true capacity arbitrarily well. The detailed analysis proceeds as follows.
Step 1. Consider an input x(t) that is constrained to the interval [−T, T ]. Since h(t) is of finite duration 2L 0 , the channel output r(t) = h(t) * x(t) + η(t) will be affected by the input only when t ∈ [−T − L 0 , T + L 0 ]. Define a window operator and its complement operator such that and Then for any linear sampling operator P with impulse response q(t, τ ), the sampled output is P (r(t)) = P (w T (r(t))) + P w ⊥ T (r(t)) . One can easily observe that the component P w ⊥ T (r(t)) contains no information about x(t), and is statistically independent of P (w T (r(t))) due to the whiteness assumption of the noise. In other words, the sampling input outside the interval [−T − L 0 , T + L 0 ] does not improve capacity at all. Consequently, it suffices to restrict attention to the class of sampling systems whose system input is constrained to the interval [−T − L 0 , T + L 0 ].
Step 2. Construct a periodization of the above sampled channel model with finite input duration. Set the impulse response q p T +L0 (t, τ ) of the preprocessor of the periodized sampling system to be a periodic extension of q(t, τ ) in the block [−T − L 0 , T + L 0 ] × [−T, T ]. Specifically, if τ = k ·2 (T + L 0 )+τ r for some k ∈ Z and τ r ∈ [−T −L 0 , T +L 0 ], then Apparently, q p T +L0 (t, τ ) corresponds to a periodic preprocessing system with period 2 (T + L 0 ).
Suppose without loss of generality that the indices of the sample times that fall in [−T −L 0 , T +L 0 ] are 0, 1, · · · , K −1, i.e. {k | t k ∈ [−T − L 0 , T + L 0 ]} = {0, 1, · · · , K − 1}. We can then set the sampling set Λ p T +L0 of the periodized system such that for any sampling time t k ∈ Λ p T +L0 , we have where x ∆ = max {n | n ∈ Z, n ≤ x}. Clearly, this forms a periodic sampling set with period 2 (T + L 0 ). The definition of Beurling density ensures that for any > 0, there exists a T D such that for every T > T D , T +L0 ≤ f s + . Due to the finite-duration assumption of h(t), our construction (18) guarantees that the input x(t) within time interval [2k(T + L 0 ) − T, 2k(T + L 0 ) + T ] will only affect the sampled output at the kth time block [(2k − 1) (T + L 0 ), (2k + 1) (T + L 0 )], as illustrated in Fig. 6. Since the noise η(t) is assumed to be white, the noise components across different time blocks are independent. In fact, the intervals [2k(T +L 0 )+T, (2k +1)(T +L 0 )−T ] (k ∈ Z) act effectively as guard zones in order to avoid leakage of signals across different time blocks. Based on the above argument, we can separate codewords of duration 2T in [2k(T +L 0 )+T, (2k+1)(T +L 0 )−T ] (k ∈ Z) on the analog channel by a guard zone 2L 0 (as illustrated in Fig. 6). The ratio of guard space to the length of the time block vanishes as T → ∞, and there is no intersymbol interference under the new system we construct. By our capacity definition, for any δ > 0, there exists a T 0 such that ∀T > T 0 , we have T + L 0 T < 1 + δ, and T T + L 0 > 1 − δ. Consequently, where C P p denotes the capacity under our periodized sampling system. The inequality (i) follows from the following three arguments: T is the information rate when we observe the samples within the interval [−T, T ], which is smaller than the information rate, termedĈ P T , when we observe all samples within [−T − L 0 , T + L 0 ];

•Ĉ P
T is equivalent to the maximum information rate achievable by the periodized system, under the constraint that there is no input signal transmitted over the guard zones. Clearly, this rate will be smaller than the capacity without this transmission constraint, which is T +L0 T C P p . Here, the multiplication factor T +L0 T arises from the fact that we only use a portion T T +L0 of time for transmission; • Since the total energy over each transmission block is P T and each guard zone has zero power, the average power allocated to the transmitted signal is T T +L0 P .
We know from Corollary 1(b) that where the last inequality arises from observing that C u (f s , P ) is monotonically non-decreasing in f s and P . Putting (20) and (21) together yields as soon as T > max {T 0 , T D }. Since and δ can be chosen arbitrarily small, we have that when h(t) is of finite duration and η(t) is white.

C. General Upper Bound: Infinite-Duration h(t)
We now investigate the capacity bound when h(t) is not time-limited. We would like to prove that for any given sampling system P and any > 0, there exists T 1 such that for any T > T 1 , one has C P T ≤ C u (f s , P ) + . Our proof proceeds by comparing the original channel with a truncated channel whose channel responseh(f ) satisfies Let ξ > 0 be an arbitrary small constant, and L 1 chosen such We further constrain the input and the observed sampled output to the time interval [−T, T ]. For both the original and truncated channel, the sampled noise is not white, which motivates us to first perform prewhitening. Suppose without loss of generality that the sampled times within [−T, T ] are {t i | 1 ≤ i ≤ K T }. For convenience of notation, we introduce a linear operatorP T associated with the sampling system such that P T (r(t)) = [y [1], y [2], · · · , y[K T ]] ,  [2] . . .
Denote byq(t i , τ ) the impulse response associated with this sampling system. Then, the noise componentP T (η(t)) can be whitened by left-multiplying it with a K T -dimensional square matrix W −1/2 P defined by The invertibility is guaranteed by our assumptions. To see this, if we denote byη PP T (η(t)) the K T -dimensional "prewhitened" noise, then one can verify that for every i and j, or, equivalently, As a result, the covariance ofη obeys If we denote byP w ∆ = W − 1 2 P ·P T and letq w (t i , τ ) represent its associated impulse response, then the above calculation reveals thatˆ∞ −∞q indicating that {q w (t i , ·) , 1 ≤ i ≤ K T } forms a set of orthonormal sequences in the corresponding Hilbert space. For an operator A with an impulse response a(t, τ ) (−T ≤ τ ≤ T , t ∈ {t i | 1 ≤ i ≤ K T }) and input domain D (A), we denote by A F the generalized Frobenius norm of the operator A with respect to its associated domain, namely, Recall thatq w (t i , ·) (1 ≤ i ≤ K T ) forms orthonormal sequences. By Bessel's inequality [45], an operator A with for every τ ∈ [−T, T ], which immediately gives Denote by {λ i } and {λ i } the set of squared singular values associated with the original sampled channel operatorP w G and the operatorP wG of the truncated sampled channel, respectively. Here, G andG represent respectively the operator associated with the original channel response and the truncated channel response. We can obtain some properties of {λ i } and {λ i } as stated in the following lemma.
For any ξ > 0, there exists T 0 such that for every T > T 0 , one has 5+ε for some small ε > 0. Then there exists T 0, such that for every T > T 0, , one has λ i −λ i ≤ ξ.
Proof: See Appendix C. For notational simplicity, define two functions as follows and for some water level ν. Note that if ν is chosen such that F T (ν, {λ i }) = P , then Apparently, both C P T (P ) and C P T (ν, {λ i }) are non-decreasing functions of {λ i }, which implies that and where ν is determined by Here, ξ > 0 is some arbitrarily small constant. In fact, one can easily verify that C P T ν, max λ i , ξ 1 3 with ν determined by (28) is no larger than the sum capacity of two separate channels with respective eigenvalues {λ i } and λ i := ξ 1 3 each with power allocation P . In other words, ≤ C P T (P ) + For any positive water level ν and some small constant ξ > 0, the Lipschitz constants of the functions are bounded above in magnitude by 1 2 ξ −1/3 and ξ −2/3 , respectively. Using the same water level ν, the corresponding power for both channels can be computed as Combining Lemma 2 and the Lipschitz constants of f 2 (x) immediately suggests that: there exists T 0, such that for any T > T 0, , one has Similarly, we can bound Combining (31), (32) and (22) suggests that · C u f s + , P + (f s + ) ξ 1 3 , where (33) is a consequence of (30). Since δ, , and ξ can all be made arbitrarily small, it follows that completing the proof.

APPENDIX B PROOF OF LEMMA 1
The proof is restricted to the channel with white noise, i.e. S η (f ) ≡ 1. It is straightforward to extend the analysis to colored noise through the argument presented in the first paragraph of Appendix A.
Our proof proceeds in the following three steps. 1) We first introduce several correlation functions and compute the Fourier series associated with them. These quantities are crucial in deriving the capacity expression.
In particular, when the sampling system is periodic, the infinite correlation matrices are block Toeplitz. 2) When constrained to a finite time interval [−nT q , nT q ], the sampled output is a finite vector. The sampled noise is in general not white, which motivates us to whiten it first. In fact, the covariance matrix of the sampled noise can be easily derived in terms of the proposed correlation functions. 3) For any time interval [−nT q , nT q ], the capacity is obtained through the Karhunen Loeve expansion. Specifically, the capacity depends on the eigenvalues of the associated system operator, which is related to the correlation functions. The asymptotic properties of block Toeplitz matrices guarantee the convergence when n → ∞, which allow us to derive in closed form the sampled channel capacity.

A. Correlation functions and Fourier series
For a concatenated linear system consisting of the channel filter followed by the sampling system, we denote by its system output seen at time t o due to an impulse input at time t i . For notational convenience, we define q k (τ ) := q (t k , τ ) as the sampling output response at time t k due to an impulse input to the sampling system at time τ . Two output autocorrelation functions are defined as follows and For notational simplicity, we use R hq (k, l) (resp. R q (k, l)) and R hq (t k , t l ) (resp. R q (t k , t l )) interchangeably. When the sampling system is periodic with period T q , one can easily see that both [R hq (k, l)] The spectral properties associated with the system operators are captured by Fourier series matrices F hq , F qq , F h and F q . Specifically, F hq is an f s T q -dimensional square matrix such that: for any frequency f and all 1 ≤ k, i ≤ f s T q , (37) and Besides, for every frequency f , we define an f q T s × ∞ dimensional matrix F q (f ) and an infinite square diagonal matrix F h (f ) such that for all l ∈ Z and 1 ≤ k ≤ f q T s : where Q k (f ) ∆ = F (q k (·)) = F (q(t k , ·)). The key properties of the above autocorrelation functions and Fourier series are summarized in the following lemma.
Lemma 3. The Fourier series matrices satisfy: and Proof: See Appendix D.

B. Noise whitening
Denote by Q k (·) the sampling operator associated with the sample time t k such that Q k (x) The correlation of noise components Q k (η) at different times can be calculated as is a zero-mean Gaussian vector with covariance matrix R q .
We now constrain both the transmit interval and the observation interval to [−nT q , nT q ]. Let where the sampled output sequence satisfies Introduce two 2nf s T q -dimensional truncated autocorrelation matrices R n hq and R n q such that for all −nf s T q < k, l ≤ nf s T q , R n hq k,l = R hq (t k , t l ) , R n q k,l = R q (t k , t l ) .
Clearly, the noise components of y n exhibit a covariance matrix R n q , which motivates to whiten it first.
By left multiplying y n with R n q − 1 2 , we obtain a new input-output relation as where {η [k]} are i.i.d. Gaussian random variables each of unit variance. Denote byq (t k , τ ) the equivalent impulse response of this new system. The truncated output autocorrelation function R ñ q is given as R ñ q k,l = Rq (t k , t l ) = ∞ −∞q (t k , τ )q * (t l , τ ) dτ , satisfying by construction.

C. Capacity via asymptotic properties of block Toeplitz matrices
While both R n q and R n hq are block Toeplitz matrices, R ñ q is in general not a block Toeplitz matrix. By exploiting the asymptotic equivalence in Toeplitz matrix theory [46], one can see that R ñ q is asymptotically equivalent to a block-Toeplitz matrix generated by the Fourier series Therefore, the asymptotic spectral properties of a block-Toeplitz matrix (e.g. [44]) state that for any nondecreasing continuous function g(t) with a bounded slope, one has The capacity of the sampled channel with an optimal water level ν p can now be calculated as where (47) is a consequence of (45). The water level ν is computed through the following parametric equation which by (45) is asymptotically equivalent to by change of variables. This establishes the claim.
(2) We consider now the scenario where equal power allocation is employed. Classical MIMO channel capacity results [47] indicate that the optimal power allocation for the transmitter is to allocate equal amount of power in all transmit branches. It remains to see how much power is allocated to the branch associated with λ i R ñ q . In fact, if the transmitter knows the channel bandwidth, almost all power (except for negligible leakage due to finitetime approximation) will be allocated inside the channel bandwidth [0, W ]. Therefore, by the Shannon-Nyquist sampling theorem, all transmit signals can be equivalently transformed to a delta train ∞ i=−∞ x i δ(t−i/W ), where x i 's are randomly generated transmit signals. Consider the input time block [−nT q , nT q ], then there are equivalently 2nT q W transmit branches inside this time block. Since the total power is P tot = 2nT q P , the power allocated to each transmit branch is given by As a result, the sampled capacity under equal power allocation is computed as C P eq (P ) = lim APPENDIX C PROOF OF LEMMA 2 (1) Let G andG denote respectively the operators associated with g(t) andg(t). Then, the triangle inequality yields From (23) one can easily show that for any ξ > 0, there exists a T 0 such that for every T > T 0 , one has Additionally, suppose that´∞ −∞ |h(t)| 2 dt ≤ C g < ∞. Then, we have This together with (48) (2) We can also bound the sum of eigenvalues as follows (3) If g(t) = O 1 t 1+ , then one can further control Therefore, applying Weyl's Theorem [48] yields that Therefore, for any small ξ > 0, there exists a constant T 0, such that for every T > T 0, , one has λ i −λ i < ξ.
By the periodicity assumption of the sampling system, one can derive R hq t k+afsTq , t l+bfsTq =¨q (t k + aT q , τ k ) R h (τ l − τ k ) q * (t l + bT q , τ l ) dτ k dτ l =¨q (t k , τ k − aT q ) R h (τ l − τ k ) · q * (t l + (b − a) T q , τ l − aT q ) dτ k dτ l = R hq t k , t l+(b−a)fsTq .

Observing that
we can see that (F hq ) k,i is simply the Fourier transform of the sampled sequence of R h * q k * q − * i . The properties of the Fourier transform suggest that where Q k (f ) := F (q k ). By definition in (37), one can write which immediately leads to This allows us to express F hq as Similarly, the equality F qq = F q F * q is then an immediate consequence of (49) by setting F h = I. In 2012 she was elected to the Young Israel Academy of Science and to the Israel Committee for Higher Education, and elected an IEEE Fellow. In 2013 she received the Technion's Award for Excellence in Teaching, the Hershel Rich Innovation Award, and the IEEE Signal Processing Technical Achievement Award. She received several best paper awards together with her research students and colleagues. She received several best paper awards together with her research students and colleagues. She is the Editor in Chief of Foundations and Trends in Signal Processing. In the past, she was a Signal Processing Society Distinguished Lecturer, member of the IEEE Signal Processing Theory and Methods and Bio Imaging Signal Processing technical committees, and served as an associate editor for the IEEE Transactions On Signal Processing, the EURASIP Journal of Signal Processing, the SIAM Journal on Matrix Analysis and Applications, and the SIAM Journal on Imaging Sciences.