# The peculiar motions of early-type galaxies in two distant regions – V. The Mg–𝝈 relation, age and metallicity

## 1 INTRODUCTION

The primary aim of the EFAR project (Wegner et al. 1996; Paper 1) is to use the tight correlations between the global properties of early-type galaxies embodied in the Fundamental Plane (FP: Djorgovski & Davis 1987, Dressler et al. 1987) to measure relative distances to clusters of galaxies in order to investigate peculiar motions and the mass distribution on large scales. However these global relations also constrain the dynamical properties and evolutionary histories of early-type galaxies. For example, Renzini & Ciotti (1993) show that the tilt of the FP implies a range in mass-to-light ratio $`M/L`$ among ellipticals of less than a factor of three, while the low scatter about the FP implies a scatter in $`M/L`$ at any location in the plane of less than 12%. Similar reasoning has been used to constrain the star formation history of cluster ellipticals using the colour–magnitude relation (Bower et al. 1992, Kodama & Arimoto 1997). Recently the FP, Mg–$`\sigma `$ and colour–magnitude relations have been followed out to higher redshifts and used to show that the early-type galaxies seen at $`z`$$``$1 differ from present-day early-type galaxies in a manner consistent with passive evolutionary effects (van Dokkum & Franx 1996, Ziegler & Bender 1997, Kelson et al. 1997, Ellis et al. 1997, Kodama & Arimoto 1997, Stanford et al. 1998, Kodama et al. 1998, Bender et al. 1998, van Dokkum et al. 1998).

In this paper we consider the relation between the central velocity dispersion $`\sigma `$ and the strength of the magnesium lines at a rest wavelength of 5174Å for the early-type galaxies in the EFAR sample. This Mg–$`\sigma `$ relation connects the dynamical properties of galaxy cores with their stellar populations. The remarkably small scatter about this relation (Burstein et al. 1988, Guzmán et al. 1992, Bender et al. 1993, Jørgensen et al. 1996, Bender et al. 1998), and its distance-independent nature, make it a potentially useful constraint on models of the star formation history of early-type galaxies and a test for environmental variations in the FP (Burstein et al. 1988, Bender et al. 1996).

There are, however, some problems with using the Mg–$`\sigma `$ relation for probing galaxy formation. Two of these problems are apparent from the stellar population models (e.g. Worthey 1994, Vazdekis et al. 1996): (i) both age and metallicity contribute to the Mg linestrengths in comparable degree, so that a spread in linestrengths could be due to either a range of ages or a range of metallicities or some combination; (ii) the Mg linestrengths are not particularly sensitive indicators—at fixed metallicity a difference in age of a factor of ten only results in a change of 0.05-0.1 mag, while at fixed age a difference of 1 dex in metallicity gives a change of 0.1–0.2 mag. Thus Mg linestrength measurements must be accurate in order to yield useful constraints on the ages and metallicities of stellar populations, and the Mg–$`\sigma `$ relation on its own can only supply constraints on combinations of age and metallicity and not one or the other separately.

Recently Trager (1997) has suggested that the tightness of the Mg–$`\sigma `$ relation may be the result of a ‘conspiracy’, in that there appears to be an anti-correlation between the ages and metallicities of the stellar populations in early-type galaxies at fixed mass which acts to reduce the scatter in the Mg linestrengths. Trager takes the accurate H$`\beta `$, Mg and Fe linestrengths from González (1993) and applies the stellar population models of Worthey (1994) to derive ages and abundances from line indices with different dependences on age and metallicity. He finds that at fixed velocity dispersion the ages and abundances lie in a plane of almost constant Mg linestrength, leading him to predict little scatter in the Mg–$`\sigma `$ relation even for large differences in age or metallicity—a factor of ten in age (from 1.5 Gyr to 15 Gyr) gives a spread in Mg<sub>2</sub> of only 0.01–0.02 mag. This conclusion depends on the appropriateness of the single stellar population models and requires confirmation from further high-precision linestrength measurements. It can also be tested using the high-redshift samples now becoming available.

In a similar vein, a number of authors (Ferreras et al. 1998, Shioya & Bekki 1998, Bower et al. 1998) have recently re-examined whether the apparent passive evolution of the colour–magnitude relation out to $`z`$$``$1 really implies a high redshift for the bulk of the star-formation in elliptical galaxies. They conclude that in fact such evolution can be consistent with a rather broad range of ages and metallicities if the galaxies assembling more recently are on average more metal-rich than older galaxies of similar luminosity.

As well as studies focussing on the evolution of the galaxy population, there have also been investigations of possible variations with local environment. Guzmán et al. (1992) have suggested that there are systematic variations in the Mg–$`\sigma `$ relation which affect estimates of relative distances based on the FP. They report a significant offset in the zeropoint of the Mg–$`\sigma `$ relation between galaxies in the core of the Coma cluster and galaxies in the cluster halo. Jørgensen and co-workers (1996, 1997) examine a sample of 11 clusters and find a weak correlation between Mg linestrength and local density within the cluster which is consistent with this result. Similar offsets are claimed between field and cluster ellipticals by de Carvalho & Djorgovski (1992) and Jørgensen (1997), although Burstein et al. (1990) find no evidence of environmental effects. Such systematic differences could result from different star-formation histories in different density environments, producing variations in the mass-to-light ratio of the stellar population. FP distance measurements would then be subject to environment-dependent systematic errors leading to spurious peculiar motions. Where data for field and cluster ellipticals come from different sources, however, the possibility also exists that any zeropoint differences are due to uncertainties in the relative calibrations rather than intrinsic environmental differences.

The Mg–$`\sigma `$ relation has thus become an important diagnostic for determinations of both the star formation history and the peculiar motions of elliptical galaxies. Here we examine the Mg–$`\sigma `$ relation in the EFAR sample, which includes more than 500 early-type galaxies drawn from 84 clusters spanning a wide range of environments. In §2 we summarise the relevant properties of the sample and the techniques used to determine the Mg$`b`$ and Mg<sub>2</sub> linestrength indices, the central velocity dispersions $`\sigma `$, and the errors in these quantities. We present the Mg–$`\sigma `$ relation in §3 and investigate how it varies from cluster to cluster within our sample, and with cluster velocity dispersion, X-ray luminosity and X-ray temperature. In §4 we compare our results with the predictions of stellar population models in order to derive constraints on the ages, metallicities and mass-to-light ratios of early-type galaxies in clusters. In particular, we consider the constraints on the dispersion in the ages and metallicities from the intrinsic scatter in the Mg–$`\sigma `$ relation on its own, and in combination with the intrinsic scatter in the FP. Our conclusions are given in §5.

## 2 THE DATA

Here we give a short description of our sample and dataset, with emphasis on the velocity dispersions and line indices used in this paper. The interested reader can find more detail on the sample selection in Paper 1 (Wegner et al. 1996); on the measurement, calibration and error estimation procedures for the spectroscopic parameters in Paper 2 (Wegner et al. 1998); and on the structural and morphological properties of the galaxies in Paper 3 (Saglia et al. 1997).

### 2.1 The sample

The EFAR sample of galaxies comprises 736 mostly early-type galaxies in 84 clusters. These clusters span a range of richnesses and lie in two regions toward Hercules–Corona Borealis and Perseus–Pisces–Cetus at distances of between 6000 km s<sup>-1</sup> and 15000 km s<sup>-1</sup>. In addition to this program sample we have also observed 52 well-known galaxies in Coma, Virgo and the field in order to provide a calibrating link to previous studies.

The EFAR galaxies are listed in Table 2 of Paper 1, and comprise an approximately diameter-limited sample of galaxies larger than about 20 arcsec with the visual appearance of ellipticals. Photometric imaging (Paper 3) shows that 8% are cDs, 12% are pure Es and 49% are bulge-dominated E/S0s; thus 69% of the sample are early-type galaxies, with the remaining 31% being spirals or barred galaxies. We have obtained spectroscopy for 666 program galaxies, measuring redshifts, velocity dispersions and linestrength indices (Paper 2). We have used the redshifts we obtained together with literature redshifts for other galaxies in the clusters in order to assign program galaxies to physical clusters. We have used the combined redshift data for these physical clusters to estimate cluster mean redshifts and velocity dispersions. The early-type galaxies in our sample span a wide range in luminosity, size and mass: they have absolute magnitudes from $`M_R`$=$``$24 to $``$18 ($`M_R`$=$``$21.6; H<sub>0</sub>=50 km s<sup>-1</sup> Mpc<sup>-1</sup>), effective radii from 1 h$`{}_{}{}^{1}{}_{50}{}^{}`$ kpc to 70 h$`{}_{}{}^{1}{}_{50}{}^{}`$ kpc ($`R_e`$=9.1 h$`{}_{}{}^{1}{}_{50}{}^{}`$ kpc) and central velocity dispersions from less than 100 km s<sup>-1</sup> to over 400 km s<sup>-1</sup> ($`\sigma `$=220 km s<sup>-1</sup>). The sample is thus dominated by early-type galaxies with luminosities, sizes and masses typical of giant ellipticals.

### 2.2 The measurements

We summarise here the procedures used in measuring the redshifts, velocity dispersions and Mg linestrengths; full details are given in Paper 2.

Redshifts and velocity dispersions were measured from each observed galaxy spectrum using the IRAF task fxcor. Linestrength indices on the Lick system were determined using the prescription given by González (1993). The Mg$`b`$ and Mg<sub>2</sub> indices were both measured: Mg<sub>2</sub> because it is the index most commonly measured in previous work, and Mg$`b`$ because it could be measured for more objects (as it requires a narrower spectral range) and is better-determined (being less susceptible to variations in the non-linear continuum shape). We find it more convenient to express the ‘atomic’ Mg$`b`$ index in magnitudes like the ‘molecular’ Mg<sub>2</sub> index rather than as an equivalent width in Ångstroms, since this puts these two indices on similar footings. The conversion is

$$\text{Mg}b^{}=2.5\mathrm{log}_{10}\left(1\frac{\text{Mg}b}{\mathrm{\Delta }\lambda }\right),$$
(1)   
where $`\mathrm{\Delta }\lambda `$ is the index bandpass (32.5Å for Mg$`b`$).

Error estimates for each quantity were derived from detailed Monte Carlo simulations, calibrated by comparisons of the estimated errors with the results obtained from repeat measurements (over 40% of our sample had at least two spectra taken). Two sorts of corrections were applied to the dispersions and linestrengths: (i) an aperture correction, based on that of Jørgensen et al. (1995), to account for different effective apertures sampling different parts of the galaxy profile, and (ii) a run correction to remove systematic errors between different observing setups. After applying these corrections, individual measurements for each galaxy were combined using a weighting scheme based on the estimated errors and the overall quality of the spectrum.

The median estimated errors in the final combined values are $`\mathrm{\Delta }\sigma /\sigma =9.1\%`$ (i.e. $`\mathrm{\Delta }\mathrm{log}\sigma =0.040`$ dex), $`\mathrm{\Delta }\text{Mg}b^{}=0.013`$ mag and $`\mathrm{\Delta }\text{Mg}\text{2}=0.015`$ mag. The distribution of estimated errors for each quantity is shown in the upper panel of Figure 1. The lower panel of the figure shows how the error estimates were calibrated against the repeat observations: the distribution of the ratio of rms error to estimated error for objects with repeat measurements is compared to the predicted distribution assuming the estimated errors are the true errors. The initial error estimates from the simulations have been re-scaled to give the best match (under a K-S test) to the rms errors from the repeat measurements. A re-scaling by factors of 0.85 and 1.15 respectively gives good agreement for the errors in $`\sigma `$ and Mg$`b`$; adding 0.005 mag likewise gives good agreement for the errors in Mg<sub>2</sub>.

A comparison with the literature (Paper 2, Figure 13) shows that our dispersions are consistent with previous measurements by Davies et al. (1987), Guzmán (1993), Jørgensen (1997), Lucey et al. (1997) and Whitmore et al. (1985). For the subset of galaxies in common, we compared our linestrengths with the definitive Lick system measurements of Trager et al. (1998) in order to derive the small zeropoint corrections required to calibrate our measurements to the Lick system (Paper 2, Figures 14 & 15); the overlap of our Mg<sub>2</sub> measurements with those of Lucey et al. (1997) also shows consistency (Figure 16, Paper 2).

## 3 THE Mg–$`\sigma `$ RELATION

### 3.1 The global relation

In this section we investigate the global Mg–$`\sigma `$ relation found amongst the entire sample of EFAR galaxies with early-type morphological classifications (cD, E or E/S0; see definitions in Paper 3) for which we obtained linestrength measurements. The Mg$`b^{}`$$`\sigma `$ relation is shown in Figure 2a and the Mg<sub>2</sub>$`\sigma `$ relation in Figure 2b.

In order to fit linear relations with intrinsic scatter in the presence of significant measurement errors in both variables, arbitrary censoring of the dataset and a broad sample selection function, we have developed a comprehensive maximum likelihood (ML) fitting procedure (Saglia et al., in preparation). Excluding galaxies with dispersions less than 100 km s<sup>-1</sup> or selection probabilities less than 10%, and also outliers with low likelihoods, the ML fits to the Mg$`b^{}`$$`\sigma `$ relation (490 galaxies) and the Mg<sub>2</sub>$`\sigma `$ relation (423 galaxies) are:

Mg$`b^{}`$ $`=`$ $`(0.131\pm 0.017)\mathrm{log}\sigma (0.131\pm 0.041),`$ (2)   
Mg<sub>2</sub> $`=`$ $`(0.257\pm 0.027)\mathrm{log}\sigma (0.305\pm 0.064).`$ (3)   
These fits are shown in Figure 2 as solid lines. The ratio of the slopes of these relations is consistent with the Mg<sub>2</sub>-Mg$`b^{}`$ relation we obtained in Paper 2: Mg<sub>2</sub>$``$1.94Mg$`b^{}`$$``$0.05. Monte Carlo simulations of the dataset and fitting process, the results of which are displayed in Figure 3, show that there is no bias in the ML estimates of the slopes and zeropoints, and provide reliable estimates of the uncertainties in the fit.

The ML fits can be compared to simple regressions of Mg$`b^{}`$ and Mg<sub>2</sub> on $`\mathrm{log}\sigma `$. These regressions are shown in the figure as dashed lines, and are:

Mg$`b^{}`$ $`=`$ $`(0.104\pm 0.011)\mathrm{log}\sigma (0.067\pm 0.026),`$ (4)   
Mg<sub>2</sub> $`=`$ $`(0.199\pm 0.016)\mathrm{log}\sigma (0.168\pm 0.038).`$ (5)   
As expected, the simple regressions yield slopes which are biased low due to the presence in the data of errors in the abscissa as well as the ordinate, and also the intrinsic scatter in the relation. Slightly less-biased results are obtained by least squares regression minimising the orthogonal residuals (cf. Jørgensen et al. 1996):

Mg$`b^{}`$ $`=`$ $`(0.109\pm 0.012)\mathrm{log}\sigma (0.078\pm 0.027),`$ (6)   
Mg<sub>2</sub> $`=`$ $`(0.215\pm 0.017)\mathrm{log}\sigma (0.205\pm 0.041).`$ (7)   
These least squares fits and their uncertainties are obtained using the slopes regression program written by E.D.Feigelson and described in Isobe et al. (1990) and Feigelson & Babu (1992). The uncertainties are under-estimated because these regressions do not properly account for the measurement errors or the selection functions. We conclude that previous determinations of the slope of the Mg–$`\sigma `$ relation are likely to be biased low whenever the dataset being fitted had significant errors in the velocity dispersions (as is generally the case). Hereafter we adopt the ML fits to the Mg–$`\sigma `$ relation.

The distributions of the residuals in Mg$`b^{}`$ and Mg<sub>2</sub> about the ML fits are shown in the insets to Figures 2a and 2b. In order to minimise the effects of outliers, we robustly estimate the scatter about the Mg–$`\sigma `$ relations as half the range spanned by the central 68% of the data points. We find an observed scatter of 0.022$`\pm `$0.002 mag about the Mg$`b^{}`$$`\sigma `$ relation and 0.031$`\pm `$0.003 mag about the Mg<sub>2</sub>$`\sigma `$ relation. Excluding outliers, the distributions of residuals are very well fitted by Gaussians parametrised by the median residual and the robustly estimated scatter. There is no evidence for a tail of negative residuals such as noted by Burstein et al. (1988) and Jørgensen et al. (1996). As the latter authors point out, the presence of such a tail is sensitive to the adopted slope of the Mg–$`\sigma `$ relation. Some giant ellipticals do, however, have intrinsically weak Mg linestrengths for their velocity dispersions (Schweizer et al. 1990).

The estimates of the intrinsic scatter about the relations that are provided by the ML fit may be exaggerated by outliers or by deviations of the underlying distribution of galaxies in the Mg–$`\sigma `$ plane from a bivariate Gaussian. We therefore drop the assumption of an intrinsic bivariate Gaussian distribution in the Mg–$`\sigma `$ plane and use Monte Carlo simulations based on the observed distribution of dispersions and linestrengths and their estimated errors (accounting for both measurement errors and run correction errors). These simulations assume only that there is a global linear Mg–$`\sigma `$ relation about which there is Gaussian intrinsic scatter. We vary this intrinsic scatter and compute the robust estimate of the observed scatter about the fit (the half-width of the central 68% of the residuals) for the simulated distributions. The results of these simulations are presented in Figure 4, which shows the normalised likelihood distributions for the intrinsic scatter in Mg$`b^{}`$ and Mg<sub>2</sub> given the observed scatter. We find that to account for the observed scatter in the relations we require an intrinsic scatter of 0.016$`\pm `$0.001 mag for Mg$`b^{}`$$`\sigma `$ and 0.023$`\pm `$0.002 mag for Mg<sub>2</sub>$`\sigma `$. The ratio of the intrinsic scatter in Mg<sub>2</sub> to the intrinsic scatter in Mg$`b^{}`$ is slightly lower than expected from the observed Mg<sub>2</sub>Mg$`b^{}`$ relation, Mg<sub>2</sub>$``$1.94Mg$`b^{}`$$``$0.05 (see Paper 2).

Table 1 compares our fits to the Mg–$`\sigma `$ relations obtained by other authors, and gives the observed scatter $`\delta `$Mg<sub>obs</sub> and the intrinsic scatter $`\delta `$Mg<sub>int</sub> in the relations obtained in each case. For both Mg$`b^{}`$$`\sigma `$ and Mg<sub>2</sub>$`\sigma `$ the slopes we obtain are about 25% steeper than those obtained by most previous authors. This is not due to a difference in our data, but stems from our use of the ML method rather than regressions. In this situation regressions are biased towards flatter slopes than the true relation because they ignore the intrinsic scatter, the presence of errors in both variables and the selection function of the dataset. The standard or orthogonal regression fits to our data, which our simulations show under-estimate the slope of the relations, give results very similar to those obtained by other authors.

If we divide the sample by morphological type, we find that the cDs have a zeropoint which is 0.009 mag higher than that of the other early-type galaxies in Mg$`b^{}`$, and 0.014 mag higher in Mg<sub>2</sub>. These differences in the zeropoints are readily apparent in the distributions of residuals about the global Mg–$`\sigma `$ relations (see the insets to Figures 2a&b), and are significant at the 3$`\sigma `$-level. Despite these zeropoint offsets, including or excluding the cDs changes the scatter about the ML fit by less than its uncertainty, as they make up only 10% of the whole sample.

We find no significant differences, however, if we compare the Mg–$`\sigma `$ relations for the two volumes of space, the Hercules-Corona-Borealis and Perseus-Pisces-Cetus regions, from which our sample is drawn. The two regions have Mg–$`\sigma `$ relations with slopes and zeropoints which are consistent both with each other and with the overall Mg–$`\sigma `$ relations, providing a check that there are no gross systematic environmental differences between these two regions.

### 3.2 Cluster-to-cluster variations

We do not have enough galaxies per cluster to fit both the slope and the zeropoint of the Mg–$`\sigma `$ relations on a cluster-by-cluster basis, even in our best-sampled clusters. We therefore limit ourselves to investigating the variation in the Mg–$`\sigma `$ zeropoint. To this end we measured the median offset in Mg$`b^{}`$ and Mg<sub>2</sub> from the global fits given above for the clusters with three or more linestrength measurements (75 clusters for Mg$`b`$ and 72 for Mg<sub>2</sub>). Note that we only used galaxies that are cluster members based on their redshifts (see Paper 2). The results are not changed significantly if we use all clusters, or only clusters with five or more measurements.

The top panels of Figure 5 show these zeropoint offsets as a function of cluster ID number (CID), while the middle panels show the distributions of the offset values. The robustly-estimated scatter in the zeropoint offsets is 0.012$`\pm `$0.002 mag in Mg$`b^{}`$$`\sigma `$ and 0.019$`\pm `$0.004 mag in Mg<sub>2</sub>$`\sigma `$, showing that the relations are remarkably uniform among the aggregates of galaxies in the EFAR sample. The bottom panels in the figure plot the same offsets as a function of redshift, showing that there is no dependence of the relations on relative distance within the sample.

This scatter in the zeropoint offsets could purely be a consequence of a galaxy-to-galaxy scatter in a global Mg–$`\sigma `$ relation, or it could also require a variation in the zeropoint of the relation from cluster to cluster. These possibilities were examined by extending the simulations described in the previous section, adding a further source of scatter to the Mg–$`\sigma `$ relation in the form of an intrinsic variation between clusters in the zeropoint of the relation. For simplicity we assume that this variation also has a Gaussian distribution.

We find that if we make the extreme assumption that there is cluster-to-cluster scatter but no intrinsic scatter between galaxies within a cluster, then zeropoint variations between clusters with an rms of 0.009 mag in Mg$`b^{}`$ and 0.015 mag in Mg<sub>2</sub> are required to recover the observed cluster-to-cluster scatter. However this model under-predicts the observed scatter about the global relation, giving 0.017$`\pm `$0.001 mag for Mg$`b^{}`$ and 0.025$`\pm `$0.002 mag for Mg<sub>2</sub> compared to the actual values of 0.022$`\pm `$0.002 mag and 0.031$`\pm `$0.003 mag. On the other hand, if we assume that there is no zeropoint variation between clusters, then the intrinsic scatter between galaxies required to recover the observed scatter in the global relation (0.016 mag in Mg$`b^{}`$ and 0.023 mag in Mg<sub>2</sub>; see previous section) predicts a scatter in the cluster zeropoints of 0.012$`\pm `$0.001 mag in Mg$`b^{}`$ and 0.016$`\pm `$0.002 mag in Mg<sub>2</sub>, which is consistent with the observed values of 0.012$`\pm `$0.002 mag and 0.019$`\pm `$0.004 mag within the joint errors.

We conclude that there is no evidence for significant intrinsic zeropoint variations between clusters, since sampling a galaxy population drawn from a single global relation with intrinsic scatter consistent with the observations can account for the zeropoint differences between our clusters.

### 3.3 Variation with cluster properties

As there is very little change in the zeropoint of the Mg–$`\sigma `$ relation from cluster to cluster, it follows that there can be at most only a weak dependence of the zeropoint on the properties of the clusters. Here we investigate the effect of cluster properties on the stellar populations as reflected in the Mg–$`\sigma `$ zeropoints, considering cluster velocity dispersions, X-ray luminosities and X-ray temperatures (all indicators of cluster mass). The cluster dispersions come from Table 7 of Paper 2, using redshifts both from EFAR and from the ZCAT catalogue (Huchra et al. 1992; version of 1997 May 29). X-ray luminosities and temperatures are available for 26 of our 84 clusters in the homogeneous and flux-limited catalogue of X-ray properties of Abell clusters by Ebeling et al. (1996) based on ROSAT All-Sky Survey data. The X-ray luminosities are determined to a typical precision of about 20%. In order to have comparable precision in the cluster velocity dispersions, we only use clusters with dispersions computed from at least 20 galaxy redshifts; this also leaves 26 clusters, 17 of which are in common with the X-ray subsample.

Figure 6 shows the offsets in the Mg–$`\sigma `$ relations as functions of $`\mathrm{log}\sigma _{cluster}^2`$, $`\mathrm{log}L_X`$ and $`kT`$. Applying the Spearman rank correlation statistic, we find that there is no significant correlation between the Mg–$`\sigma `$ offsets and any of these quantities, and thus no evidence for a trend in the zeropoint of the Mg–$`\sigma `$ relation with cluster mass. Weighted regressions give best-fit relations and their uncertainties:

$`\mathrm{\Delta }\text{Mg}b^{}`$ $`=`$ $`(0.002\pm 0.009)\mathrm{log}\sigma _{cl}^2+(0.016\pm 0.050),`$ (8)   
$`\mathrm{\Delta }\text{Mg}\text{2}`$ $`=`$ $`(0.001\pm 0.011)\mathrm{log}\sigma _{cl}^2+(0.002\pm 0.060).`$ (9)   
If we take a complementary approach, splitting the clusters into two subsamples about the median value of $`L_X`$ and fitting the Mg–$`\sigma `$ relations to the galaxies of the high-$`L_X`$ and low-$`L_X`$ clusters separately, we again find no significant differences in the slopes or the zeropoints of the fits, which are compatible with the global fits obtained above.

## 4 DISCUSSION

There are at least four main questions which can be addressed using the above results.

(i) What are the theoretical implications of the lack of correlation between the mass of a cluster and the zeropoint of the Mg–$`\sigma `$ relation for cluster galaxies?

(ii) What effect do the stellar population differences implied by the observed variations in the Mg–$`\sigma `$ relation have on Fundamental Plane estimates of distances and peculiar velocities?

(iii) What constraint does the intrinsic scatter about the Mg–$`\sigma `$ relation place on the spread in age, metallicity and mass-to-light ratio amongst early-type galaxies in clusters?

(iv) What further constraints on these quantities result from combining the scatter about the Mg–$`\sigma `$ relation with the intrinsic scatter about the Fundamental Plane?

### 4.1 Mg–$`\sigma `$ zeropoint and cluster mass

The small scatter in the zeropoint of the Mg–$`\sigma `$ relation from cluster to cluster, and in particular the lack of correlation between the Mg–$`\sigma `$ zeropoint and the cluster mass, seems to imply that the mass over-density on Mpc scales in which an early-type galaxy is found has little connection with its stellar population and star-formation history.

The variation of the Mg–$`\sigma `$ relation with cluster properties has previously been studied in a sample of 11 nearby clusters by Jørgensen et al. (1996) and Jørgensen (1997). Following Guzmán et al. (1992), these authors look for a trend in Mg<sub>2</sub>$`\sigma `$ offsets with the ‘local density’ within clusters. The estimator of local density used is $`\rho _{cluster}=\mathrm{log}\sigma _{cluster}^2\mathrm{log}R`$, where $`R`$ is the projected distance of the galaxy from the cluster centre. Since $`R`$ is only a lower limit on the galaxy’s true distance from the cluster centre, this is a rather poor estimator of the true local density. Jørgensen et al. find that the residuals in Mg<sub>2</sub> show a weak trend<sup>1</sup><sup>1</sup>1The sign of the trend in equation 6 of Jørgensen (1997) is incorrect; the coefficient should be $`+`$0.009 (Jørgensen, priv.comm.) with local density, $`\mathrm{\Delta }\text{Mg}\text{2}0.009\rho _{cluster}`$. Since the residuals do not correlate with radius within the cluster (see Figure 3 of Jørgensen (1997)), but do show a significant correlation with cluster velocity dispersion, $`\mathrm{\Delta }\text{Mg}\text{2}0.02\mathrm{log}\sigma _{cluster}^2`$ (least-squares fit to the data in Figure 5 of Jørgensen (1997)), we would argue that a more straightforward interpretation of their results is a correlation of Mg<sub>2</sub>$`\sigma `$ zeropoint with total cluster mass rather than local density.

A correlation of this amplitude is formally consistent at the 2$`\sigma `$ level with the distribution of Mg<sub>2</sub> offsets versus $`\mathrm{log}\sigma _{cluster}^2`$ for the EFAR data (see equation 9); transforming Jørgensen’s result via the Mg<sub>2</sub>Mg$`b^{}`$ relation gives a correlation which is consistent at the 1.4$`\sigma `$ level with equation 8. We conclude that any correlation between the Mg–$`\sigma `$ relation zeropoint and the cluster mass is sufficiently weak (of order $`\mathrm{\Delta }\text{Mg}\text{2}0.02\mathrm{log}\sigma _{cluster}^2`$ or less) that it is not reliably established by the existing data, which are consistent with no correlation at all.

Semi-analytic models for the formation of elliptical galaxies, which previously neglected metallicity effects (see Kauffmann 1996, Baugh et al. 1996), are only now beginning to incorporate chemical enrichment and successfully reproduce the general form of the observed colour–magnitude and Mg–$`\sigma `$ relations (Kauffmann & Charlot 1998). In consequence, there are as yet no reliable predictions for the variation of the Mg–$`\sigma `$ relation zeropoint with cluster mass. The limits given above, together with limits on the difference in Mg–$`\sigma `$ zeropoints for field and cluster ellipticals (Burstein et al. 1990, de Carvalho & Djorgovski 1992, Jørgensen 1997), should provide valuable additional constraints and encourage further development of chemical enrichment models within a hierarchical framework for galaxy and cluster formation.

### 4.2 Systematic effects on FP distances

We now consider the effects on FP distance estimates of systematic differences in the stellar populations of early-type galaxies from cluster to cluster. In §3.2 we found that the observed cluster-to-cluster variations in the Mg–$`\sigma `$ zeropoint were consistent with sampling a single global Mg–$`\sigma `$ relation with intrinsic scatter between galaxies, and did not require intrinsic scatter between clusters. Here we turn the question around and ask how much intrinsic cluster-to-cluster scatter is allowed by the observations.

From simulations using the model described in §3.2, incorporating intrinsic scatter both between galaxies and between clusters, we find that the maximum cluster-to-cluster scatter allowed within the 1$`\sigma `$ uncertainties in the scatter in the global Mg–$`\sigma `$ relation and the cluster zeropoints is approximately 0.005 mag in Mg$`b^{}`$ and 0.010 mag in Mg<sub>2</sub>. For our best-fit ML Mg–$`\sigma `$ relations and a FP given by $`R`$$``$$`\sigma ^\alpha I^\beta `$ with $`\alpha `$$``$1.27, this level of cluster-to-cluster scatter would lead to rms errors in FP distance estimates of up to 10%. These systematic errors, resulting from differences in the mean stellar populations between clusters, would apply even to clusters in which the FP distance errors due to stellar population differences between galaxies had been made negligible by observing many galaxies in the cluster.

We emphasise that our results here do not require any cluster-to-cluster scatter, but are consistent with cluster-to-cluster scatter corresponding to systematic distance errors between clusters with an rms of up to 10%. We therefore cannot determine from the Mg–$`\sigma `$ relation alone whether systematic differences in the mean stellar populations between clusters contribute significantly (or at all) to the errors in FP estimates of distances and peculiar velocities. A more effective way of testing for such systematic differences is by directly comparing each cluster’s zeropoint offset from the global Mg–$`\sigma `$ relation to the ratio of its FP and Hubble distance estimates; this approach will be investigated in a future paper.

### 4.3 Stellar population models

To answer the questions concerning the typical age, metallicity and mass-to-light ratio of early-type galaxies which were raised at the start of this discussion, we need to employ stellar population models. We use the predictions from the single stellar population models of Worthey (1994) and Vazdekis et al. (1996), noting the many caveats given by these authors regarding their models. To simplify our analysis, we fit Mg$`b^{}`$, Mg<sub>2</sub> and $`\mathrm{log}M/L_R`$ as linear functions of logarithmic age ($`\mathrm{log}t`$, with $`t`$ in Gyr) and metallicity ($`\mathrm{log}Z/Z_{}`$), for galaxies with ages greater than 4 Gyr and metallicities in the range $``$0.5 to $`+`$0.5. For the model of Worthey (1994; Salpeter IMF) we obtain

Mg$`b^{}`$ $``$ $`0.058\mathrm{log}t+0.086\mathrm{log}Z/Z_{}+0.077`$ (10)   
Mg<sub>2</sub> $``$ $`0.107\mathrm{log}t+0.182\mathrm{log}Z/Z_{}+0.147`$ (11)   
$`\mathrm{log}M/L_R`$ $``$ $`0.825\mathrm{log}t+0.184\mathrm{log}Z/Z_{}0.169.`$ (12)   
Figure 7 compares this fit to Worthey’s model in the case of the predicted dependence of Mg$`b^{}`$ and $`\mathrm{log}M/L_R`$ on age and metallicity. The figure shows that for ages of 5 Gyr or greater the fit and the model are in satisfactory agreement for all metallicities.

For the model of Vazdekis et al. (1996; bimodal IMF, $`\mu `$=1.35) we have

Mg$`b^{}`$ $``$ $`0.051\mathrm{log}t+0.083\mathrm{log}Z/Z_{}+0.084`$ (13)   
Mg<sub>2</sub> $``$ $`0.115\mathrm{log}t+0.187\mathrm{log}Z/Z_{}+0.137`$ (14)   
$`\mathrm{log}M/L_R`$ $``$ $`0.673\mathrm{log}t+0.251\mathrm{log}Z/Z_{}0.216,`$ (15)   
in agreement with the fit obtained by Jørgensen (1997). There is good agreement between the predictions of the two models for the dependence of Mg<sub>2</sub> and Mg$`b^{}`$ on age and metallicity, and moderately good agreement for the dependence of $`M/L_R`$.

Note that the same change in the Mg indices is produced by changes in age, $`\mathrm{\Delta }\mathrm{log}t`$, and metallicity, $`\mathrm{\Delta }\mathrm{log}Z/Z_{}`$, if $`\mathrm{\Delta }\mathrm{log}t/\mathrm{\Delta }\mathrm{log}Z/Z_{}3/2`$. This is the ‘3/2 rule’ of Worthey (1994), which applies to many of the Lick line indices, leaving them degenerate with respect to variations in age and metallicity. However age and metallicity produce the same change in $`\mathrm{log}M/L_R`$ only if $`\mathrm{\Delta }\mathrm{log}t/\mathrm{\Delta }\mathrm{log}Z/Z_{}1/3`$ or 1/4, so that measurements of mass-to-light ratios can in principle be combined with Mg linestrengths to break the age/metallicity degeneracy.

### 4.4 Dispersion in age, metallicity and $`M/L`$

In the following analysis we infer the dispersion in the ages and metallicities of early-type galaxies by comparing the scatter in the Mg–$`\sigma `$ relation with the predictions of the single stellar population models described in the previous section. This analysis uses the stellar population models to predict differential changes in the quantities of interest, and not absolute values. It is also important to remember that by the dispersion in age or metallicity we mean the dispersion in these quantities at fixed $`\mathrm{log}\sigma `$ or, equivalently, the dispersion after the overall trend with $`\mathrm{log}\sigma `$ is accounted for. Thus the dispersion in age or metallicity we infer is the dispersion at fixed galaxy mass, not the distribution of ages and metallicities as a function of galaxy mass (which is related to the slope of the Mg–$`\sigma `$ relation and the distribution of galaxies along it).

Single stellar populations models specified by (amongst other parameters) a unique age and a unique metallicity can only provide an approximation to real galaxies, whose stellar contents must necessarily span a range (though perhaps a narrow one) of ages and metallicities. Since the global Mg indices can be quite sensitive to the detailed metallicity distribution (Greggio 1997), some of the scatter we observe may be due to galaxy-to-galaxy differences in the shape of the metallicity distribution rather than a dispersion in the mean metallicity or age.

A further complication is presented by the over-abundance of Mg with respect to Fe (compared to the solar ratio) in the cores of early-type galaxies (e.g. Peletier 1989, Gorgas et al. 1990, Worthey et al. 1992). As a comparison of Figures 27 shows, the models discussed in the previous section are unable to account for the highest observed Mg linestrengths. Tantalo et al. (1998) have produced single stellar population models including the effects of \[Mg/Fe\] variations and find that

$$\mathrm{\Delta }\text{Mg}\text{2}0.099\mathrm{\Delta }[\mathrm{Mg}/\mathrm{Fe}]+0.089\mathrm{\Delta }\mathrm{log}t+0.166\mathrm{\Delta }\mathrm{log}Z/Z_{}$$
(16)   
Comparing this equation with those above, we see that the differential dependence on age and metallicity is similar to that predicted by Worthey (1994) and Vazdekis et al. (1996). However, any intrinsic scatter in the \[Mg/Fe\]–$`\sigma `$ relation will contribute additionally to the intrinsic scatter in the Mg<sub>2</sub>$`\sigma `$ relation and reduce the dispersion in age and metallicity required to account for the observations.

For these reasons, and also because of other potential sources of intrinsic scatter such as dark matter, rotation, anisotropy, projection effects and broken homology, the estimates of the dispersion in age and metallicity derived here must be considered as upper limits.

With these caveats in mind, we proceed to use the model fits given in the previous section to infer the dispersion in age or metallicity based on the observed intrinsic scatter of 0.016 mag in Mg$`b^{}`$$`\sigma `$ and 0.023 mag in Mg<sub>2</sub>$`\sigma `$. For ease of interpretation we quote the dispersions in age and metallicity as the fractional dispersions $`\delta t/t\delta \mathrm{log}t/\mathrm{log}e`$ and $`\delta Z/Z\delta \mathrm{log}Z/\mathrm{log}e`$. In applying the models in what follows, we adopt the mean of the coefficients for the two models and give the dispersions in age and metallicity corresponding to the intrinsic scatter about the Mg$`b^{}`$$`\sigma `$ relation. Using the intrinsic scatter obtained from the Mg<sub>2</sub>$`\sigma `$ relation would give results that are $``$30% smaller, since the observed ratio of the intrinsic scatters is $`\delta \text{Mg}\text{2}/\delta \text{Mg}b^{}`$$``$1.4, rather than about 2 as would be expected either from the observed Mg<sub>2</sub>Mg$`b^{}`$ relation or from the models. We use the scatter in Mg$`b^{}`$ rather than Mg<sub>2</sub> because our goal is to establish upper limits on the dispersions in age and metallicity. The estimated errors in the intrinsic scatter lead to uncertainties in the dispersions of 5–10%.

If age variations in single stellar populations are the only source of scatter then the dispersion in age is $`\delta t/t`$=67%, whereas if metallicity variations are the sole source then the dispersion in metallicity is $`\delta Z/Z`$=43%. Similarly, the observed difference in the Mg–$`\sigma `$ relation zeropoint for the cD galaxies implies that these objects are either older or more metal-rich than normal E or E/S0 galaxies. If the zeropoint differences are interpreted as age differences, cDs are on average 40% older than typical E or E/S0 galaxies (i.e. as old as the oldest early-type galaxies); if the zeropoint differences are interpreted as metallicity differences, cDs have metallicities on average 25% higher than typical E or E/S0 galaxies (i.e. as high as the most metal-rich early-type galaxies).

We can also use the model fits to estimate the approximate change in $`M/L_R`$ corresponding to a change in the Mg line indices. If these changes are caused by age variations alone, then we find that $`\mathrm{\Delta }\mathrm{log}M/L_R7\mathrm{\Delta }\text{Mg}\text{2}`$ and $`\mathrm{\Delta }\mathrm{log}M/L_R14\mathrm{\Delta }\text{Mg}b^{}`$; if, however, they are due only to variations in metallicity we have $`\mathrm{\Delta }\mathrm{log}M/L_R1.2\mathrm{\Delta }\text{Mg}\text{2}`$ and $`\mathrm{\Delta }\mathrm{log}M/L_R2.6\mathrm{\Delta }\text{Mg}b^{}`$. Thus the change in $`\mathrm{log}M/L_R`$ is about 5 times larger if the observed change in the Mg indices is due to age differences rather than metallicity differences. The intrinsic scatter in the Mg$`b^{}`$$`\sigma `$ relation implies a dispersion in mass-to-light ratio of 50% if due to age variations, but only 10% if due to metallicity variations.

This predicted scatter in $`M/L`$ is in fact a scatter in luminosity or surface brightness (since that is all the models deal with). We can therefore readily establish the effect of this scatter on distances estimated using the Fundamental Plane (FP) if the scatter in $`M/L`$ is uncorrelated with the galaxies’ sizes and dispersions, as indeed is the case for the EFAR sample (at least for galaxies with $`\sigma `$$`>`$100 km s<sup>-1</sup>). For a FP given by $`R`$$``$$`\sigma ^\alpha I^\beta `$, with $`R`$ the effective radius and $`I`$ the mean surface brightness within this radius, if the scatter in $`M/L`$ is simply a scatter in $`I`$ we have $`\mathrm{\Delta }\mathrm{log}R=\beta \mathrm{\Delta }\mathrm{log}M/L`$. Most determinations of the FP, including our own, yield $`\beta 0.8`$ (e.g. Dressler et al. 1987, Jørgensen et al. 1996, Saglia et al. 1998).

Combining this relation with the dependence of $`M/L`$ on the Mg line indices obtained above, we find that the scatter in the Mg–$`\sigma `$ relation corresponds to an intrinsic scatter in relative distances estimated from the FP of 40% if due to age variations, or 8% if due to metallicity variations. As the intrinsic scatter in the FP is found to be in the range 10–20% (Djorgovski & Davis 1987, Jørgensen et al. 1993, Jørgensen et al. 1996), one cannot explain both the scatter in the Mg–$`\sigma `$ relation and the scatter in the FP as the result of age variations alone or metallicity variations alone (unless the single stellar population models are incorrect or there are significant galaxy-to-galaxy differences in the metallicity distributions). Suitable combinations of age variations and metallicity variations can, however, account for the measured intrinsic scatter in both the Mg–$`\sigma `$ and FP relations.

### 4.5 Combined Mg–$`\sigma `$ and FP constraints

As a simple model, we assume that the scatter in the FP and the Mg–$`\sigma `$ relations (at fixed $`\mathrm{log}\sigma `$) is entirely due to variations in age and metallicity (at fixed galaxy mass). These variations are further assumed to have Gaussian distributions in $`\mathrm{log}t`$ and $`\mathrm{log}Z/Z_{}`$ with dispersions $`\delta \mathrm{log}t\delta t/t\mathrm{log}e`$ and $`\delta \mathrm{log}Z\delta Z/Z\mathrm{log}e`$ and correlation coefficient $`\rho `$ ($``$1$``$$`\rho `$$``$1). While a Gaussian distribution of metallicities at fixed galaxy mass is a reasonable initial hypothesis for describing variations in the chemical enrichment process, the single-peaked shape of the assumed lognormal distribution for the mean ages may not realistically represent the star-formation history (even for galaxies of the same mass). The dispersion in age inferred under this model should therefore be considered only as a general indication of the time-span over which early-type galaxies of fixed mass formed the bulk of their stellar population.

Writing the scatter in Mg linestrengths and FP residuals as $`\delta _{\mathrm{Mg}}`$ and $`\delta _{\mathrm{FP}}`$ and the dispersion in $`\mathrm{log}t`$ and $`\mathrm{log}Z/Z_{}`$ as $`\sigma _t`$ and $`\sigma _Z`$, this simple model relates the scatter in the observed quantities to the dispersion in age and metallicity by:

$`\delta _{\mathrm{Mg}}^2`$ $`=`$ $`a_t^2\sigma _t^2+2\rho a_ta_Z\sigma _t\sigma _Z+a_Z^2\sigma _Z^2`$ (17)   
$`\delta _{\mathrm{FP}}^2`$ $`=`$ $`b_t^2\sigma _t^2+2\rho b_tb_Z\sigma _t\sigma _Z+b_Z^2\sigma _Z^2.`$ (18)   
Here $`a_t`$ and $`a_Z`$ are the coefficients of $`\mathrm{log}t`$ and $`\mathrm{log}Z/Z_{}`$ for Mg, and $`b_t`$ and $`b_Z`$ the coefficients for $`\mathrm{log}R=0.8\mathrm{log}M/L`$, derived from the mean of the linear fits to the two stellar population models given in §4.3.

Figure 8 shows the constraints on the variations in age and metallicity (assumed for now to be uncorrelated) which are imposed by the measured intrinsic scatter in the Mg–$`\sigma `$ relations and the intrinsic dispersion in $`\mathrm{log}M/L_R`$ inferred from the intrinsic scatter in the FP. The intrinsic scatter we find about the Mg$`b^{}`$$`\sigma `$ and Mg<sub>2</sub>$`\sigma `$ relations is then consistent with dispersions in age and metallicity on an elliptical locus defined by equation 17 (with $`\rho `$=0) in the $`\delta t/t`$$`\delta Z/Z`$ plane. The different loci for Mg$`b^{}`$ and Mg<sub>2</sub> (the solid lines in Figure 8) result from the difference between the observed ratio of the scatter in Mg<sub>2</sub> to that in Mg$`b^{}`$ and the predicted ratio from the model, and give some indication of uncertainties both in the intrinsic scatter about the Mg–$`\sigma `$ relations and in the model predictions. A second constraint is similarly obtained from the intrinsic scatter in distance (i.e. in $`\mathrm{log}R`$) about the FP using equation 18 (again with $`\rho `$=0). The dashed lines in Figure 8 correspond to intrinsic scatter about the FP of 10%, 15% and 20%.

The important point to note about the figure is that, as mentioned in §4.3, the dependences of the Mg linestrengths and mass-to-light ratio on age and metallicity are quite different, so that (if variations in age and metallicity are uncorrelated) the two sets of constraints are nearly orthogonal. Thus the region of the $`\delta t/t`$$`\delta Z/Z`$ plane that is consistent with the scatter in both the Mg–$`\sigma `$ relation and the FP is quite limited. If we use the intrinsic scatter in the Mg$`b^{}`$$`\sigma `$ relation and assume a 20% intrinsic scatter in $`\mathrm{log}R`$ about the FP (at the upper end of the quoted range—see, e.g., Djorgovski & Davis (1987) or Jørgensen et al. (1996)), we obtain approximate upper limits on the dispersions in age and metallicity of $`\delta t/t`$=32% and $`\delta Z/Z`$=38%. If, however, we use the intrinsic scatter in the Mg<sub>2</sub>$`\sigma `$ relation and adopt an intrinsic FP scatter of 10% (as obtained for Coma by Jørgensen et al. 1993), then we obtain approximate lower limits of $`\delta t/t`$=15% and $`\delta Z/Z`$=27%.

Similar arguments allow us to evaluate the relative contributions of the dispersions in age and metallicity to the errors in distance estimates derived from the FP. For the fiducial case ($`\delta `$FP=20%, $`\delta `$Mg$`b^{}`$=0.016 mag and $`\rho `$=0), where $`\delta t/t`$=32% and $`\delta Z/Z`$=38%, the mean stellar population model implies that the dispersion in age gives an intrinsic FP scatter of 19% while the dispersion in metallicity gives 7%. In fact for most of the plausible range of dispersions in age and metallicity shown in Figure 8, it is the dispersion in age which dominates the intrinsic scatter about the FP. Only for the lowest plausible age dispersion and the highest plausible metallicity dispersion ($`\delta t/t`$=11% and $`\delta Z/Z`$=43%, corresponding to $`\delta `$FP=10% and $`\delta `$Mg$`b^{}`$=0.016 mag) does the contribution to the FP scatter from the dispersion in metallicity achieve equality with the contribution from the dispersion in age.

The constraints on the dispersions change if there is a significant correlation (or anti-correlation) between the variations in age and metallicity. Figure 9 shows how the constraints corresponding to the upper limits $`\delta `$FP=20% and $`\delta `$Mg$`b^{}`$=0.016 mag (corresponding to the thick lines in Figure 8) are modified as the correlation coefficient $`\rho `$ varies over its full range from $``$1 to 1. Note that for $`\rho =\pm 1`$ we have $`\delta t/t3/2\delta Z/Z`$. The main point to extract from this figure is that if the variations in age and metallicity have a correlation coefficient in the range $``$0.5$`<`$$`\rho `$$`<`$1, then the dispersions in age and metallicity vary by only $`\pm `$6% and $`\pm `$12% respectively about the values inferred in the uncorrelated case. Only if the age and metallicity variations are strongly anti-correlated ($`\rho `$$``$$``$1; i.e. younger galaxies are more metal-rich) do we obtain significantly different solutions, with a broader allowed range in both age and metallicity ($`\delta t/t`$ as large as 57% and $`\delta Z/Z`$ as large as 80%). This conclusion is complementary to that reached by Ferreras et al. (1998), who find that the apparently passive evolution of the colour–magnitude relation observed in high-redshift clusters does not necessarily imply a common epoch of major star-formation if younger galaxies are on average more metal-rich.

We can test the degree of correlation between the variations in age and metallicity by examining the joint distribution of residuals about the Mg$`b^{}`$$`\sigma `$ and FP relations. This distribution is shown for the EFAR data set in Figure 10a. There is no evidence for a correlation between the residuals in this figure; the Spearman rank correlation coefficient between the residuals is 0.084, and is not significant at the 2$`\sigma `$ level.

In order to investigate the expected distribution of residuals in the presence of the estimated measurement errors, we have performed Monte Carlo simulations of the EFAR data using the models for the dispersion in age and metallicity discussed above. Figure 10b shows a simulation with $`\delta t/t`$=32% and $`\delta Z/Z`$=38%; these are the values derived from the intrinsic scatter in the Mg$`b^{}`$$`\sigma `$ relation and a FP scatter of 20% when there is no correlation between age and metallicity. The simulated distribution resembles the observed distribution, although there is a weak but significant anti-correlation between the residuals (due to the dominance of the age variations in the FP residuals) which is not apparent in the EFAR data. Over 100 such simulations, a two-dimensional K-S test (Press et al. 1992) gives a median probability of 0.3% that this distribution and the observed distribution are the same.

Figures 10c&d show simulated distributions for the cases where the intrinsic scatter in Mg$`b^{}`$$`\sigma `$ is due to age alone or metallicity alone. Neither case is consistent with the observed distribution, supporting the claim that neither age nor metallicity can be solely responsible for the scatter in both the Mg$`b^{}`$$`\sigma `$ relation and the FP. Figures 10e–h show simulated distributions for four cases where the variations in age and metallicity are correlated (with $`\rho `$=+1, +0.5, $``$0.5 and $``$1 respectively). The perfectly correlated and perfectly anti-correlated cases are not consistent with the observed distribution. However Figure 10g shows that a distribution with no significant correlation between the Mg$`b^{}`$$`\sigma `$ and FP relation residuals is produced when $`\rho `$=$``$0.5. A two-dimensional K-S test gives a median probability over 100 such simulations of 1.7% that this distribution and the observed distribution are the same. This relatively low probability may reflect a problem with the model, although it may simply be due to sampling uncertainty (the probabilities under this test vary between simulations with an rms of a factor of 6) or non-Gaussian outliers in the EFAR residuals. The point to be emphasised is that a model with a moderate degree of anti-correlation between age and metallicity appears to give significantly better agreement with the observed distribution than a model in which age and metallicity are uncorrelated.

## 5 CONCLUSIONS

We have examined the Mg–$`\sigma `$ relation for early-type galaxies in the EFAR sample. We fit global Mg$`b^{}`$$`\sigma `$ and Mg<sub>2</sub>$`\sigma `$ relations (equations 2 and 3) that have slopes about 25% steeper than those obtained by most previous authors. This difference results not from the data itself but from an improved fitting procedure: we apply a comprehensive maximum likelihood approach which correctly accounts for the biases introduced by both the sample selection function and the significant errors in both Mg and $`\sigma `$. The observed scatter about the Mg–$`\sigma `$ relations is 0.022 mag in Mg$`b^{}`$ and 0.031 mag in Mg<sub>2</sub>; the intrinsic scatter in the global relations, estimated from Monte Carlo simulations, is 0.016 mag in Mg$`b^{}`$ and 0.023 mag in Mg<sub>2</sub>.

With too few galaxies per cluster to reliably determine the full relation for each cluster separately, we fix the slopes of the relations at their global values in order to investigate the variation in the zeropoint from cluster to cluster. We find that the zeropoint has an observed scatter between clusters of 0.012 mag in Mg$`b^{}`$ and 0.019 mag in Mg<sub>2</sub>, and that this observed scatter is consistent with the small number of galaxies sampled in each cluster being drawn from a single global relation with intrinsic scatter between galaxies as given above—i.e. the observations do not require any scatter in the Mg–$`\sigma `$ zeropoint between clusters. The allowed range for the intrinsic scatter between clusters corresponds to cluster-to-cluster systematic errors in Fundamental Plane distances and peculiar velocities with an rms anywhere in the range 0–10%. We therefore cannot determine from the Mg–$`\sigma `$ relation alone whether systematic differences in the mean stellar populations between clusters contribute significantly (or at all) to the errors in distances and peculiar velocities obtained using the Fundamental Plane.

We have also examined the variation in the Mg–$`\sigma `$ relation with cluster properties. Our cluster sample ranges from poor clusters to clusters as rich as Coma, having velocity dispersions from 300 km s<sup>-1</sup> to 1000 km s<sup>-1</sup> and X-ray luminosities spanning 0.3–8$`\times `$10<sup>44</sup> erg s<sup>-1</sup>. We do not detect a significant correlation of Mg–$`\sigma `$ zeropoint with cluster velocity dispersion, X-ray luminosity or X-ray temperature, nor is there any significant difference in the Mg–$`\sigma `$ relations obtained by fitting the galaxies in the high-$`L_X`$ clusters and low-$`L_X`$ clusters separately. The predominant factor in the production of Mg in these early-type galaxies (and presumably other $`\alpha `$-elements and perhaps their metallicity and star-formation history in general) is thus galaxy mass and not cluster mass. These observations place constraints on semi-analytic models for the formation of elliptical galaxies, which are now beginning to incorporate chemical enrichment and should soon be able to make reliable predictions for the variation of the Mg–$`\sigma `$ relation with cluster mass.

We apply the single stellar population models of Worthey (1994) and Vazdekis et al. (1996) to place upper limits on the global dispersion in the ages, metallicities and $`M/L`$ ratios of early-type galaxies of given mass using the intrinsic scatter in the global Mg–$`\sigma `$ relation. We infer an upper limit on the dispersion in $`M/L_R`$ of 50% if the scatter in Mg–$`\sigma `$ is due to age differences alone, or 10% if it is due to metallicity differences alone. These correspond to upper limits on the dispersion in relative galaxy distances estimated from the Fundamental Plane (FP) of 40% (age alone) or 8% (metallicity alone). Since the intrinsic scatter in the FP is found to be 10–20%, one cannot (within the context of the single stellar population models) explain both the scatter in the Mg–$`\sigma `$ relation and the scatter in the FP as the result of age variations alone or metallicity variations alone.

We therefore determine the joint range of dispersions in age and metallicity which are consistent with the measured intrinsic scatter in both the Mg–$`\sigma `$ and FP relations. For a simple model in which the galaxies have independent Gaussian distributions in $`\mathrm{log}t`$ and $`\mathrm{log}Z/Z_{}`$, we find upper limits of $`\delta t/t`$=32% and $`\delta Z/Z`$=38% at fixed galaxy mass. If the variations in age and metallicity are not independent, but have correlation coefficient $`\rho `$, we find that so long as $`\rho `$ is in the range $``$0.5 to 1 these limits on the dispersions in age and metallicity change by only $`\pm `$6% and $`\pm `$12% respectively. Only if the age and metallicity variations are strongly anti-correlated ($`\rho `$$``$$``$1) do we obtain significantly higher upper limits, with $`\delta t/t`$ as large as 57% and $`\delta Z/Z`$ as large as 80%. The distribution of the residuals from the Mg–$`\sigma `$ and FP relations is only marginally consistent with a model having no correlation between age and metallicity, and is better-matched by a model in which age and metallicity variations are moderately anti-correlated ($`\delta t/t`$$``$40%, $`\delta Z/Z`$$``$50% and $`\rho `$$``$$``$0.5), with younger galaxies being more metal-rich.

Stronger bounds on the dispersion in age and metallicity amongst early-type galaxies of given mass will require more precise measurements of the deviations from the Mg–$`\sigma `$ relation and the Fundamental Plane and also improved models for the dependence of the line indices and mass-to-light ratio on age and metallicity. Further powerful constraints can also be obtained by measuring the intrinsic scatter in the Mg–$`\sigma `$ and FP relations at higher redshifts, since the linestrengths and mass-to-light ratio have different dependences on age.

## Acknowledgements

MMC acknowledges the support of a DIST Collaborative Research Grant. DB was partially supported by NSF Grant AST90-16930. RLD thanks the Lorenz Centre and Prof. P.T. de Zeeuw. RKM received partial support from NSF Grant AST90-20864. RPS acknowledges the financial support by the Deutsche Forschungsgemeinschaft under SFB 375. GW is grateful to the SERC and Wadham College for a year’s stay in Oxford, to the Alexander von Humboldt-Stiftung for making possible a visit to the Ruhr-Universität in Bochum and to NSF Grants AST90-17048 and AST93-47714 for partial support. The entire collaboration benefitted from NATO Collaborative Research Grant 900159 and from the hospitality and monetary support of Dartmouth College, Oxford University, the University of Durham and Arizona State University. Support was also received from PPARC visitors grants to Oxford and Durham Universities and PPARC rolling grant ‘Extragalactic Astronomy and Cosmology in Durham 1994-98’. We thank the referee, Prof. Alvio Renzini, for a critique which resulted in substantial improvements to the paper.