{'fraction_non_alphanumeric': 0.02791403123789854, 'fraction_numerical': 0.01891054601781335, 'mean_word_length': 5.160834990059642, 'pattern_counts': {'":': 0, '<': 4, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
The perceived assortativity of social networks: Methodological problems and solutions

Networks describe a range of social, biological and technical phenomena. An important property of a network is its degree correlation or assortativity, describing how nodes in the network associate based on their number of connections. Social networks are typically thought to be distinct from other networks in being assortative (possessing positive degree correlations); well-connected individuals associate with other well-connected individuals, and poorly-connected individuals associate with each other. We review the evidence for this in the literature and find that, while social networks are more assortative than non-social networks, only when they are built using group-based methods do they tend to be positively assortative. Non-social networks tend to be disassortative. We go on to show that connecting individuals due to shared membership of a group, a commonly used method, biases towards assortativity unless a large enough number of censuses of the network are taken. We present a number of solutions to overcoming this bias by drawing on advances in sociological and biological fields. Adoption of these methods across all fields can greatly enhance our understanding of social networks and networks in general.


Introduction
Network theory is a useful tool that can help us explain a range of social, biological and technical phenomena (Pastor-Satorras et al 2001;Girvan and Newman 2002;Krause et al 2007). For instance, network approaches have been used to investigate diverse topics such as the global political and social system (Snyder and Kick 1979;Nemeth and Smith 2010) to the formation of coalitions among-individuals (Kapferer 1969;Zachary 1977;Thurman 1979;Voelkl and Kasper 2009). Networks can be described using a number of local (related to the individual) and global (related to the whole network) measures. One important global measure is degree correlation or assortativity (we use the latter term for brevity), which was formally defined by Newman (2002), although Pastor-Satorras et al. (2001) had calculated an analogous measure previously. The assortativity of a network measures how the probability of a connection between two nodes (individuals) in a network depends on the degrees of those two nodes (the degree being the number of connections each node possesses). The measure quantifies whether those with many connections associate with others with many connections (assortative networks or networks with assortativity), or if "hubs" form where well connected individuals are connected to many individuals with few other connections (disassortative networks or networks with dissassortativity). If the tendency for nodes to be connected is independent of each other's degrees a network has neutral assortativity.
Assortativity is calculated as the Pearson's correlation coefficient between the degrees of all pairs of connected nodes, and ranges from -1 to 1 (Newman 2002). The Pastor-Satorras method involves plotting the degree of each node against the mean degree of its neighbours, and judging the network assortatative if the slope is positive and disassortatative if the slope is negative (Pastor-Satorras et al 2001). In this article we will focus on the Newman measure, as it has been more commonly used by the scientific community, gives a coefficient bounded between -1 and 1 rather than a slope, and typically is supported by a statistical test, something general absent from the reporting of the Pastor-Satorras method.
Assortativity is a key property to consider when understanding how networks function, especially when considering social networks. A network's robustness to attacks is increased if it is assortative (Jing et al 2007;Hasegawa et al 2012). However, the speed of information transfer and ability to act in synchrony is increased in disassortative networks (Di Bernado et al 2007;Gallos et al 2008). Thus the assortativity of a social network in which an individual is embedded can have a substantial impact on that individual.
Networks are typically found to be neutrally or negatively assortative (Newman 2002;Newman and Park 2003;Whitney and Alderson 2008). When considering all networks comprised of eight nodes, Estrada (2011) observed that only 8% of over 11,000 possible networks were assortative. Despite this general trend, social networks are often said to differ from other networks by being assortative (Newman 2003;Newman and Park 2003). This has led to those finding disassortativity in networks of online interactions (Holme et al 2004), mythical stories (Mac Carron andKenna 2012) or networks of dolphin interactions (Lusseau and Newman 2004) to suggest they are different to typical human social networks. While the generality of assortativities in social networks has been questioned (Whitney and Alderson 2008;Hu and Wang 2009), a wide variety of recent research still states that this is a property typical of social networks (e.g. (Estrada 2011;Palathingal and Chirayath 2012;Mac Carron and Kenna 2013;Litvak and van der Hofstad 2013;Thedchanamoorthy et al 2014;Araújo et al 2014;Furtenbacher et al 2014)). With assortativity being a key network property, and subject of interest in a range of fields, this topic requires clarification.
In this paper we review assortativity in the networks literature, with emphasis on social networks. We assess the generality of the hypothesis that social networks tend to be distinct from other kinds of networks in their assortativities and explore whether the precise method of social network construction influences this metric. We go on to show how particular methodologies of social network construction could result in falsely assortative networks, and present a number of solutions to this by drawing on advances in sociological and biological fields.

Assortativity in social and other networks
Random networks should be neutrally assortative (Newman 2002). However, simulations by Franks et al. (2009) showed that random social networks constructed using group-based methods are assortative unless extensive sampling is carried out. Group-based methods are where links are formed between individuals not when they directly interact, but when they both are found in the same group, or contribute to a joint piece of work e.g. co-author a paper or appear in a band together. We hypothesised that the suggestion that social networks possess assortativity was due to a preponderance of social networks constructed using groupbased methods in the early literature. We thus conducted a literature search, recording the (dis)assortativity of networks, network types (social or non-social) and for the social networks, method of construction (direct interactions of group-based). We expect that social networks built using group-based methods would be more assortative than the other two classes of network, which would be similarly assortative.
1.1 Literature search -Method Assortativity has been calculated for a wide range of networks in the last decade. We compiled a list of the assortativity of published networks based on the table in Whitney & Alderson (Whitney and Alderson 2008), literature searches with the terms "degree correlation" and "assortativity", and examining the articles citing Newman (2002). If it could not be determined how the network was constructed, or how large it was, the network was excluded. We did not include the average assortativity when reported from a range of similar networks when the individual scores were not reported. We also did not include assortativities of networks from studies re-analysing existing datasets, to avoid pseudoreplication. Only undirected networks were considered; see Piraveenan et al. (2012) for a review of assortativities in directed networks. We then classified these networks as non-social networks, social networks constructed using direct interactions, or social networks constructed using group-based methods. We then compared assortativity across network classes using a Kruskal-Wallis test as assortativity scores were not normally distributed. If this revealed significant differences among network classes, we then compared network classes to each other using Wilcoxon rank sum tests and to the a neutral assortativity of zero with Wilcoxon signed rank tests.

Literature search -Results
In published papers we found assortativities for 88 networks that met our criteria for inclusion, see Table 1. 52 of these were social networks, of which 25 were constructed using group-based methods. The assortativities for the network classes are shown as boxplots in Fig. 1. The Kruskal-Wallis tests indicated that there were differences among groups (Kruskal-Wallis χ 2 = 26.8, d.f. = 2, p < 0.001). All networks types were different from each other, with the group-based social networks being more assortative than both other classes of network, and the direct social networks being more assortative than the non-social networks (all Wilcoxon rank sum tests, group based social networks vs. direct social networks, W = 214, n(direct) = 27, n(group-based) = 25, p = 0.024; group based social networks vs. non-social networks, W = 783, n(group-based social networks) = 25, n(non-social network) = 36, p < 0.001; direct social networks vs. non-social network, W = 716, n(direct social networks = 27), n(non-social network) = 36, p = 0.001). The direct social networks possessed neutral assortativities (mean = 0.054, Wilcoxon signed rank test, V = 213, n = 27, p =0.572), while the group-based social networks were assortative (mean = 0.157, Wilcoxon signed rank test, V = 288, n = 25, p < 0.001) and the non-social networks were disassortative (mean = -0.117, Wilcoxon signed rank test, V = 94.5, n = 36, p < 0.001).    Wilcoxon signed-rank tests).

Literature search -Conclusions
To confirm our hypothesis, that a preponderance of group-based methods makes it appear as if social networks typically possess assortativity, we found that group-based social networks were more assortative than direct social networks and non-social networks. However, direct social networks were still more assortative than non-social networks, which showed disassortativity on average. This therefore indicates that social networks only tend to be assortative if they are constructed with group-based methods, and non-social networks are typically disassortative. The processes that create dissortative networks with is a large and active research area (e.g. Yang 2014; Mussmann et al. 2014) and so we will not investigate this result further here. We also note that many of the direct social networks in our table were based on interactions online. This is perhaps problematic if interactions online are fundamentally different to human interactions in the real world. Our choice of networks was simply based on what was available. Therefore, to confirm whether the assortativities of offline and online human social networks made using the same method (i.e. with direct interactions) are different, more offline human networks using direct interactions need to be constructed and published.
The bias towards assortativity when using group-based methods has been used has previously been noted (Newman 2003;Croft et al 2008), but this appears to have escaped the notice of much of the research community, who continue to state that assortativity is a characteristic quality of a social network (e.g.  2009)). This may be because no clear rationale for this has been presented, nor solutions offered to combat it. We now go on to present why we think this assortativity was being discovered with group-based methods, if they are erroneous, and how to avoid incorrect estimates of assortativity in the future.

Methodological pitfalls and false assortativity
In this section we will discuss the issues that exist when using assortativity to describe the structure of social networks. We explore how methodologies used to sample networks could influence their assortativities, and whether changing how assortativity is calculated may also be important. Understanding the consequences of the method used to sample networks on properties such as assortativity could have important implications for our understanding of networks in general, as well as contributing to the further development of relevant analytical techniques.
2.1 Group-based networks and assortativity In situations where it is not possible or feasible to directly observe social interactions, social networks are constructed using group-based methods. This approach has been applied both in constructing collaboration networks in humans e.g. jazz musicians, scientific co-authorship networks and film actors (Watts and Strogatz 1998;Newman 2001;Gleiser and Danon 2003) and also in networks based on co-occurrence in a social group in animals e.g. song birds, dolphins and sharks (Lusseau 2003;Mourier et al 2012;Aplin et al 2012). Networks built using this method assume that every member of the group is associating with every other member of the group at each sampling census. This seems perfectly reasonable, hence this method for constructing social networks has been used by many studies (Newman 2004;Croft et al 2008). The assumption that meaningful social networks can be constructed based on this type of co-occurrence data has been termed "Gambit of the group" in the animal behaviour literature (Whitehead and Dufault 1999), which hints at the risk involved. This broad usage of group-based methods makes it very important that we fully understand the implications of using this method on the network and individual-level metrics calculated.
One study has previously investigated the influence of variation in sampling effort of group-based networks on various network-level metrics, including assortativity ( be zero (Newman 2002), was always positive if an insufficient number of censuses were completed before network construction (Franks et al. 2009; our emphasis). Crucially, any network not sampled intensively enough could show assortativity, either due to too few censuses or by not sampling enough of the population. Therefore, the use of group-based methods can produce a sampling bias, and requires further statistical analysis to determine the importance of the results obtained.

Modelling group-based sampling
We wanted to extend the findings of Franks et al. (2009) to demonstrate how assortativity changes as individuals are recorded in increasing numbers of groups. The aim was to show that in a system where freely-moving individuals form social groups randomly (as opposed to a system based on random social networks), assortativity would decline as the effect of two individuals being seen in the same group diminished. We used a simulation-based approach where individuals associated randomly to construct networks using a group-based approach in a simple population of 100. Individuals were allocated randomly between 20 possible groups during each "census" using a symmetric Dirichlet distribution (with a uniform shape parameter) to define the size of each group. This enabled us to generate variation around a fixed expected group size, whilst maintaining a fairly consistent number of groups in each census (occasionally the size of a group could be zero). Any remaining individuals were allocated to a random group, meaning that all individuals were sampled in each census.
Association data was collected over 20 censuses, with this process repeated for 10 different repeats of the simulation. After each census cumulative association data was recorded and the networks were dichotomised to create binary networks. In these networks individuals are either connected if they were observed in the same group at least once, or not connected if never observed together. This is necessary as assortativity is an unweighted measure i.e. only the number of different associations an individual had are counted, not the frequency with which it associated with them. We discuss issues related to this method later. We then calculated assortativity for each set of cumulative association data. Simulations were carried out and degree correlations calculated in R 3.0.1 (http://www.R-project.org).
We plot the results of these simulations in Fig. 2. From this plot it is clear that networks possess assortativity at a low number of censuses, and become gradually more neutrally assortative as the number of censuses increases (Fig. 2). This pattern emerges because of how group-based approaches are used in network construction. At low numbers of censuses, individuals that have been found in the same groups will both be connected and have similar degree, thus giving the network assortativity (Newman 2003;Newman and Park 2003). As the number of censuses increases however, this connection will gradually break down. Therefore, our simple simulation model shows that the assortativity found using group-based sampling approaches is highly dependent on the number of censuses completed.
Indeed a low number of censuses can lead to many network measures being distorted (Franks et al 2009;Perreault 2010). This is therefore an important consideration when deciding the sampling regimes used when constructing social networks using this method. The significance of the assortativity can only be established by comparison to appropriate null networks that truly highlight what aspects of the real network are interesting. This is achieved by randomly resampling observations using the correct group-size distribution (Bejder et al 1998), a method that will be outlined in more detail in section 3.2.

Figure 2
The relationship between sampling effort (in terms of number of association events observed) and assortativity as a result of random social interactions in a simulated population.
For each census the black point represents the median value, the dark grey box the interquartile range and the light grey box the range from 10 runs of the simulation.

Filtering networks
Filtering networks involves taking a weighted (or valued) network, which contains information about how strong associations are as well as whether they are present or absent, and removing edges below a certain weight i.e. infrequent or unimportant associations. This results in a binary network, with only edges above a certain weight present. The effect of dichotomizing networks to transform them from weighted to binary has been studied for a number of network-level metrics (Franks et al 2009). The effect of this on assortativity is striking. While there was a limited effect of filtering at a low threshold (and therefore removing few edges), filtering at a high threshold (removing relatively many edges) had a considerable impact on the assortativity even when a high number of censuses on the simulated networks was completed. While for unfiltered, weighted random networks the assortativity reduced to zero as expected when a sufficient number of censuses were completed, this was not the case for networks filtered with a high threshold. These networks typically remained assortative at all levels of sampling (Fig. 3 of Franks et al. 2009), despite being based on networks that were originally random and would be expected to neutrally assortative. For example, removing all edges with a weight of less than 0.5 (out of 1) meant that the assortativity reached 0.4 after 10 censuses, compared to 0.1 after 10 censuses if the threshold weight was set at 0.2. Many studies that have used group-based approaches to construct social networks have filtered the social networks produced before analysis (Croft et al 2008), and it seems likely that the use of this approach may have played an additional role in inflating the degree correlation calculated for social networks.

Solutions
There are a number of solutions available to reduce the occurrence of erroneous assortativity that address each of the major issues outlined above. These methods have been developed in different parts of the social network literature, and if used collectively can greatly improve our understanding of the true variation in assortativity in all kinds of social networks.
3.1 Increased sampling When using group-based approaches, increasing the number of censuses above a threshold should produce a more accurate measure of the network's true assortativity (Franks et al 2009). Thus a suitable minimum number of censuses completed is required whenever using group-based approaches. If 80% of the population of interest can be sampled, then 10 censuses should be sufficient information on social interactions to accurately calculate network metrics. If only 40% of the population can be sampled, then 15-20 censuses may be required (Franks et al 2009). Collaboration networks, a key example of social networks that possess assortativity, typically do not have sampling periods. Instead all papers published over an agreed time period are looked at. To allow better comparison with networks and techniques that do have sampling periods, collaborations could be examined over particular, regular time periods, then each time period used as an "observation" from which to generate associations and so networks. However, if altering the sampling regime is not feasible, ensuring erroneous conclusions are not reached requires more rigorous statistical analysis.
3.2 Use of null models Socio-biologists typically use resampling procedures to account for group-based sampling (and other, random) effects in their animal social networks (Bejder et al 1998;Whitehead and Dufault 1999). Resampling the observations used to construct the observed network using the original group-size data, and potentially additional biologically meaningful constraints, provides a large number of null networks against which to compare the observed network and test relevant hypotheses (Bejder et al 1998). This method has been continually revised to take into account the non-independence of group sightings (Sundaresan et al 2009) and the risk of producing biased group compositions (Krause et al 2009). This approach has been used in a number of studies (e.g. Wey et al. 2013;Aplin et al. 2013) and means that assortativity would only be considered interesting if the null model does not also display assortativity. We found one example of this approach being used in human social networks, in a study performed on a network of a board of directors (Newman and Park 2003). However, no statistical tests were used for this comparison, and the method does not seem to have been universally adopted by the wider social network community. The importance of adopting this method is further highlighted the results of Franks et al. (2009) which show that assortativity can be expected in networks where individuals randomly interact.
Random permutations are used by other network analysis techniques such as quadratic assignment procedure (QAP) to overcome problems with structural autocorrelation (Krackhardt 1988), but the Bejder et al. (1998) method and extensions go further by randomly permuting the raw data, rather than the network matrix, to account for the distribution of group sizes and any other biases in sampling. Appropriate null models like these are the most effective way to make statistical inferences about networks constructed using group-based approaches, where controlling for effects such as group-size is of considerable importance. There have been some recent examples of appropriate null models being used to test various hypotheses in the wider social networks literature (e.g. (Hanhijarvi et al 2009;La Fond and Neville 2010)) and we add to their calls for this to be more widely embraced.

Analysing weighted networks
Weak interactions are potentially important (e.g. Granovetter 1973;Farine 2014), and removing them can further increase error in the degree correlations calculated (Kossinets 2006;Franks et al 2009). If a population is observed for an extended period of time, and a large number of censuses are performed, the probability that two individuals are never observed to associate approaches zero. In a binary network, this would result in all individuals having the same, maximum, degree (which would be the population size -1), and therefore the assortativity would increase before becoming undefined. Therefore, it is preferable to use weighted, unfiltered networks. For example, if collaboration networks are based on some measure of the information contained in an email (Garton et al 2006), or the relative roles of those involved (Rowe et al 2007), you could then produce weighted networks that would be more informative to study and be less likely to be falsely assortative. The analysis of weighted networks is becoming increasingly manageable (e.g. Opsahl et al 2010), reducing the need to filter networks and analyse them as binary data. As such, this approach should only be used when absolutely required (Noldus and Van Mieghem 2015), preferably with filtering using low thresholds (few edges removed). The filtering of some networks in unavoidable, for example in brain imaging data which is likely to be very noisy (Iturria-Medina et al 2007), but we still recommend the lowest amount of filtering possible while accounting for edge uncertainty. Additionally, it would also be highly beneficial to continue to develop social network analysis in weighted networks to further reduce the requirement for binary network data, for example by continuing the development of exponential random graph models that can be used in weighted networks (Krivitsky 2012;Krivitsky 2015). networks constructed using direct interactions on average possess neutral assortativity (see above), indicating they are not prone to the same problems with false assortativities that networks built using group-based approaches are.

Modern technology
Missing associations, and therefore edges in the network, is a problem largely unique to social networks. This can be prevented in human social networks, especially collaboration networks, by examining information-rich communications data. Information rich interpersonal communications such as texts, tweets and emails can be used to construct networks on a very large scale with a high degree of accuracy (Choudhury and Mason 2010).
In animal social networks a solution is to use reality mining (Krause et al 2013), the concept of which has been borrowed from the sociology social network literature (Eagle and Pentland 2005). In these studies, modern technologies such as GPS trackers or proximity loggers are These methods substantially reduce the number of missing edges in the network.
Additionally, even when they don't allow us to move towards constructing networks using direct interactions, they can make it much more feasible to complete a large number of censuses if a group-based approach is still required. Furthermore, with careful thought technologies such as these could be used to complement the study of human social networks, potentially combined with data gathered using other methods. For instance, in the work place, are those an employee physically associates with e.g. at the water cooler or on a cigarette break, the same as those they communicate with electronically? What about different types of electronic communication, e.g. Facebook messages compared to work emails? Such application of modern technologies will only enhance the ability of scientists to measure social interactions in a wide range of networks.
3.6 Alternatives to the Newman degree correlation measure Recent research has suggested that calculating the correlation coefficient proposed by Newman (2002) may not be appropriate for large networks, as it tends to mis-estimate the assortativity, especially in disassortative networks (Litvak and van der Hofstad 2013). Litvak and van der Hofstad (2013) suggested that using a method that ranks the degree of nodes, rather than using their absolute values (like a Spearman's rank correlation coefficient), may produce more valid results. Using this measure, the assortativity was consistent across a wide range of network sizes and typically consistently different from those calculated using Newman's assortativity coefficient (Fig. 1 of Litvak and van der Hofstad 2013). It may be that changing the way that assortativity is calculated could reduce the differences between group-based networks, other social networks, and different network types. This can be combined with the Bejder et al. (1998) method to control for both group size variation and biases associated with whole network size.
Alternatively, several authors have proposed individual-orientated metrics to quantify the tendency for well-connected individuals to connect to other individuals. The "Rich-clubs" of Zhou and Mondragon (Zhou and Mondragón 2004;Colizza et al 2006), and the number of "differences" in node degree between neighbours of Thedchanamoorthy et al. (2014) may both be robust to these pitfalls. Understanding the consequences of using different statistical approaches for calculating degree correlations should be the subject of further modelling work, in order to determine whether this can have an important influence to the properties with which a network is prescribed.

Conclusions
Assortativity has often been suggested to be a property of social networks that makes them different from non-social networks. We show that it is likely that this phenomenon may be driven by the data collection methods, resulting in previous studies overestimating the extent to which social networks possess assortativity. With alterations to methods or analyses however, this problem with erroneous assortativities being described can be mitigated.
Through the use of methods initially developed in biology and sociology, we suggest that careful application of sampling and statistics can afford us increased confidence that finding assortativity in social networks is genuine, and not an artefact of the methodology used to construct them. It may well be that social networks are unique amongst types of network; there are certainly good reasons why they could have developed to be more assortative than non-social networks e.g. increased robustness (Jing et al 2007;Hasegawa et al 2012). However, given that information transfer and the ability to act in synchrony are greater in networks that show disassortativity (Di Bernado et al 2007;Gallos et al 2008), it would perhaps be surprising if this property is as universal as has previously been described (Newman 2002;Newman and Park 2003).
Many of the methods discussed have been developed in one part of the social networks literature, and would benefit researchers in other fields greatly. Furthermore, following the integration of methods used in these different fields, it is likely that new insights will become available and further progress made in our ability to make inferences about assortativity in networks more generally.