{'fraction_non_alphanumeric': 0.028867593339342475, 'fraction_numerical': 0.00555054793870677, 'mean_word_length': 4.945423776618249, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Limiting Tags Fosters Efficiency

Tagging facilitates information retrieval in social media and other online communities by allowing users to organize and describe online content. Researchers found that the efficiency of tagging systems steadily decreases over time, because tags become less precise in identifying specific documents, i.e., they lose their descriptiveness. However, previous works did not answer how or even whether community managers can improve the efficiency of tags. In this work, we use information-theoretic measures to track the descriptive and retrieval efficiency of tags on Stack Overflow, a question-answering system that strictly limits the number of tags users can specify per question. We observe that tagging efficiency stabilizes over time, while tag content and descriptiveness both increase. To explain this observation, we hypothesize that limiting the number of tags fosters novelty and diversity in tag usage, two properties which are both beneficial for tagging efficiency. To provide qualitative evidence supporting our hypothesis, we present a statistical model of tagging that demonstrates how novelty and diversity lead to greater tag efficiency in the long run. Our work offers insights into policies to improve information organization and retrieval in online communities.

over time on the social bookmarking site del.icio.us, an early example of a social tagging system that allowed any user to add free-form labels to web pages. Their finding implies that tags became less useful over time for the purpose of information retrieval on del.icio.us. However, the study did not specify whether this was a generic property of social tagging systems, or how system managers could counter this trend to improve tag efficiency.
This work. In this paper, we observe improved information organization and retrieval in a tagging system that imposes a limit on the number of tags used to annotate documents. Limiting the number of tags and at same time simplifying the creation of composite tags, may nudge users to create fewer but more specific tags, which improves tag efficiency [26,46].
We study a large-scale dataset of tagged questions from Stack Overflow, 1 a popular online community dedicated to answering questions (Q&A) on programming-related topics. On Stack Overflow, users can annotate each question they ask with up to five tags. We use information-theoretic measures to quantify the capacity of tags to organize and retrieve information. Specifically, we compute tag entropy, the conditional entropy of questions given tags, and the mutual information of questions and tags as measures for information content, information retrieval, and tag descriptiveness, respectively. These three measures assess tag efficiency. Finally, we explore the mechanism leading to observed trends by simulating a statistical model of tagging. The model, which is inspired by models of Web growth [25] and evolution of biological and technological systems [40], captures the growth of tags through novelty, diversity, and reinforcement.
Findings and contributions. While the amount and complexity of information on Stack Overflow grows over time, increasing the demands for organizing that information, our findings indicate that capacity of tags to retrieve information stabilizes after an initial period of decline. Further, we also observe a steady increase in tag descriptiveness. This is in contrast to an earlier study of social tagging which showed a decreasing capacity of tags to describe and identify documents [9]. We hypothesize that improvement in tag efficiency is linked to strict limits on the number of tags imposed by Stack Overflow. We show that questions with fewer than five tags exhibit even higher tag efficiency over time. The difference in tag efficiency is most pronounced in the comparison between questions with five tags and those with only one tag, and this difference becomes smaller when the number of tags increases. Limiting tags therefore appears to encourage users to (i) create new tags by concatenating existing ones, based on the implicit tag hierarchy on Stack Overflow and to (ii) diversify their tag selection by using more specific rather than popular tags. The additional effort in annotating content contributes towards higher tag efficiency while reducing the number of tags to describe a question. Finally, with our statistical model we (i) qualitatively recreate a range of behaviors observed in real systems, and (ii) illustrate how novelty and diversity in tag usage improve tagging efficiency. This qualitative evidence thus supports our empirically-grounded information retrieval hypotheses.
Our results are of practical relevance for managers aiming to optimize their tagging systems. Specifically, our findings suggest that limiting the number of tags people can use to annotate content may improve the efficiency of tags for information retrieval. However, system managers may wish to consider the trade-offs between simplifying information organization (e.g., by recommending popular tags) and encouraging elaborate content annotation (e.g., through manual creation of descriptive tags), as we link gains in information organization to decreasing tag efficiency.

RELATED WORK
Social Tagging. Tagging, a way to categorize content in online platforms, has witnessed broad usage in content aggregation websites, such as del.icio.us [13], and social networks, such as Twitter or Instagram [44], via the hashtag mechanism. Previous work has studied the function of tags for information seeking purposes [8,19], as well as for social advocacy, such as #MeToo [33] and #BlackLivesMatter [36]. Social tagging systems vary on who can tag (document owner or anyone), what tags are used (freeform or dictionary terms), and how many tags can be added.
Information organization. Furnas et al. [13] see the social tagging process as a collective effort of organizing information without predefined term vocabulary. Several studies analyzed what motivates users to create tags in order to design better user interfaces [30], or model information retrieval. For example, Körner et al. [26] characterized users according to their tagging motivations and concluded with the observation that the semantics, and hence, information retrieval efficiency is best achieved through verbose tags. Collaborative systems for knowledge creation (e.g., Wikipedia or Stack Overflow) benefit from shared information organization tools such as social tagging as they support establishment of common domain understanding [41]. While automatic extraction of keywords using modern natural language processing and deep learning methods are frequently applied to support information organization efforts (e.g., by presenting a set of keywords for users to select from) [2,3,34], establishment of a shared vocabulary among users supports individual and collaborative sensemaking [7,29,31]. In particular, Mamykina et al. showed the enhanced ability of individual users of social tagging systems to recollect relevant information, and stressed the importance of controlled vocabularies and expert moderation [31], whereas Bagheri and Ensan [3] showed the abilities of tags in organizing content in collaborative software projects. In another line of work, Golder and Huberman [15] found that tags exhibit a strong popularity bias. More recently, Trattner et al. [39] found a recency bias, in which tag usage strongly depended on time.
In our work, we ask whether and how user interface design choices, such as limiting the number of user-assigned tags, support information organization in the long term.
Information retrieval. There are two main streams of research on improving information retrieval with tags. Firstly, tag recommender systems aim to predict tags for users on a document of interest. Building on seminal work of Heymann et al. [20], tag recommendation systems have evolved to incorporate models and insights from natural language processing [28] or cognitive science [27]. Further, tags help in profiling users in recommender systems [45]. For instance, Klasnja-Milicevic [24] enhanced a standard tensor factorization method to improve recommendations in an educational setting, Enrich et al. [12] applied cross-domain tags to remedy cold-start problem in recommender systems, and Zuo et al. [47] applied auto-encoders and deep learning to extract more precise user profiles for recommendations. Secondly, considering tag navigation, previous work proposed a tag exploration system to help users grasp certain topics [23].
Helic et al. [18], however, challenged the underlying hypothesis that tagging systems also support efficient navigation.
In a later work, Helic and Strohmaier proposed an algorithm to improve navigational efficiency of tag hierarchies [17].
We position our work in the second stream of research, as we inspect the efficiency of design choices on tag-based information retrieval. In particular, a study of del.icio.us, which allowed users to annotate Web pages with tags, demonstrated that the efficiency of tags for information retrieval declined, as indicated by increasing entropy (i.e., uncertainty) of a document given a set of tags used to describe it [9]. We extend this work with a longitudinal study of tag efficiency in a system that limits the number of tags users can apply to a document, to examine if this improves tag efficiency over time. Our research thus offers evidence-based design recommendations for tagging system managers.
Orthogonal research [14] found a link between constraints and creative production on social media, therefore it may not be clear a priori whether limiting the number of tags supports information retrieval. Does a tag limit stifle users' creativity or do users adhere to the limit and employ specific and descriptive tags? We find evidence for the latter.
Improving tagging systems. Researches have proposed a variety of ways to improve social tagging systems. Cantador et al. [5] propose mapping of tags to a set of predefined ontology classes from external sources such as WordNet or Wikipedia with the goal of reducing noise in tag collections. Similarly, Zubiaga et al. [46] suggest that clustering user tagging profiles supports categorization of tagged resources. Further, Meo et al. [10] compare tagging and social behavior of the users with the semantics emerging from the tags that they use. Their findings support applications such as recommender systems, user profile merging and estimation of user similarity across platforms. On the other hand, Xie et al. [42] extract communities of similar users from their tagging profiles to enrich future tag suggestions. Jabeen et al. [22] provide a comprehensive review of approaches to extracting semantics from social tagging datasets.
Our work extends this line of research by analyzing how user interface design influences and potentially improve the information retrieval efficiency of tagging systems.
Growth models for information systems. Finally, stochastic models for information systems are critical to better understand the patterns seen in data. Kleinberg et al. [25] designed a model of the Web growth in which the new Web pages are created iteratively with pages stochastically deciding whether to copy the links from an already existing page or to create new links uniformly at random. This preferential attachment mechanism [4] results in a rich-getricher dynamics and a skewed degree distribution. Tria et al. [40] developed a model that extends a basic preferential attachment mechanism by also modeling the time correlation in novelties, such as new Wikipedia webpages.
We explore a model of the growth of resources and tags, which reproduces a broad range of behaviors observed in real-world social tagging systems.

MATERIALS AND METHODS
Data. We study Stack Overflow, a large Q&A community dedicated to answering questions related to programming. We obtained data on Stack Overflow from August 2008 through May 2019 2 , comprising 17.7 million questions. Some descriptive statistics of the growth of questions and tags in the data are shown in Figure 1. We focus on questions and tags users employ to categorize questions. Users can assign up to five unique tags per question by picking terms from a pre-defined vocabulary. Only experienced users with enough reputation on Stack Overflow 3 may propose new tags.
To analyze the tag efficiency in Stack Overflow we study the evolution of (a) information content (quantifying how much information is contained in the system), (b) information retrieval (measuring the difficulty of finding a question via tags), and (c) tag descriptiveness (quantifying the utility of tags in describing questions) over time. We use information-theoretic measures to compute these quantities, following the work by Chi and Mytkowicz [9].
Information content. In addition to measuring a tagging system's growth as counts over time, we also consider how the distributions of tags and questions change by computing their entropy. Mathematically, the entropy is given by: Entropy measures the information content of a random variable by computing the number of bits needed to encode the uncertainty in that random variable. Thus, this metric captures uncertainty in choosing a question or tag from the empirical distribution of questions or tags. For example, do users prefer to use popular or niche tags? The former case corresponds to a low level of entropy (or equivalently a low information content or a low level of uncertainty when selecting a tag at random) as many users simply reuse a popular tag over and over again. In contrast, the infrequently-used niche tags are described by a high level of entropy.
Question and tag entropy that grows with time are therefore desirable properties of a tagging system as they signal that the amount and complexity of information in the system is increasing. On the other hand, constant or decaying question and tag entropy indicate a saturated system in which no new information is added. This can either happen if no new questions or tags are created (constant entropy) or if the distributions become heavily skewed with time (constant or decaying entropy) due to, for example, strong popularity bias.
In a question-answering system, the probability of randomly choosing a question is inversely proportional to the number of questions |Q|, which usually appear only once [1]. Therefore, ( ) corresponds to the entropy of a uniform distribution, which is log |Q|. Tags, however, do not occur only once, and hence ( ) is the relative frequency of a tag .

Information retrieval.
We use the conditional entropy of questions given tags as a proxy for tag retrieval efficiency. Intuitively, this metric captures the uncertainty in identifying a question after a specific tag has been selected by a user. The lower this uncertainty the more efficiently tags can identify questions. As such, the conditional entropy measures how difficult is to retrieve a question given a set of tags and preferably this conditional entropy decays with time.
Formally, we define the conditional entropy of questions ∈ Q given tags ∈ T as follows: where ( ) is as defined above and ( | ) is the conditional probability of a question given a tag , i.e., the probability of picking a single question out of all questions employing a certain tag, or, mathematically, one divided by the number of questions with a given tag . The calculation of ( | ) only holds for Stack Overflow where distinct tags are added to each question by a single creator of that question. In systems where multiple users can tag resources with repeating tags ( | ) corresponds to the conditional relative frequency of a question given the tag [9].
The conditional entropy is minimized at zero-this happens when tags uniquely identify questions (one-to-one mapping to questions), i.e., when the information of tags suffices to retrieve questions with no uncertainty left.
Tag descriptiveness. Our third information-theoretic measure captures the descriptiveness of tags, i.e., how much tags tell us about questions. This metric is the mutual information between questions and tags, defined as: This measure is akin to a distance between the question entropy and the conditional entropy of questions given tags, i.e., it quantifies the reduction of uncertainty in questions due to the knowledge of tags. If it is large, then knowledge of Increasing entropy is better-the system has not saturated yet and every new question and assigned tags contribute to information content. We measure retrieval difficulty as the conditional entropy of questions given tags. Lower entropy implies easier question retrieval via tags. We observe in (c) that the retrieval difficulty stabilizes after an initial phase of rapid growth. The tag descriptiveness is captured with the mutual information between tags and questions. Higher values imply better descriptiveness. We observe in (d) that mutual information grows over time. All metrics indicate increasing tag efficiency on Stack Overflow, in contrast to del.icio.us [9].
tags greatly reduces uncertainty in questions, and thus we say tags are very descriptive of questions. Otherwise, tags do not convey much information about questions as they are largely independent of them.
In an efficient tagging system mutual information grows over time, meaning that with time tags tell us more about questions. Note that this seamlessly fits with the other two desirable properties of a tagging system: (a) growing

TAG EFFICIENCY ON STACK OVERFLOW
Information content. We observed in Figure 1 that there is a pronounced growth in questions and tags over time, with a linearly increasing growth in recent years. The tag and question entropy gives insight in whether this steady growth of tags and questions also translates into growth of information content.
We find that the entropy of questions and of tags increase throughout the observation period, as shown in Figure 2a This behavior is expected for questions as the entropy is |Q|. Growing ( ) indicates that users contribute additional information when they assign tags to their questions, either by creating novel tags or by assigning more diverse and unpopular tags, which both lead to a less skewed tag distribution over time. Increasing question and tag entropy are desirable in a social tagging system, as growing information content is evidence of a healthy user community making significant contributions to the system. Information retrieval. In Figure 2c, we show the conditional entropy of questions given tags, Eq. 2, as a measure of the difficulty to retrieve specific content using tags. The conditional entropy increases steadily, therefore content becomes increasingly difficult to retrieve. However, this trend subsides after 2016, signaling that the difficulty of retrieval begins to stabilize, despite increasing entropy. Notably, this result contrasts with measurements performed in previous work, which observed a continuous decrease in tag retrieval efficiency on a social bookmarking site called del.icio.us [9].
Tag descriptiveness. Finally, we study the mutual information of questions and tags, Eq. 3, i.e., how well tags describe questions. In Figure 2d, we observe that ( ; ) steadily increases over time: thus tags tell us increasingly more about We find empirical evidence for these theoretical considerations in data. Figures 3a-b show the monthly rate of new tags on Stack Overflow and the Gini coefficient as a measure of inequality of the tag distribution. We observe a decaying but a substantial tag novelty rate even eleven years after the system inception (Figure 3a), and an increasingly diverse usage of tags as we observe a downwards trend in inequality in recent years (Figure 3b). The starting value and growth of the Gini coefficient before 2014 indicates the presence of a strong tag popularity bias in the initial phase of Stack Overflow, which continuously subsides afterwards.
Tags specificity and composite tags. Growing levels of tag descriptiveness in the long-term indicate that users become more specific and precise in their tag usage with time. Hence, we now investigate tag specificity as another mechanism that may play a key role in an increasing tag efficiency. Stack Overflow supports a kind of tag hierarchy, where a more general tag may form a superset for more specific tags. For example, there is a tag termed "google-chrome" for general questions related to that Web browser. Further, there are also other, more specific tags on that topic, such as "google-chrome-devtools" or "google-chrome-extension". We term such tags composite tags, and we operationalize their usage as an indicator for the specificity of tags being applied to questions on Stack Overflow. We measure the specificity of the tags (by computing the mean tag length in characters) and the usage of composite tags (by computing the mean monthly relative frequency of tags with at least one dash "-" 4 ) over time in Figure 3c and Figure 3d, respectively.
We find that tags become longer and use of composite tags increases, which may indicate growing tag specificity.
This observation may indicate stable tag retrieval efficiency in the long-term: the use of specific tags may help to uniquely identify questions especially given the tag limit, and thereby keep the uncertainty in question retrieval via tags stable-a result which was also found in previous work [31].

DISCUSSION
Single vs. multiple tags. Trying to replicate the finding that information content, tag retrieval efficiency, and descriptiveness decrease over time in tagging systems [9], we instead find that on Stack Overflow, a system with comparable growth dynamics, information content and tag descriptiveness steadily grow, whereas retrieval efficiency stabilizes.
While the system analyzed in previous work [9], a social bookmarking system called del.icio.us, remarkably differs in purpose and scope from the Q&A system we study, we observe comparable activity dynamics but diverging developments in the organization of content. The first substantial difference between the two systems is related to the multiple tagging of resources (Web pages on del.icio.us and questions on Stack Overflow). On del.icio.us multiple users can tag the same Web page, which potentially increases the skew in the distribution of resources (resulting in a less than maximal resources entropy). On contrary, on Stack Overflow every user creates a new question and assigns the tags at the time of creation. Hence, there is no reinforcement of questions as no other user can assign additional tags to an already existing question. This leads to a uniform question distribution and the maximal question entropy.
Effects of tag limit. Apart from the dynamics of resource creation and tagging, another reason for the discrepancy between different social tagging systems may lie in the per-document tag limit, which Stack Overflow imposes but del.icio.us does not. We believe that limiting the number of tags encourages users to create more specific tags (e.g., composite tags on Stack Overflow), which in turn weakens popularity bias as users assign more novel and diverse tags.
To understand the impact of this design choice, we inspect the tag retrieval efficiency (cf. Eq. 2) for different numbers of tags, as a proxy for the effects of imposing stricter limits on the number of tags per question. We plot the results of this exercise in Figure 4a. We observe the lowest conditional entropy, i.e., highest tag efficiency for the strictest limit of one tag per question. As the upper tag limit increases, the tag efficiency also gradually decreases to the level of questions with up to five tags. We observe that this gradual decrease is nonlinear: the difference in tag efficiency between the one and two tag limit is larger than between two and three tags, and this difference becomes successively smaller as the tag limit increases further. These observations suggest setting a tag limit nudges users to employ specific, descriptive tags [26], which in turn help organize and curate information.
Effects of composite tags. In addition to the tag limit, simplicity of concatenating several tags with increasing specificity potentially accelerates the usage of composite tags (regardless of the tag limit) and contributes to tag efficiency on Stack Overflow. We analyze this effect by measuring, again via Eqs. 1, 2, and 3, the tag content, tag retrieval efficiency, and tag descriptiveness of questions with at least one composite tag vs. those without. We remark, with Figures 4b-d, that those with composite tags feature (a) higher tag content, also with a higher growth rate than the content of simple non-composite tags, (b) higher retrieval efficiency, and (c) higher tag descriptiveness again with a more pronounced growth as compared to simple tags. This observation validates the usage of composite tags as a solution which allows for specific tagging, and thus improved organization of information items [31]. Nevertheless, on Stack Overflow the simple tags are also remarkably efficient as we observe increasing tag content, stabilizing retrieval efficiency and increasing tag descriptiveness of simple tags only (cf. blue lines in Figures 4b-d). This result is in stark contrast to del.icio.us [9] where all three metrics developed in the opposite directions. Hence, when controlling for the usage of composite tags, we still see a residual effect of growing tag efficiency on Stack Overflow, which we attribute to imposed tag limits and resulting increased levels of novelty and diversity even in the usage of simple tags.
A total disentanglement of the effects of the tag limit and other user interface choices is not possible: we can not decisively answer the question whether the tag limit or simplicity of creation leads to an increased use of composite tags and hence to a growing tag efficiency. We only observe that the combination of the tag limit and easy creation of composite tags results in novelty and diversity in tag usage and induces a strong positive effect on tag efficiency.
Reducing popularity bias. Finally, we also reason about another feature of Stack Overflow's tagging mechanisms that may have positively impacted information organization: as users type the name of a tag for a question, an auto-complete system suggests pre-existing tags. Crucially, though Stack Overflow currently sorts suggested tags by number of questions, this auto-complete system is not a tag recommendation system. As such, users are expected to provide descriptive tags on their own, and not rely on tag recommendations which may induce a stronger popularity bias [9].

Tag Growth Model
To gain a further insight into our results, we present a model of social tagging that qualitatively reproduces the range of contrasting behaviors observed on Stack Overflow and del.icio.us [9]. Similarly to´the models of Web growth [25] and the evolution of social or technological systems [40], we base our model on an intuition that tagging decisions are correlated over time. In particular, our model is based on the mechanism of reinforcement (users copy decision of other users), novelty (users introduce new tags), and diversity (users diversify over popular as well as niche tags). Note that the goal of our model is to qualitatively study the dynamics of tagging behavior given a particular configuration of the model parameters and not to assess how user interface choices may translate to a specific set of model parameters.
Theoretically, our model closely resembles Polya's urn model. The model considers an urn containing some number of balls of different colors. At each step a ball is drawn randomly from the urn and then placed back together with a new ball of the same color, hence reinforcing the selected color. The reinforcement mechanism of the Polya's urn model results in a rich-get-richer dynamics and leads to power law distributions of the balls in the urn [43]. Various variants of this model have been used extensively to model the power law phenomena in physics and computer science [35,38,40].
Reinforcement-novelty model. In the reinforcement-novelty model we start with two urns, one for the resources and one for the tags. At each discrete time step a new user tags a resource. First, the user decides with probability whether she wants to reinforce a decision previously made by another user or whether she wants to introduce a novel tag with probability 1 − . In the case of reinforcement, the user selects a ball from the resource urn, then puts this ball back together with another ball of the same kind into the resource urn. This reinforcement mechanism correlates the decision of the current user to the past decisions of other users, thus introducing a popularity bias into user decisions.
On the other hand, in the case of novelty the user simply creates a new resource and adds it to the urn. After the user selects the resource, she then decides how many tags to assign to that resource. We model this using random variable representing a limit on tags. We use a binomial random variable ( , ) with corresponding to the tag limit and such that reflects the empirical mean of tags per resource from a tagging system. For each individual tag, the user adopts the same mechanism as in the case of resources. Thus, she first decides with probability whether to copy an already existing tag or to assign a novel tag with probability 1 − . In the case of reinforcement, the user first draws a tag from the tag urn, then puts this tag and one more copy of it back into the urn. In the case of a novel tag, the user creates a new tag and puts it into the tag urn. After the user assigns the last tag, we repeat this process for a given number of users . Finally, we inspect the information-theoretic measures to quantify tag efficiency.
Reinforcement-novelty-diversity model. To account for diversity in selection of tags during the reinforcement phase, we adapt the reinforcement-novelty model by changing the ball selection process from the tag urn. In particular, instead of using the classical Polya's urn mechanics, which selects tags proportional to their relative frequencies, we first smooth the frequencies. For smoothing we chose the softmax function: where is the relative frequency of tag , |T | is the current number of tags, and is the diversity factor, which controls the strength of smoothing. When → ∞ (strong diversity), then 1/ → 0 leading to (f) → 1/|T |, i.e., a high level of diversity results in a uniform tag distribution. On the other hand, when → 0 (weak diversity), then 1/ → ∞ resulting in the softmax function with probability 1 for the maximal relative frequency with all other probabilities being 0 (i.e, the extreme case of the popularity bias). Thus, we can also interpret the quantity 1/ as the popularity factor. Note that the smoothing towards uniform distribution always happens when the differences between individual relative frequencies are smaller than the diversity factor . As the differences between relative frequencies are always from [0, 1] setting the diversity to e.g., 1 achieves smoothing, whereas setting diversity to a value close to 0 induces a strong popularity bias.
Comparison with existing models. We briefly compare our model with two most prominent models of the growth of information systems: (a) the model of correlated novelties by Tria et al. [40], and (b) the copying model by Kleinberg [25].
The first difference is that in our model we have two sequences, one for the resources and one for the tags. Two other models have only one sequence each, e.g., the Web pages in Klenberg's model, or Wikipedia pages in Tria's model.
Another important difference between our reinforcement-novelty-diversity variant and the other two models is the smoothing mechanism, which we use to reduce the popularity bias. Further, in contrast to the model of correlated novelties, there are no explicit correlations in our model, neither within the urns nor between the urns, although in actuality there will be correlations when selecting tags for a given resource, e.g. due to semantics. However, there are = 1 and = 1) with a strong popularity bias, we observe a deteriorating tag performance across all measures. (b) In a system with diverse selection of tags, tag efficiency improves but still decays over time. (c) In a reinforcement-novelty system with = 0 (each user creates a new question) and = 0.98 (most recent observed rate of tag reinforcement in Stack Overflow) but without diversity, tag entropy and mutual information develop more positively, but the tag retrieval efficiency decays quickly. (d) Finally, in a system with reinforcement, novelty, and diversity (e.g. Stack Overflow), we observe increasing tag efficiency with a stabilizing conditional entropy.
implicit correlations between popular resources and tags that are frequently selected together due to the reinforcement.
Finally, in the correlated novelties model the reinforcement and novelty rates are dynamically changing with time. The model follows Heap's law [16], which implies that the novelty rate decreases with time as −1 , where is typically estimated from the data (e.g., = 0.78 for del.icio.us) [40]. We estimate the Heap's law exponent for Stack Overflow as = 0.57 (distribution head fit) and = 0.315 (full data fit). Slightly lower values of the Heap's law exponent may be potentially caused by the limit on the number of tags per question and proposal of new tags on Stack Overflow. We leave the extension of the model to dynamic novelty and diversity rates for future work.
Fitting model parameters. We sketch here shortly how model parameters may be estimated from the data. Reasonable estimates for novelty 1 − and 1 − are fractions of newly introduced resources respectively tags as this is maximum likelihood estimator for the Bernoulli random variable. For diversity , we could adopt a numerical optimization for computing maximum likelihood estimator for the product of the softmax functions.
Simulations. We simulate the growth of the tagging system for = 4000 users (larger numbers create analogous effects). We draw the number of tag assignments per user from a (5, 0.6) distribution with mean of 3, which corresponds to the empirical mean of the number of tags per question on Stack Overflow (Fig. 1b). We then perform a series of experiments with various variants of the model including reinforcement-novelty, reinforcement-diversity, and reinforcement-novelty-diversity with different values for reinforcement probabilities and , and diversity factor . Figure 5 shows simulation results. The model qualitatively reproduces the behavior of social tagging systems. In a reinforcement-only system (e.g., del.icio.us [9]) we observe that initial phase of increasing tag efficiency is followed by a sharp drop and a steady decay in all of three information-theoretic measures (Fig. 5a). Extending the model to include diversity ( = 1), improves decay rate slightly but still decreases performance in the long-term (Fig. 5b). In the next model variant, we set = 0 to model question creation in Stack Overflow, where each user creates a new resource (question). Further, we set = 0.98 modelling weak novelty in tag creation but without diversity. The system performance improves as we observe the growth of tag entropy and a weak but steady increase in mutual information.
Still, the conditional entropy increases with the number of users, signaling a drop in the retrieval efficiency (Fig. 5c). Finally, we model a system with novelty ( = 0, = 0.98) and diversity ( = 1) and are able to recover the empirical Below the white lines the rate of change is positive (similarly to Stack Overflow), above the lines negative (similarly to del.icio.us). The high levels of novelty in tag and resources usage lead to an efficient system, whereas stronger reinforcement of both tags and resources causes a steep drop in tag efficiency. Without diverse selection of tags (a), the performance drop is more pronounced, while diversity in the tag usage (b) increases robustness of the system with respect to tag efficiency.
results from Stack Overflow. There is a constant increase in tag entropy and tag descriptiveness and a weak rate of increase in the conditional entropy of questions given tags indicating a stabilizing tag retrieval performance ( Fig. 5d).
In our next experiment, we iterate over the entire range [0, 1] for both and , keeping other parameters fixed.
We measure the average rate of change of all three information-theoretic measures for the last 1000 users, after the systems enter a stable phase. We repeat this experiment two times, the first time without diversity and the second time with diversity ( = 1). Figure 6 shows the heatmaps of the rate of change of mutual information between resources and tags for those two experiments. In both experiments, we observe a radial decrease in the rate of change starting from the origin, which corresponds to a pure novelty system ( = 0, = 0). The rate of change is 0 along the white lines, and is negative above the lines corresponding to a system of decaying tag performance, such as del.icio.us. At the point of pure reinforcement ( = 1, = 1) the system performance is the worst and mutual information rate of change is most negative. We also observe a substantial difference between the model with and without diversity.
In the reinforcement-novelty-diversity model, the drop in the rate of change is much slower, indicating a robust tag performance as a consequence of a more diverse tag selection. To save space, we do not show the heatmaps for tag entropy and conditional entropy: both of which show similar deteriorating tag performance with increasing reinforcement probabilities.

Limitations
This work analyzes and helps explain the long-term organization of information in Stack Overflow. While we believe that the imposed tag limit relates to the development in tag efficiency, we do not claim a causal relationship between the limit in tags and the tag efficiency. Platform-level differences between the dataset studied in previous work [9] and Stack Overflow may confound the attribution of better tag efficiency in Stack Overflow to the tag limit ad composite tags. For example, on Stack Overflow only experienced users are allowed to create new tags-experienced users have a better understanding of the domain and a better overview of the tags that are still missing in the system and can, hence, introduce more specific and precise tags, which are beneficial for tag efficiency. Previous work has already suggested the importance of expert moderation for community sense-making via tags [31]. Another confounding factor may be the limited scope of Stack Overflow, which focuses on computer science and programming-users of Stack Overflow have typically computer science background and this supports a more efficient knowledge organization as compared to a more general topic. Explicitly controlling for specific community characteristics may allow for a fairer comparison, and applying our analyses to other Stack Exchange topics and online communities would generalize our study. Alternatively, deriving some kind of counter-factual world from the observational Stack Overflow dataset might help to understand the impact of the tag limit. In a counter-factual world, there would not be limits to tags and we would observe how tag efficiency evolves over time. As this is not possible (even for Stack Overflow managers), one approximation to such a setting may involve extracting and studying descriptive terms from the body of a question, which consists of free-form text and thus does not suffer from the same restriction as tags do. Overall, we believe that there is an opportunity for future work to explore causal disentanglement of factors which may confound the impact of tag limits on tag efficiency.
Finally, our tag growth model can inform system operators qualitatively about the system evolution but can not produce exact quantitative predictions. By adopting a parameter fitting procedure as we sketched above, short-term predictions are possible-we leave the performance evaluation of such a predictive model for future work. Nevertheless, we see the utility of our model in possibility to explore other similar platforms.

CONCLUSION
In this work, we studied information growth and organization on the Q&A website Stack Overflow. In particular, we inspected the effect of tag limits on the tag efficiency. To that end, we employed a set of information-theoretic metrics, which captured information growth and information organization. We found that, while information is growing in Stack Overflow, the decrease in tag efficiency stabilizes while tag descriptiveness increases, in contrast to previous results which indicated steadily declining tag efficiency [9]. Exploring alternative explanations for Stack Overflow's long-term stability in tag efficiency, our analyses uncover the importance of the tag limit and composite tags in supporting annotation of information while maintaining long-term tag efficiency. Thus, this work indicates that system managers may trade-off between annotation capacity and tag efficiency with the choice of threshold for the tag limit per resource.
In the future, we will compare and extend our study to other Stack Exchange topics and Q&A systems such as Yahoo! Answers, or even generalize it to systems with alternative information organization approaches, such as Wikipedia. A study of these different approaches for information organization may be formulated within our information-theoretic framework. As such, extending our operationalization of information theory to learn more about information growth and organization poses an exciting prospect for future work. Furthermore, it will be critical to use a controlled experiment or causal analysis to infer the causal effect of tag limits and other designs to improve information organization.