{'fraction_non_alphanumeric': 0.027340258299170854, 'fraction_numerical': 0.007705822519495731, 'mean_word_length': 5.165716457620676, 'pattern_counts': {'":': 0, '<': 1, '<?xml version=': 0, '>': 1, 'https://': 2, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Exploring Author Gender in Book Rating and Recommendation

Collaborative filtering algorithms find useful patterns in rating and consumption data and exploit these patterns to guide users to good items. Many of the patterns in rating datasets reflect important real-world differences between the various users and items in the data; other patterns may be irrelevant or possibly undesirable for social or ethical reasons, particularly if they reflect undesired discrimination, such as discrimination in publishing or purchasing against authors who are women or ethnic minorities. In this work, we examine the response of collaborative filtering recommender algorithms to the distribution of their input data with respect to a dimension of social concern, namely content creator gender. Using publicly-available book ratings data, we measure the distribution of the genders of the authors of books in user rating profiles and recommendation lists produced from this data. We find that common collaborative filtering algorithms differ in the gender distribution of their recommendation lists, and in the relationship of that output distribution to user profile distribution.


INTRODUCTION
The evaluation of recommender systems has historically focused on the accuracy of recommendations [25,42]. When it is concerned with other characteristics, such as diversity, novelty, and user satisfaction [28,31,49], it often continues to focus on traditionallyunderstood information needs. But this paradigm, while irreplaceable in creating products that deliver immediate value, does not tell the whole story of a recommender system's interaction with its users, content creators, and other stakeholders.
In recent years, public and scholarly discourse has subjected artificial intelligence systems to increased scrutiny for their impact on their users and society. Much of this has focused on classification systems in areas of legal concern for discrimination, such as criminal justice, employment, and housing credit decisions. However, there has been interest in the ways in which more consumerfocused systems such as Uber [41], TaskRabbit [23], and search engines [35] interact with issues of bias, discrimination, and stereotyping.
Social impact is not a new concern in recommender systems. Balkanization [44] or filter bubbles, popularized by Pariser [39], are one example of this concern: do recommender systems enrich our lives and participation in society or isolate us in echo chambers? Recommender systems are intended to influence their users' behavior in some way; if they did not, there would be little reason to operate them. Understanding the ways in which recommender systems actually interact with past, present, and future user behavior is a prerequisite to assessing the ethical, legal, moral, and social ramifications of that influence.
In this paper, we report observational results from our investigation into how recommender systems interact with author gender in book data and associated consumption and rating patterns. Our first step towards that end is to characterize the distribution of author genders in existing book data sets and the response of common collaborative filtering algorithms to that distribution. We address four research questions: RQ1 How are author genders distributed in book catalog data? RQ2 How are author genders distributed in users' book reading histories? RQ3 What is the distribution of author genders in the recommendations users receive from common collaborative filtering algorithms? This measures the overall behavior of the recommender algorithm(s) with respect to author gender. RQ4 How do individual users' gender distributions propagate into the recommendations that they receive? This measures the personalized gender behavior of the algorithms.
While we expect recommender algorithms to propagate patterns in their input data, due to the general principle of "garbage in, garbage out", the particular ways in which those patterns do or do not propagate through the recommender is an open question that we seek to illuminate -recommender systems do not always propagate all input data patterns [7].
The purpose of this paper is not to make any normative claims regarding the distributions we observe, simply to describe the current state of the data and algorithms. We do not currently have sufficient data to determine whether the distributions observed in available data represents under-or over-representation, or what the "true" values are. We hope that our observations can be combined with additional information from other disciplines and from future work in this space to develop a clearer picture of the ways in which recommender systems interact with their surrounding sociotechnical ecosystems.
In the remainder of this paper, we describe our data set and experimental methodology, results of our experiment with existing algorithms, and some future directions for this line of research. The supplementary material archive accompanying this paper contains code to re-run our experiment and analyses.

BACKGROUND AND RELATED WORK
Our present work builds on work in both recommender systems and in bias and fairness in algorithmic systems more generally.

Recommender Systems
Recommender systems have long been deployed for helping users identify relevant items amongst large sets of possibilities [1,11]. Of particular interest to our current work is collaborative filtering (CF) systems, which use patterns in user-item interaction data to estimate which items a particular user is likely to find useful.
While recommender evaluation and analysis often focuses on the accuracy of recommendations [25,42], there has been significant work on non-accuracy dimensions of recommender behavior. Perhaps the best-known is diversity [49], sometimes considered along with novelty [28,45]. Lathia et al. [32] examined the temporal diversity of recommender systems, studying whether they changed their recommendations over time. Other work has quantified recommendation bias with respect to classes of items [29].

Social Impact of Recommendations
Recommender systems researchers have been concerned for how recommenders interact with various individual and social human dynamics. One example is balkanization or filter bubbles [39,44], mentioned earlier; recent work has sought to detect and quantify the extent to which recommender algorithms create or break down their users' information bubbles [37] and studied the effects of recommender feedback loops on users' interaction with items [27].
Other work seeks to use recommender technology to promote socially-desirable outcomes such as energy savings [43], better encyclopedia content [8], and new kinds of relationships [40].

Representation in the Book Industry
There are efforts in many segments of the publishing industry to improve representation of women, ethnic minorities, and other historically underrepresented groups. Multiple organizations undertake counts of books and book reviews to assess the representation of women and nonbinary individuals in the literary landscape [38,46]. We seek to understand how recommendation algorithms interact with such efforts: are they a help, a hindrance, or a neutral conduit?

Bias and Fairness in Algorithmic Systems
Questions of bias and fairness in computing systems are not new; Friedman and Nissenbaum [18] considered early on the ways in which computer systems can be (unintentially) biased in their design or impact. In the last several years, there has been increasing interest in the ways that machine learning systems are or are not fair. Dwork et al. [10] and Friedler et al. [17] have presented definitions of what it means for an algorithm to be fair. Feldman et al. [16] provide a means to evaluate arbitrary machine learning techniques in light of disparate impact, a standard for the fairness of decision-making processes adopted by the U.S. legal system.
Bias and discrimination often enter a machine learning system through the input data: the system learns to replicate the biases in its inputs. This has been demonstrated in word embeddings [3] and predictive policing systems [15,34], among others.
Burke [4] lays out some of the ways in which questions of fairness can apply to recommender systems. In particular, he considers the difference between "C-fairness", in which consumers or users of the recommender system are treated fairly, and "P-fairness", where the producers of recommended content receive fair treatment. Burke et al. [5] and Yao and Huang [48] have presented algorithms for C-fair collaborative filtering, and Ekstrand et al. [13] examine C-fairness in the accuracy of recommendation lists.
Our present study focuses on P-fairness. This dimension has not seen as much direct research, although it is related to historical concerns such as long-tail recommendation and item diversity [29]. Kamishima et al. [30] have presented algorithms for P-fair recommendation. In this paper, we present an offline empirical analysis of the P-fairness of several classical collaborative filtering algorithms and their underlying training data.

DATA AND METHODS
We address our questions through an experiment using publiclyavailable book data and common collaborative filtering techniques.

Data Sources and Integration
In order to analyze the demographic biases of user consumption patterns and resulting recommendations, we link multiple data sets to associate user ratings with book data. The accompanying code archive contains JavaScript, SQL, and R code for data import and integration, along with the LensKit recommender experiment code.

Book Consumption and Rating Data.
We use two public sources of user consumption data: BookCrossing [49] and Amazon Book Reviews [36]. These data sets provide our historical user profiles and the training data for our collaborative filtering algorithms. Both are general reading data sets, consisting of user ratings for books across a wide range of genres and styles.
3.1.2 Book Metadata. We obtain book metadata, particularly author lists, by pooling records from Open Library 1 and the Library of Congress (LOC) MARC Open-Access Records 2 .
3.1.3 Author Gender Data. We obtain author information from the Virtual Internet Authority File (VIAF) 3 , a directory of author information compiled from authority records from the Library of Congress and other libraries around the world. Author gender identity is one of the available fields for many records.
The MARC21 data model [33] employed by the VIAF is flexible in its ability to represent author gender identities, supporting an open vocabulary and begin/end dates for the validity of an identity. Unfortunately, the VIAF does not use flexibility -all its assertions are "male", "female", or "unknown". This is a significant limitation that we discuss more fully in Section 5.1.

Linking Data Sets.
We link book data with rating data by ISBN. To decrease sparsity, improve data linking coverage, and recommend at the level of creative works instead of individual editions, we link related ISBNs. We form a bipartite graph of IS-BNs and records (LOC records, OpenLibrary "edition" records, and OpenLibrary "work" records when available) and consider each connected component to be a "book". Rarely (less than 1% of ratings) this causes a user to have multiple ratings for a book; we resolve multiple ratings by taking the median of their ratings.
Because OpenLibrary, LOC, and VIAF do not share linking identifiers, we must link books to authority records by author name. Each VIAF authority record can contain multiple name entries, recording different forms or localizations of the author's name. OpenLibrary author records also carry multiple known forms of the author's name. After normalizing names to improve matching (cleaning punctuation and ensuring both "Last, First" and "First Last" forms are available), we locate all VIAF records containing a name that matches one of the listed names for the first author of any OpenLibrary or LOC records in a book's cluster. If all records that contain an assertion of the author's gender agree, we take that to be the author's gender; if there contradicting gender statements, we code the book's author gender as "ambiguous".
We selected this strategy to ensure good coverage while maintaining reasonable confidence in classification. Less than 5% of rated books have 'ambiguous' author genders. Figure 1 summarize the results of integrating these data sets. In BookCrossing, approximately 75% of ratings are for books whose first author's gender we can identify; in Amazon, this drops to about 35% (27% of books). "% Female Books" and "% Female Ratings" indicate the fraction of known-gender books (or ratings) with female authors. While the data is somewhat sparse, it has sufficient coverage for us to perform 1 https://openlibrary.org/developers/dumps 2 https://www.loc.gov/cds/products/marcDist.php 3 http://viaf.org/viaf/data/   a meaningful analysis. We also report coverage of the Library of Congress data itself, as a rough approximation of books published irrespective of whether they are rated.

Experimental Design
Our main experiment has several steps: (1) Import and integrate data, detect author genders.
(2) Sample 1000 users, each of whom has rated at least 5 books with known author gender, for analysis. This sampling keeps the analysis computationally tractable while considering enough users to draw statistically valid conclusions.    compare with user profile distribution (RQ4). Gender could correlate with book rating behavior in two ways: the selection of books to read and/or rate, and the rating values given to those books the user has chosen to read. To account for these distinct effects, we run two versions of this experiment: one uses the rating values with explicit-feedback algorithm configurations; the other ignores rating values, treating each implicit or explicit as a "read" event, and operates the collaborative filters in implicit feedback mode. For BookCrossing, we perform these variants with separate user samples: explicit data sampled from the BookCrossing data set excluding implicit-only ratings (BXE), and implicit data sampled from the whole data set discarding rating values (BXA). For Amazon (AZ), we use a single user sample but compute recommendations both with and without rating values.

Recommending Books
We used the LensKit toolkit [12] to produce 50 recommendations for each of our 1000 sample users using the following algorithms: • UserUser, a user-based collaborative filter [24].
We tuned each algorithm by optimizing nDCG on a train-test split of the consumption data. Full configurations are in the accompanying code; as we examine the behavior of algorithms, not test their effectiveness, precise details are omitted for space.

Statistical Analysis
We model user rating behaviors using a hierarchical Bayesian model [20] for the observed number of books by female authors out of the set of books with known authors. We extend this to model recommendation list distributions as a linear function of user profile distributions plus random variance. We select this strategy for three reasons: the variance in user profile sizes makes it difficult to directly compare gender proportions between users, a hierarchical Bayesian model allows us to integrate information across users to estimate a user's tendency even when they have not rated very Table 3: Summary of key model parameters and variables.
Variable Description n u Number of known-gender books rated by user u y u Number of female-authored books rated by u θ u Probability of a known-author book rated by u being by a female author (smoothed author-gender balance) µ Expected user gender balance, in log-odds (E[logit(θ u )]) σ 2 Variance of user gender balancē n ua Number of known-gender books algorithm a recommended to user ū y ua Number of female-authored books a recommended to ū θ ua Gender balance of algorithm a's recommendations for u s a Regression slope of algorithm a (its responsiveness to user profile tendency) b a Intercept of algorithm a σ 2 a Residual variance of algorithm a (its variability unexplained by user tendencies) many books, and integrated Bayesian models enable us to robustly infer a number of parameters in a manner that clearly quantifies uncertainty and avoids many of the multiple-comparison problems that often plague this kind of analysis [21]. Figure 2 shows the plate diagram for our model, and Table 3 summarizes the key variables.

User Profiles.
For each user, we observe n u , the number of books they have rated with known author genders, and y u , the number of female-authored books they have rated. From these observations, we estimate each user's author-gender tendency θ u using a logit-normal model 4 to address RQ2. We also model n u as a random variable with a negative binomial distribution to produce more realistic predicted observations for unseen users to test model fit. We use the following joint probability as our likelihood model: logit(θ u ) is the log odds of a known-gender book rated by user u being written by a female author, and µ and σ are the mean and standard deviation of this user author-gender tendency. Negative values indicate a tendency towards male authors, and positive values a tendency towards female authors. θ j is the corresponding probability or proportion in the range [0, 1]. When sampling from the fitted model, we produce a predicted θ ′ , n ′ , y ′ , and observed ratio y ′ /n ′ for each sample in order to estimate the distribution of unseen user profiles.

Recommendation
Lists. For RQ3 and RQ4, we model recommendation list gender distributions by extending our Bayesian model to incorporate observed recommendation distributions via a linear regression based on a user's smoothed proportion and per-algorithm slope, intercept, and variance. This results in the following formula for estimatingθ ua : For recommendation lists, we omit the binomial distribution used for modeling user profiles, and instead directly computeθ ua = (ȳ ua + 1)/(n ua + 2). This change is because recommendations are not independent between users, and the highly consistent recommendations produced by some algorithms cause a binomial model to fit poorly (observed proportions are severely underdispersed). The regression residual ϵ ua captures variance in the relationship between users' and algorithms' recommendation proportions, and giving it per-algorithm variance allows for some algorithms being more consistent in their output than others. The result is that s a captures how much an algorithm's output gender distribution varies with the input profile distribution, and σ 2 a its variance independent of the input distribution.

Implementation.
We fit and sample models with STAN 2.17.3 [6], drawing 10,000 samples per model (4 NUTS chains each performing 2500 warmup and 2500 sampling iterations). We report results with the posterior predictive distributions of the parameters of interest, as estimated by the sampling process.

RESULTS
In this section we present our experimental results with existing algorithms on our three data sets. We begin with characterizing the profiles of our sample users, and then proceed to analyze the resulting recommendations.

Baseline Distribution
The statistics of our underlying data sets, as presented in Tables 1-2, address RQ1. If we consider LOC as a representative sample of books-in-print, 29.2% of books with known author genders are female-authored; when considering all books appearing in our catalog data set, the proportion drops to 23.7%. Rating data has a more balanced distribution: 39.9% of known-gender books in BookCrossing are written by women, and 45.3% of ratings of known-gender books are for female-authored books. In Amazon the proportions are closer to baseline but are still higher (29.5% of books and 36.2% of ratings). If women are underrepresented among published authors, these results suggest that such underrepresentation is reduced in ratings from the systems we consider.

User Profile Characteristics
Our investigation into RQ2 focuses on the distribution of users' author-gender tendencies, as represented by the proportion of known-gender books in each author's profile that are written by female authors. Figure 3 shows the distribution of user profile sizes,   and Figure 4 shows the distribution of observed author gender proportions. Table 4 presents user profile summary statistics. The Bayesian model from Section 3.4.1 provides more rigorous, smoothed estimates of this distribution. Table 4 describes the numerical results of this inference. The key parameters are µ, the average user's author-gender tendency in log-odds; σ , the standard deviation of user author-gender tendencies; and sampled θ values, the distribution of which describes the distribution of user author-gender tendencies expressed as expected proportions. Figure 4 shows the densities of the author-gender tendency distribution, along with the densities of projected and actual observed  To support direct comparison of the densities of observations and predictions, we resampled observed proportions with replacement to yield 10,000 observations. We observe a population tendency to rate male authors more frequently than female authors in all data sets (µ < 0), but to rate female authors more frequently than they would be rated were users drawing books uniformly at random from the available set. The average user author-gender tendency is slightly closer to an even balance than the set of rated books. We also found a large diversity amongst users about their estimated tendencies (s.d. of predicted θ exceeds 0.2; inferred siдma > 1; both even-odds and book population proportions are within one s.d. of estimated means). This means that some users are estimated to strongly favor female authored books, even if these users are outnumbered by those that primarily read male-authored books. The Amazon data set has the strongest tendency (µ = −0.82, mean(θ ) = 0.37, sd(θ ) = 0.28), with a particular spike in highly-male profiles.

Recommendation List Distributions
Our first step in understanding how collaborative filtering algorithms respond to this data bias is to examine the distribution of recommender list tendencies (RQ3). As described in 3.3, we produced 50 recommendations from each algorithm. Table 5 shows the basic coverage statistics of these algorithms along with corresponding user profile statistics. Users for which an algorithm could not produce recommendations are rare. We also computed the extent to which algorithms recommend different items to different users; "% Dist. " is the percentage of all recommendations that were distinct items. Algorithms that repeatedly recommend the same items will be consistent in the gender distributions of their recommendations. Table 6 provides the mean tendency for recommendation lists produced by each of our algorithms, plus the tendency of Most Popular and Highest Average Rating recommenders. Figure 5 shows the density of observed recommendation list proportions.
All recommenders were more consistent in their tendencies than the underlying user profiles. UserUser, Item Item, and PF exhibit significant variance in the items they recommend and the gender distributions of their output lists, though all are more concentrated than the user profile distribution. Their mean tendencies are comparable to input profile tendencies, as well as the popular-item tendency; the exact relationship varies from data set to data set. The MF algorithm barely personalizes at all, likely due to the high sparsity of the data; as a result, its observed tendency is much  more concentrated. In the BookCrossing data, it tends to favor male authors more than the underlying data would support; in implicit feedback mode, it is highly biased towards male authors with respect even to the baseline distributions.

From Profiles to Recommendations
Our extended Bayesian model (Section 3.4.2) allows us to address RQ4: the extent to which our algorithms propagate individual users' tendencies into their recommendations (RQ4). Figure 5 shows the posterior predictive and observed densities of recommender author-gender tendencies, and Figure 6 shows scatter plots of observed recommendation proportions against user profile proportions with regression curves (regression lines in logodds space projected into probability space). Table 7 contains the parameters inferred for each regression line.
The k-NN algorithms are responsive to individual users' historical tendencies, as indicated by the positive slopes (s a ) with credible intervals excluding zero. MF is almost entirely unresponsive; PF responds some but not nearly so much as the k-NN algorithms and exhibits higher independent variance (σ 2 a ). The posterior model also does not fit as well for PF, because of its combination of responsiveness and global consistency. Our model can fit generally unresponsive curves such as MF, and generally responsive curves such as the k-NN models; PF sits in an awkward place. Visual inspection of the scatter plot suggests that there is a strong component with consistent tendencies, but the regression may accurately model the remaining users. Future work will use a model that can better account for some global consistency.

Summary
RQ1 -Baseline Gender Distribution Known books are significantly more likely to be written by men than by women; representation among rated books is more balanced. RQ2 -User Input Gender Distributions User are diffuse in their rating tendencies, with an overall trend favoring male authors but less strongly than the baseline distribution. RQ3 -Recommender Output Distributions Different CF techniques produce recommendations with quite different distributions. Matrix factorization on BookCrossing produced reliably male-biased recommendations, while nearestneighbor and PF techniques were closer to the user profile tendency while being less diffuse than their inputs. Some algorithm and data set combinations resulted in recommendations that were more balanced than their inputs. RQ4 -Distribution Propagation Most algorithms reflected some of each user's profile tendency in their recommendations; this effect was substantially stronger for implicitfeedback recommendations than explicit-feedback. Classical matrix factorization did not exhibit significant personalization of any kind, likely due to data sparsity.

DISCUSSION
We found that users in the BookCrossing data set exhibit mild, diffuse tendency towards books written by men; users in the Amazon data set exhibit a somewhat stronger but still highly diffuse tendency. Both tendencies are more evenly balanced than the set of available of books. Collaborative filtering algorithms trained on this data exhibit remarkably different behavior; some learn substantially stronger and more consistent tendencies, in some cases producing lists more imbalanced than the item universe. Others propagate users' tendencies into their recommendation lists. Nearest-neighbor recommenders in implicit feedback mode also propagated much of each user's profile tendencies into their recommendations. One interpretation of this is that they are partially picking up on a user's preference for books by male or female authors and reflecting this preference in the recommendations, which is what we would expect from a personalized recommender algorithm. The matrix factorization technique we tested consistently exhibited a much stronger bias towards male authors than was present in the input data, and was largely oblivious to individual users' preferences or biases (or, indeed, their book preferences). It also did not produce very accurate recommendations in our parameter tuning compared to the other algorithms.
The answer to the question "how do recommenders interact with gender distributions?" is therefore not simple. It has good company with other questions of the social impact of recommendations; for example, contrary to the filter bubble hypothesis, recommender algorithms had a diversifying effect on users' viewing portfolios in one movie recommendation service [37]. Exact answers likely depend on algorithm, application, and a number of other variables.

Limitations of Data and Methods
Our data and approach has a number of limitations that are important to note. First, book rating data is extremely sparse, and the BookCrossing data set is small, providing a limited picture of users' reading histories and reducing the performance of some algorithms. In particular, the high sparsity of the data set caused the MF algorithm to perform particularly poorly on offline accuracy metrics, so these findings may not be representative of its behavior in the wild; future work will need to test them across a range of recommender effectiveness levels and stages of system cold-start.
Second, our data and statistical methods only account for binary gender identities. While the MARC21 Authority Format supports flexible gender identity records (including multiple possiblyoverlapping identities over the course of an author's life and nonbinary identities from an open vocabulary), VIAF does not seem to use this flexibility. The result is that gender minorities are not represented, or are misgendered, in the available data; we agree with Hoffmann [26] that this is a significant problem.
Third, we test a limited set of collaborative filtering algorithms. While we have chosen algorithms with an eye for diverse behaviors and global popularity, we must acknowledge that our selection of 5 algorithms is small in the face of algorithm diversity in the field. While our ultimate goal is to understand general trends, we acknowledge that our study does not evaluate enough algorithms to make claims about the entire field.
We consider it valuable to make forward progress in understanding the interaction of information systems with social concerns using the data we have available, even if that data has significant known weaknesses. We must, however, be reflective and forthright about the limitations of the data, methods, and resulting findings, and seek to improve them in order to develop a better understanding of the human impact of computing systems. Our experimental design can be readily extended to accommodate richer or higherquality data sources and additional algorithms, and the code we provide for our experiments will facilitate such improvements. We have tested this reproducibility by re-running the experiments in the course of writing and revising this paper. Ultimately we see this as the first step in untangling a broader issue; we are actively exploring many extensions and improvements to this work.

CONCLUSION AND THE ROAD AHEAD
We have conducted an initial inquiry into the response of collaborative filtering book recommenders to gender distributions in the user preference data on which they are trained. The algorithms differed in their response to these distributions. This paper is a first step in a much larger project to understand the ways in which recommendation algorithms interact with potentially discriminatory biases, and general behavior of recommendation technology with respect to various social issues. There are many future steps we see for advancing this agenda: • Obtaining higher-quality data for measuring distributions of interest in recommender inputs and outputs. This includes obtaining data on non-binary gender identities and extending our statistical methods to account for them. • Examining other content creator features, such as ethnicity, in recommendation applications. • Studying other domains and applications, such as movies, research literature, and social media. • Considering additional recommendation techniques.
• Studying change in distribution over time of both user consumption patterns and recommender outputs. • Develop more advanced algorithms that interact with various user or item characteristics of social concern; these could be developed to reflect organizational or societal goals or to help users further their individual goals [14]. • Study the effect of existing refinements, such as diversification [47,49], on recommendation distributions.
We hope to see more work in the coming years to better understand ways in which recommender systems respond to and influence their sociotechnical contexts.