{'fraction_non_alphanumeric': 0.035870132008672506, 'fraction_numerical': 0.01462798803414112, 'mean_word_length': 4.707706766917293, 'pattern_counts': {'":': 0, '<': 1, '<?xml version=': 0, '>': 2, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 1, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 12, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
A Statutory Article Retrieval Dataset in French

Statutory article retrieval is the task of automatically retrieving law articles relevant to a legal question. While recent advances in natural language processing have sparked considerable interest in many legal tasks, statutory article retrieval remains primarily untouched due to the scarcity of large-scale and high-quality annotated datasets. To address this bottleneck, we introduce the Belgian Statutory Article Retrieval Dataset (BSARD), which consists of 1,100+ French native legal questions labeled by experienced jurists with relevant articles from a corpus of 22,600+ Belgian law articles. Using BSARD, we benchmark several state-of-the-art retrieval approaches, including lexical and dense architectures, both in zero-shot and supervised setups. We find that fine-tuned dense retrieval models significantly outperform other systems. Our best performing baseline achieves 74.8% R@100, which is promising for the feasibility of the task and indicates there is still room for improvement. By the specificity of the domain and addressed task, BSARD presents a unique challenge problem for future research on legal information retrieval. Our dataset and source code are publicly available.


Introduction
Legal issues are an integral part of many people's lives (Ponce et al., 2019). However, the majority of citizens have little to no knowledge about their rights and fundamental legal processes (Balmer et al., 2010). As the Internet has become the primary source of information in response to life problems (Estabrook et al., 2007), people increasingly turn to search engines when faced with a legal issue (Denvir, 2016). Nevertheless, the quality of the search engine's legal help results is currently unsatisfactory, as top results mainly refer people to commercial websites that provide basic information as a way to advertise for-profit services (Hagan and Li, 2020). On average, only one in five persons obtain help from the Internet to clarify or solve their legal issue (Ponce et al., 2019). As a result, many vulnerable citizens who cannot afford a legal expert's costly assistance are left unprotected or even exploited. This barrier to accessing legal information creates a clear imbalance within the legal system, preventing the right to equal access to justice for all.
People do not need legal services in and of themselves; they need the ends that legal services can provide. Recent advances in natural language processing (NLP), combined with the increasing amount of digitized textual data in the legal domain, offer new possibilities to bridge the gap between people and the law. For example, legal judgment prediction (Aletras et al., 2016;Luo et al., 2017;Chen et al., 2019) may assist citizens in finding insightful patterns between their case and its outcome. Additionally, legal text summarization (Hachey and Grover, 2006;Bhattacharya et al., 2019) and automated contract review (Harkous et al., 2018;Lippi et al., 2019) may help people clarify long, complex, and ambiguous legal documents.
In this work, we focus on statutory article retrieval, which, given a legal question such as "Is it legal to contract a lifetime lease? ", aims to return one or several relevant law articles from a body of legal statutes (Kim et al., 2019;Nguyen et al., 2020), as illustrated in Figure 1. A qualified statutory article retrieval system could provide a professional assisting service for unskilled humans and help empower the weaker parties when used for the public interest.
Finding relevant statutes to a legal question is a challenging task. Unlike traditional ad-hoc information retrieval (Craswell et al., 2020), statutory article retrieval deals with two types of language: common natural language for the questions and complex legal language for the statutes. This difference in language distribution greatly complicates the retrieval task as it indirectly requires an inherent interpretation system that can translate a natural question from a non-expert to a legal question to be matched against statutes. For skilled legal experts, these interpretations come from their knowledge of a question's domain and their understanding of the legal concepts and processes involved. Nevertheless, an interpretation is rarely unique. Instead, it is the interpreter's subjective belief that gives meaning to the question and, accordingly, an idea of the domains in which the answer can be found. As a result, the same question can yield different paths to the desired outcome depending on its interpretation, making statutory article retrieval a difficult and time-consuming task.
Besides, statutory law is not a stack of independent articles to be treated as complete sources of information on their own -unlike news or recipes. Instead, it is a structured and hierarchical collection of legal provisions that have whole meaning only when considered in their overall context, i.e., together with the supplementary information from their neighboring articles, the fields and sub-fields they belong to, and their place in the hierarchy of the law. For instance, the answer to the question "Can I terminate an employment contract? " will most often be found in labor law. However, this is not necessarily true if an employer is contracting a self-employed worker to carry out a specific task, in which case the answer probably lies at the higher level of contract law. This example illustrates the importance of considering the question's context and understanding the hierarchical structure of the law when looking for relevant statutory articles.
In order to study whether retrieval models can approximate the efficiency and reliability of legal experts, we need a suitable labeled dataset. However, such datasets are difficult to obtain considering that, although statutory provisions are generally publicly accessible (yet often not in a machine-readable format), the questions posed by citizens are not.
This work presents a novel French native expertannotated statutory article retrieval dataset as its main contribution. Our Belgian Statutory Article Retrieval Dataset (BSARD) consists of more than 1,100 legal questions posed by Belgian citizens and labeled by legal experts with references to relevant articles from a corpus of around 22,600 Belgian law articles. As a second contribution, we establish strong baselines on BSARD by comparing diverse state-of-the-art retrieval approaches from lexical and dense architectures. Our results show that fine-tuned dense retrieval models significantly outperform other approaches yet suggest ample opportunity for improvement. We publicly release our dataset and source code at https: //github.com/maastrichtlawtech/bsard.

Related Work
Due to the increasing digitization of textual legal data, the NLP community has recently introduced more and more datasets to help researchers build reliable models on several legal tasks. For instance, Fawei et al. (2016) introduced a legal question answering (LQA) dataset with 400 multi-choices questions based on the US national bar exam. Similarly, Zhong et al. (2020)  Recently, Hendrycks et al. (2021) proposed a dataset for legal contract review that includes 510 contracts annotated with 41 different clauses for a total of 13,101 annotations. In the same vein, Borchmann et al. (2020) introduced a semantic retrieval dataset for contract discovery with more than 2,500 annotations in around 600 documents. Lastly, the COLIEE Case Law Corpus (Rabelo et al., 2020) is a case law retrieval and entailment dataset that includes 650 base cases from the Federal Court of Canada, each with 200 candidate cases to be identified as relevant to the base case.
Regarding statutory article retrieval, the only other publicly available dataset is the COLIEE Statute Law Corpus (Rabelo et al., 2020). It comprises 696 questions from the Japanese legal bar exam labeled with references to relevant articles from the Japanese Civil Code, where both the questions and articles have been translated from Japanese to English. However, this dataset focuses on legal bar exam question answering, which is quite different from legal questions posed by ordinary citizens. While the latter tend to be vague and straightforward, bar exam questions are meant for aspiring lawyers and are thus specific and advanced. Besides, the dataset only contains closed questions (i.e., questions with "yes" or "no" answers) and considers almost 30 times fewer law articles than BSARD does. Also, unlike BSARD, the data are not native sentences but instead translated from a foreign language with a completely different legal system. 1 As a result, the translated dataset may not accurately reflect the logic of the original legal system and language. These limitations suggest the need for a novel large-scale citizen-centric native dataset for statutory article retrieval, which is the core contribution of the present work.

Dataset Collection
We create our dataset in four stages: (i) compiling a large corpus of Belgian law articles, (ii) gathering legal questions with references to relevant law articles, (iii) refining these questions, and (iv) matching the references to the corresponding articles from our corpus.
Law articles collection. In civil law jurisdictions, a legal code is a type of legislation that purports to exhaustively cover a whole area of law, such as criminal law or tax law, by gathering and restating all the written laws in that area into a unique book. Hence, these books constitute valuable resources to collect many law articles on various subjects. We consider 32 publicly available Belgian codes, as presented in Table 3 of Appendix A. Together with the legal articles, we extract the corresponding headings of the sections in which these articles appear (i.e., book, part, act, chapter, section, and subsection names). These headings provide an overview of each article's subject. As preprocessing, we use regular expressions to clean up the articles of specific wording indicating a change in part of the article by a past law (e.g., nested brackets, superscripts, or footnotes). Additionally, we identify and remove the articles repealed by past laws but still present in the codes. Eventually, we end up with a corpus C = {a 1 , · · · , a N } of N = 22, 633 articles that we use as our basic retrieval units.
Questions collection. We partner with Droits Quotidiens (DQ), 2 a Belgian organization whose mission is to clarify the law for laypeople. Each year, DQ receives and collects around 4,000 emails from Belgian citizens asking for advice on a personal legal issue. Thanks to these emails, its team of six experienced jurists keeps abreast of Belgium's most common legal issues and addresses them as comprehensively as possible on its website. Each jurist is an expert in a specific field (e.g., "family", "housing", or "work") and is responsible for answering all questions related to that field. Given their qualifications and years of experience in providing legal advice in their respective fields, the experts can be considered competent enough to always (eventually) retrieve the correct articles to a given question. In practice, their legal clarification process consists of four steps. First, they identify the most frequently asked questions on a common legal issue. Then, they define a new anonymized "model" question on that issue expressed in natural language terms, i.e., as close as possible as if a layperson had asked it. Next, they search the Belgian law for articles that help answer the model question and reference them. Finally, they answer the question using the retrieved relevant articles in a way a layperson can understand. These model questions, legal references, and answers are further categorized before being posted on DQ's website (e.g., the question "What is the seizure of goods? " is tagged under the "Money → Debt recovery" category). With their consent, we collect more than 3,200 model questions together with their references to relevant law articles and categorization tags.
Assuming it takes a jurist between 5 to 20 minutes to find the relevant articles to a given question and categorize the latter. An estimate of the pecuniary value of those labeled questions is over C105,000 -3,200 questions, each requiring 10 minutes to label, assuming a rate of C200 per hour.
Questions refinement. We find that around onethird of the collected questions are duplicates. However, these duplicated questions come with different categorization tags, some of which providing additional context that can be used to refine the questions. For example, the question "Should I install fire detectors? " appears four times in total, under the following tags: "Housing → Rent → I am a {tenant, landlord} → In {Wallonia, Brussels} ". We distinguish between the tags with one or a few words indicating a question subject (e.g., "housing" and "rent") and those that provide context about a personal situation or location as short descriptive sentences (e.g., "I am tenant in Brussels."). If any, we append the contextual sentence tags in front of the questions, which solves most of the duplicates problem and improves the overall quality of the questions by making them more specific.
Questions filtering. The questions collected are annotated with plain text references to relevant law articles (e.g., "Article 8 of the Civil Code"). We use regular expressions to parse these references and match them to the corresponding articles from our corpus. First, we filter out questions whose references are not articles (e.g., an entire decree or order). Then, we remove questions with references to legal acts other than codes of law (e.g., decrees, directives, or ordinances). Next, we ignore questions with references to codes other than those we initially considered. We eventually end up with 1,108 questions, each carefully labeled with the ids of the corresponding relevant law articles from our corpus. Finally, we split the dataset into training/test sets with 886 and 222 questions, respectively.

Dataset Analysis
To provide more insight, we describe quantitative and qualitative observations about BSARD. Specifically, we explore (i) the diversity in questions and articles, (ii) the relationship between questions and their relevant articles, and (iii) the type of reasoning required to retrieve relevant articles.
Diversity. The 22,633 law articles that constitute our corpus have been collected from 32 Belgian codes covering a large number of legal topics, as presented in Table 3 of Appendix A. The articles have a median length of 77 words, but 142 articles exceed 1,000 words (the lengthiest one being up to 5,790 words), as illustrated in Figure 2b. These long articles are mostly general provisions, i.e., articles that appear at the beginning of a code and define many terms and concepts later mentioned in the code. The questions are between 5 and 44 words long, with a median of 14 words, as shown in Figure 2a. They cover a wide range of topics, with around 85% of them being either about family,   housing, money, or justice, while the remaining 15% concern either social security, foreigners, or work, as described in Table 1.
Question-article relationship. Questions might have one or several relevant legal articles. Overall, 75% of the questions have less than five relevant articles, 18% have between 5 and 20, and the remaining 7% have more than 20 with a maximum of 109, as seen in Figure 2c. The latter often have complex and indirect answers that demand extensive reasoning over a whole code section, which explains these large numbers of relevant articles. Furthermore, an article deemed relevant to one question might also be for others. Therefore, we calculate for each unique article deemed relevant to at least one question the total number of times it is cited as a legal reference across all questions. As a result, we find that the median number of citations for those articles is 2, and less than 25% of them are cited more than five times, as illustrated in Figure 2d. Hence, out of the 22,633 articles, only 1,612 are referred to as relevant to at least one question in the dataset, and around 80% of these 1,612 articles come from either the Civil Code, Judicial Code, Criminal Investigation Code, or Penal Code. Meanwhile, 18 out of the 32 codes have less than five articles men-tioned as relevant to at least one question, which can be explained by the fact that those codes focus less on individuals and their concerns.

Models
Formally speaking, a statutory article retrieval system R : (q, C) → F is a function that takes as input a question q along with a corpus of law articles C, and returns a much smaller filter set F ⊂ C of the supposedly relevant articles, ranked by decreasing order of relevance. For a fixed k = |F| ≪ |C|, the retriever can be evaluated in isolation with multiple rank-based metrics (see Section 5.1). The following section describes the retrieval models we use as a benchmark for the task.

Lexical Models
Traditionally, lexical approaches have been the de facto standard for textual information retrieval due to their robustness and efficiency. Given a query q and an article a, a lexical model assigns to the pair (q, a) a score s L : (q, a) → R + by computing the sum, over the query terms, of the weights of each query term t ∈ q in the article, i.e., First, we use the TF-IDF weighting scheme, in which where the term frequency tf is the number of occurrences of term t in article a, and the document frequency df is the number of articles within the corpus that contain term t. Then, we experiment with the BM25 weighting formula (Robertson et al., 1994), defined as where k 1 ∈ R + and b ∈ [0, 1] are constant parameters to be fixed, |a| is the article length, and avgal is the average article length in the collection. During inference, we compute a score for each article in corpus C and return the k articles with the highest scores as the top-k most relevant results to the input query.

Dense Models
Lexical approaches suffer from the lexical gap problem (Berger et al., 2000) and can only retrieve articles containing keywords present in the query. To overcome this limitation, recent work (Lee et al., 2019;Karpukhin et al., 2020;Xiong et al., 2021) relies on neural-based architectures to capture semantic relationships between the query and documents. The most commonly used approach is based on a bi-encoder model (Gillick et al., 2018) that maps queries and documents into dense vector representations. Formally, a dense retriever calculates a relevance score s D : (q, a) → R + between question q and article a by the similarity of their respective embeddings h q , h a ∈ R d , i.e., where sim : R d × R d → R is a similarity function such as dot product or cosine similarity. Typically, these embeddings result from a pooling operation on the output representations of a word embedding model: h q = pool (f (q; θ 1 )) , and h a = pool (f (a; θ 2 )) , where model f (·; θ i ) : W n → R n×d with parameters θ i maps an input text sequence of n terms from vocabulary W to d-dimensional real-valued word vectors. The pooling operation pool : R n×d → R d uses the output word embeddings to distill a global representation for the text passage -using either mean, max, or [CLS] pooling. Note that the bi-encoder architecture comes with two flavors: (i) siamese (Reimers and Gurevych, 2019;Xiong et al., 2021), which uses a unique word embedding model (i.e., θ 1 = θ 2 ) that maps the query and article together in a shared dense vector space, and (ii) two-tower (Yang et al., 2020;Karpukhin et al., 2020), which use two independent word embedding models that encode the query and article separately into different embedding spaces.
During inference, the articles are pre-encoded offline, and their representations are stored in an index structure. Then, given an input query, an exact search is performed by computing the similarities between the query representation and all pre-encoded article representations. The resulting scores are used to rank the articles such that the k articles that have the highest similarities with the query are returned as the top-k results.

Zero-Shot Evaluation
First, we study the effectiveness of siamese biencoders in a zero-shot evaluation setup, i.e., pre-trained word embedding models are applied out-of-the-box without any additional fine-tuning. We experiment with two types of widely-used word embedding models: (i) models that learned context-independent word representations, namely word2vec (Mikolov et al., 2013a,b) and fast-Text (Bojanowski et al., 2017), and (ii) models that learned context-dependent word embeddings, namely RoBERTa .
RoBERTa can process texts up to a maximum input length of 512 tokens. Although alternative models exist to alleviate this limitation (Beltagy et al., 2020;Ainslie et al., 2020), they have all been trained on English text, and there are no French equivalents available yet. Therefore, we use a simple workaround that splits the text into overlapping chunks and passes each chunk in turn to the embedding model. To form the chunks, we consider contiguous text sequences of 200 tokens with an overlap of 20 tokens between consecutive chunks.
For all zero-shot models, we use mean pooling on all word embeddings of the passage to extract a global representation for the latter and cosine similarity to score passage representations.

Training
Thereafter, we train our own siamese and two-tower RoBERTa-based bi-encoder models on BSARD. Let D = {⟨q i , a + i ⟩} N i=1 be the training data where each of the N instances consists of a query q i associated with a relevant (positive) article a + i . Using in-batch negatives (Chen et al., 2017;Henderson et al., 2017), we can create a training set i is a set of negative articles for question q i constructed by considering the articles paired with the other questions from the same mini-batch. For each training instance, we contrastively optimize the negative log-likelihood of each positive article against their negative articles, i.e., where τ > 0 is a temperature parameter to be set. This contrastive loss allows learning embedding functions such that relevant question-article pairs will have a higher score than irrelevant ones. To deal with articles longer than 512 tokens, we use the same workaround as in the zero-shot evaluation and split the long sequences into overlapping chunks of 200 tokens with a window size of 20. However, this time, we limit the size of the articles to the first 1,000 words due to limited GPU memory. Although not ideal, doing so remains reasonable given that only 0.6% of the articles in our corpus have more than 1,000 words, as mentioned in Section 3.2. Each chunk is prefixed by the [CLS] token, and we extract a global representation for the whole article by averaging the output [CLS] token embeddings of the different chunks. Here, we use the dot product to compute similarities as it gives slightly better results than cosine.

Experiments
We now describe the setup we use for experiments and evaluate the performance of our models.

Experimental Setup
Metrics. We use three standard information retrieval metrics (Manning et al., 2008) to evaluate performance, namely the (macro-averaged) recall@k (R@k), mean average precision@k (MAP@k), and mean reciprocal rank@k (MRR@k). Appendix B gives a detailed description of these metrics in the context of statutory article retrieval. We deliberately omit to report the precision@k given that questions have a variable number of relevant articles (see Figure 2c), which makes it senseless to report it at a fixed k -questions with r relevant articles will always have P@k < 1 if k > r. For the same reason, k should be large enough for the recall@k. Hence, we use k ∈ {100, 200, 500} for our evaluation.
French word embedding models. Our focus is on a non-English dataset, so we experiment with French variants of the models mentioned above. Specifically, we use a 500-dimensional skip-gram word2vec model pre-trained on a crawled French corpus (Fauconnier, 2015), a 300-dimensional CBOW fastText model pre-trained on French Web data (Grave et al., 2018), and a French RoBERTa model, namely CamemBERT (Martin et al., 2020), pre-trained on 147GB of French web pages filtered from Common Crawl. 3 Hyper-parameters & schedule. For BM25, we optimize the parameters on BSARD training set and find k 1 = 1.0 and b = 0.6 to perform best. Regarding the bi-encoder models, we optimize the contrastive loss using a batch size of 22 questionarticle pairs and a temperature of 0.05 for 100 epochs, which is approximately 20,500 steps. We use AdamW (Loshchilov and Hutter, 2019) with an initial learning rate of 2e-5, β 1 = 0.9, β 2 = 0.999, weight decay of 0.01, learning rate warm up over the first 500 steps, and linear decay of the learning rate. Training is performed on a single Tesla V100 GPU with 32 GBs of memory and evaluation on a server with a dual 20 core Intel(R) Xeon(R) E5-2698 v4 CPU @2.20GHz and 512 GBs of RAM.

Results
In Table 2, we report the retrieval performance of our models on the BSARD test set. Overall, the trained bi-encoder models significantly outperform all the other baselines. The two-tower model improves over its siamese variant on recall@100 but performs similarly on the other metrics. Although BM25 underperforms the trained bi-encoders significantly, its performance indicates that it is still a strong baseline for domain-specific retrieval. These results are consistent with those obtained on other in-domain datasets .  Regarding the zero-shot evaluation of siamese bi-encoder models, we find that directly using the embeddings of a pre-trained CamemBERT model without optimizing for the IR task gives poor results. Reimers and Gurevych (2019) noted similar findings for the task of semantic textual similarity. Furthermore, we observe that the word2vecbased bi-encoder significantly outperforms the fast-Text and BERT-based models, suggesting that pretrained word-level embeddings are more appropriate for the task than character-level or subwordlevel embeddings when used out of the box.
Although promising, these results suggest ample opportunity for improvement compared to a skilled legal expert who can eventually retrieve all relevant articles to any question and thus get perfect scores.

Discussion
This section discusses the limitations and broader impacts of our dataset.

Limitations
As our dataset aims to give researchers a welldefined benchmark to evaluate existing and future legal information retrieval models, certain limitations need to be borne in mind to avoid drawing erroneous conclusions.
First, the corpus of articles is limited to those collected from the 32 Belgian codes described in Table 3 of Appendix A, which does not cover the entire Belgian law as thousands of articles from decrees, directives, and ordinances are missing. During the dataset construction, all references to these uncollected articles are ignored, which causes some questions to end up with only a fraction of their initial number of relevant articles. This information loss implies that the answer contained in the remaining relevant articles might be incomplete, although it is still appropriate.
Additionally, it is essential to note that not all legal questions can be answered with statutes alone.
For instance, the question "Can I evict my tenants if they make too much noise? " might not have a detailed answer within the statutory law that quantifies a specific noise threshold at which eviction is allowed. Instead, the landlord should probably rely more on case law and find precedents similar to their current situation (e.g., the tenant makes two parties a week until 2 am). Hence, some questions are better suited than others to the statutory article retrieval task, and the domain of the less suitable ones remains to be determined.

Broader Impacts
In addition to helping advance the state-of-theart in retrieving statutes relevant to a legal question, BSARD-based models could improve the efficiency of the legal information retrieval process in the context of legal research, therefore enabling researchers to devote themselves to more thoughtful parts of their research. Furthermore, BSARD can become a starting point of new open-source legal information search tools so that the socially weaker parties to disputes can benefit from a free professional assisting service. However, there are risks that the dataset will not be used exclusively for the public interest but perhaps also for profit as part of proprietary search tools developed by companies. Since this would reinforce rather than solve the problem of access to legal information and justice for all, we decided to distribute BSARD under a license with a noncommercial clause.
Other potential negative societal impacts could involve using models trained on BSARD to misuse or find gaps within the governmental laws or use the latter not to defend oneself but to deliberately damage people or companies instead. Of course, we discourage anyone from developing models that aim to perform the latter actions.

Conclusion
In this paper, we present the Belgian Statutory Article Retrieval Dataset (BSARD), a citizen-centric French native dataset for statutory article retrieval. Within a larger effort to bridge the gap between people and the law, BSARD provides a means of evaluating and developing models capable of retrieving law articles relevant to a legal question posed by a layperson. We benchmark several strong information retrieval baselines that show promise for the feasibility of the task yet indicate room for improvement. In the future, we plan to build retrieval models that can handle lengthy statutory articles and inherently exploit the hierarchy of the law. In closing, we hope that our work sparks interest in developing practical and reliable statutory article retrieval models to help improve access to justice for all.

B Evaluation Metrics
Let rel q (a) ∈ {0, 1} be the binary relevance label of article a for question q, and ⟨i, a⟩ ∈ F q a result tuple (article a at rank i) from the filter set F q ⊂ C of ranked articles retrieved for question q.
Recall. The recall R q is the fraction of relevant articles retrieved for query q w.r.t. the total number of relevant articles in the corpus C, i.e., Reciprocal rank. The reciprocal rank (RR q ) calculates the reciprocal of the rank at which the first relevant article is retrieved, i.e., Average precision. The average precision AP q is the mean of the precision value obtained after each relevant article is retrieved, that is where P q,j is the precision computed at rank j for query q, i.e., the fraction of relevant articles retrieved for query q w.r.t. the total number of articles in the retrieved set {F q } j i=1 : We report the macro-averaged recall (R), mean reciprocal rank (MRR), and mean average precision (MAP), which are the average values of the corresponding metrics over a set of n queries. Note that as those metrics are computed for a filter set of size k = |F q | ≪ |C| (and not on the entire list of articles in C), we report them with the suffix "@k".
C Dataset Documentation

C.1 Dataset Nutrition Labels
As a first way to document our dataset, we provide the dataset nutrition labels (Holland et al., 2018) for BSARD in Table 4.

C.2 Data Statement
In addition to the data nutrition labels, we include the data statement (Bender and Friedman, 2018) for BSARD, which provides detailed context on the dataset so that researchers, developers, and users can understand how models built upon it might generalize, be appropriately deployed, and potentially reflect bias or exclusion.
Curation rationale. All law articles from the selected Belgian codes were included in our dataset, except those revoked (identifiable because mentioned before the article or empty content) and those with a duplicate number within the same code (namely, the articles from Act V, Book III of the Civil Code; from Sections 2, 2bis, and 3 of Chapter II, Act VIII, Book III of the Civil Code; from Act XVIII, Book III of the Civil Code; from the Preliminary Act of the Code of Criminal Instruction; from the Appendix of the Judicial Code). Not including the latter articles did not pose a vital concern because none of them were mentioned as relevant to any of the questions in our dataset. Regarding the questions, all those that referenced at least one of the articles from our corpus were included in the dataset.
Language variety. The questions and legal articles were collected in French (fr-BE) as spoken in Wallonia and Brussels-Capital region.  Table 3: Summary of the number of articles collected (after pre-processing) from each of the Belgian codes considered for BSARD, as well as the number of articles found to be relevant for at least one of the legal questions.
Speaker demographic. Speakers were not directly approached for inclusion in this dataset and thus could not be asked for demographic information. Questions were collected, anonymized, and reformulated by Droits Quotidiens. Therefore, no direct information about the speakers' age and gender distribution or socioeconomic status is available. However, it is expected that most, but not all, of the speakers are adults (18+ years), speak French as a native language, and live in Wallonia or Brussels-Capital region.
Annotator demographic. A total of six Belgian jurists from Droits Quotidiens contributed to anno-tating the questions. All have a law degree from a Belgian university and years of experience in providing legal advice and clarifications of the law. They range in age from 30-60 years, including one man and five women, gave their ethnicity as white European, speak French as a native language, and represent upper middle class based on income levels.
Speech situation.  propriate language as they were all reviewed and reworded by Droits Quotidiens to be neutral, anonymous, and comprehensive. All the legal articles were written between 1804 and 2021 and collected in May 2021. They represent strong, formal, written language containing up to 5,790 words.
Text characteristics. Many articles complement or rely on other articles in the same or another code and thus contain (sometimes lengthy) legal references, which might be seen as noisy data.
Recording quality. N/A.

Other. N/A.
Provenance appendix. N/A.

C.3 Intended Uses
The dataset is intended to be used by researchers to build and evaluate models on retrieving law articles relevant to an input legal question. Therefore, it should not be regarded as a reliable source of legal information at this point in time, as both the questions and articles correspond to an outdated version of the Belgian law from May 2021 (time of dataset collection). In the latter case, the user is advised to consult daily updated official legal resources (e.g., the Belgian Official Gazette).

C.5 Data Format
The dataset is stored as CSV files and can be read using standard libraries (e.g., the built-in csv mod-ule in Python) or the datasets library: 1 from d a t a s e t s import l o a d _ d a t a s e t 2 d a t a = l o a d _ d a t a s e t ( " a n t o i l o u i / b s a r d " )

C.6 Reproducibility
We ensure the reproducibility of the experimental results by releasing our code on Github at https: //github.com/maastrichtlawtech/bsard.

C.7 Licensing
The dataset is publicly distributed under a CC BY-NC-SA 4.0 license, which allows sharing freely (i.e., copy and redistribute) and adapt (i.e., remix, transform, and build upon) the material on the conditions that the latter is used for non-commercial purposes only, proper attribution is given (i.e., appropriate credit, link to the license, and an indication of changes), and the same license as the original is used if one distributes an adapted version of the material. In addition, the code to reproduce the experimental results of the paper is released under the MIT license.

C.8 Maintenance
The dataset will be supported and maintained by the Law & Tech Lab at Maastricht University. Any updates to the dataset will be communicated via the Github repository. All questions and comments about the dataset can be sent to Antoine Louis: a.louis@maastrichtuniversity.nl.
Other contacts can be found at https://maastrichtuniversity.nl/ law-and-tech-people.