license:cc-by-4.0 region:us
---
license: cc-by-4.0
---
With the rapid development of large language models (LLMs), the interest in the recognition of printed mathematical expressions has increased dramatically. Researchers have gradually realized that there is rich knowledge information in document data, which is crucial to further improve the performance of LLMs. Therefore, some researchers have started to investigate how to extract document information.
LLMs for solving mathematical reasoning tasks in printed academic papers are a cutting-edge research area. The recognition of printed mathematical expressions is the foundation of this field, where multi-line mathematical expressions occupy a large proportion. However, current benchmark datasets such as IM2LATEX-100K and CROHME contain only a small number of multiline mathematical expressions for evaluation, which can lead to misleading assessments of their performance, and extreme reductions in the number of multiline expressions can reduce the difficulty of the task and lead to invalid and inaccurate recognition results.
Therefore, in order to improve the accuracy of the assessment of the recognition performance of printed mathematical expressions, it is important to expand the datasets of multiline mathematical expressions and to ensure that these datasets contain sufficient complexity and diversity. This will not only help to evaluate the performance of the model more comprehensively, but also promote research progress in the field.
In addition, although the proposal of benchmark datasets has greatly accelerated the progress of mathematical formula recognition research, the following problems inevitably exist:

随着大型语言模型（LLMs）的迅猛发展，人们对印刷数学表达式识别的关注度急剧增加。研究人员逐渐意识到文档数据中存在丰富的知识信息，这对于进一步提高大型模型的性能至关重要。因此，一些研究人员开始研究如何提取文档信息。
在印刷学术论文中解决数学推理任务的LLMs是一个前沿的研究领域。对印刷数学表达式的识别是该领域的基础，其中多行数学表达式占据了很大比例。然而，目前的IM2LATEX-100K和CROHME等基准数据集仅包含少量多行数学表达式的数据进行评估，这可能会导致对其性能的误导性评估，多行表达式数量的极端减少会降低任务的难度，导致无效和不准确的识别结果。
因此，为了提高对印刷数学表达式的识别性能评估的准确性，必须扩大多行数学表达式的数据集，并确保这些数据集包含足够的复杂性和多样性。这不仅有助于更全面地评估模型的性能，还能推动该领域的研究进展。
此外，尽管基准数据集的提出大大加速了数学公式识别研究进度，但不可避免的存在以下问题：


(1) existing formula datasets, which mainly consist of rendered and handwritten formulas. These datasets have limitations in formula length and complexity, for example, IM2LATEX-100K mainly contains regular formulas and lacks long formulas that are extremely short or have complex spatial structures, and secondly, datasets such as IM2LATEX-100K have not gone through a rigorous data cleansing session, which leads to the presence of \hspace and \vspace spatial layout characters in them.

（1）现有的公式数据集，主要由渲染和手写的公式组成。这些数据集在公式长度和复杂性上存在局限性，例如，IM2LATEX-100K主要包含规则公式，缺少极短或空间结构复杂的长公式，其次IM2LATEX-100K等数据集未经过严格的数据清洗环节，导致其中存在\hspace和\vspace空间布局字符。

(2) Problems with Latex tags using space characters as separators: the vast majority of datasets use space characters as separators, which can be problematic in some cases, for example:
 $$f(x)=x^2 \quad \text{Where x is a real number}$$.
Splitting with spaces generates the label \text {W h e r e t h e r e i s a w i l l , t h e r e i s a w a y } which is rendered as follows:
 $$f ( x ) = x ^ 2 \quad \text { W h e r e x i s a r e a l n u m b e r }$$.
On the one hand this format does not match the expected output. In addition, the space character is one of the necessary elements in the label.
** With the help of the Maximum Positive Matching algorithm commonly used in Natural Language Processing may be a viable option, not based on space segmentation. **

In addition, another common mistake is that the use of space segmentation in tags may lead to rendering failures. For example, the $X '^ 2$ tag should have a corresponding label of **X ' ^ 2** if it is split by spaces, but its annotation result does not conform to the Latex specification, leading to rendering failure. **How to ensure a strict one-to-one correspondence between the image and the label sequence is the key to solving the above problem. **


（2）Latex 标签使用空格符作为分隔符的问题：绝大多数数据集使用空格符作为分隔符，在某些情况下会存在问题，例如：
 $$f(x)=x^2 \quad \text{Where x is a real number}$$
使用空格进行分割会生成\text {W h e r e t h e r e i s a w i l l , t h e r e i s a w a y } 的标签，而这种标签的渲染结果如下：
 $$f ( x ) = x ^ 2 \quad \text { W h e r e x i s a r e a l n u m b e r }$$
一方面这种格式与预期的输出不符。此外，空格符也是标签中的必要元素之一。
**借助自然语言处理中常用的最大正向匹配算法可能是一种可行方案，不依据空格分割。**

此外，另一种常见的错误是，使用空格分割标签可能导致渲染失败。例如$X '^ 2$标签如果用空格分割，其对应标签应当为**X ' ^ 2**，但其标注结果不符合Latex规范，导致渲染失败。 **如何确保图像与标签序列严格一一对应关系是解决上述问题的关键。**

(3) Since the same typographic mathematical formula can be labeled with different latex styles and is not a one-to-one mapping relationship, e.g., $x_y^m$, which can be labeled using x_y^m or x_{y}^{m}, it is obvious that labeling using a uniform LaTeX code style is necessary, which is equivalent to standardizing the sequence.

（3）由于同一印刷体数学公式可以用不同的latex风格进行标注，并不是一种一对一的映射关系，例如$x_y^m$，可以使用x_y^m，也可以使用x_{y}^{m}进行标注，显然使用统一的LaTeX代码风格进行标记是有必要的，这相当于标准化序列。

In order to solve the above problem, we constructed a 150,000 high-quality mathematical formula dataset, and all mathematical expressions in our dataset are collected from real-world academic papers** and undergo a rigorous data cleaning process, and are annotated using a unified annotation style, which ensures that only a small number of tokens can be used to cover the vast majority of mathematical application scenarios**, In addition, **In order to solve the multi-line In addition, **in order to solve the problem of lack of multi-line math formula dataset, we purposely focus on multi-line math expressions, so that 30% of the expressions in the dataset have multiple lines. The complexity of the spatial structure is also further enhanced compared to benchmark datasets such as IM2LATEX** To further ensure that the truth labels are 100% accurate, we use these labels to generate images of the mathematical expressions. We then constructed our proposed dataset based on these rendered images.

为了解决上述问题，我们构造了15万高质量数学公式数据集，我们数据集中的所有数学表达式都是从现实世界的学术论文中收集的，**并经过严格的数据清洗，使用统一的标注风格进行标注，确保了仅使用少量token就能覆盖绝大部分数学应用场景**，此外，**为了解决多行数学公式数据集匮乏问题，我们特意将重点放在多行数学表达式上，使得数据集中 30% 的表达式具有多行。且相较于IM2LATEX等基准数据集， 空间结构复杂程度也有了进一步的加强**，为了进一步确保真值标签 100% 准确，我们使用这些标签来生成数学表达式图像。然后，我们根据这些渲染的图像构建了我们提议的数据集。