Update README.md (#1)

- Update README.md (b0911100145ef4499e79bc5f92e5842d881c7082)


Co-authored-by: mrh <[email protected]>

README.md

@@ -1,20 +1,179 @@
 ---
 license: apache-2.0
 ---
-# Step-Audio
 
-Step-Audio是StepFun开源的Step-Audio智能语音交互框架，Step-Audio框架内集成了语音识别、语义理解、对话管理、语音克隆和语音生成。这种统一的架构实现了低端到端延迟，可用于全双工交互，使其适用于实时应用。
+# 1. Step-Audio-Chat
 
-Step-Audio is an open-source intelligent voice interaction framework developed by StepFun. The Step-Audio framework integrates speech recognition, semantic understanding, dialogue management, voice cloning, and speech generation. This unified architecture achieves low end-to-end latency and can be used for full-duplex interaction, making it suitable for real-time applications.
+This repository contains the Multimodal Large Language Model (LLM) component of Step-Audio. It is a 130 billion parameter multimodal LLM that is responsible for understanding and generating human speech. The model is specifically designed to seamlessly integrate functions such as speech recognition, semantic understanding, dialogue management, voice cloning, and speech generation.
 
+## 2. Evaluation
+### 2.1 LLM judge metrics(GPT-4o) on [**StepEval-Audio-360**](https://huggingface.co/datasets/stepfun-ai/StepEval-Audio-360)
+<table>
+    <caption>Comparison of fundamental capabilities of voice chat on the StepEval-Audio-360.</caption>
+    <thead>
+        <tr>
+            <th>Model</th>
+            <th style="text-align:center">Factuality (% &uarr;)</th>
+            <th style="text-align:center">Relevance (% &uarr;)</th>
+            <th style="text-align:center">Chat Score &uarr;</th>
+        </tr>
+    </thead>
+    <tbody>
+        <tr>
+            <td>GLM4-Voice</td>
+            <td style="text-align:center">54.7</td>
+            <td style="text-align:center">66.4</td>
+            <td style="text-align:center">3.49</td>
+        </tr>
+        <tr>
+            <td>Qwen2-Audio</td>
+            <td style="text-align:center">22.6</td>
+            <td style="text-align:center">26.3</td>
+            <td style="text-align:center">2.27</td>
+        </tr>
+        <tr>
+            <td>Moshi<sup>*</sup></td>
+            <td style="text-align:center">1.0</td>
+            <td style="text-align:center">0</td>
+            <td style="text-align:center">1.49</td>
+        </tr>
+        <tr>
+            <td><strong>Step-Audio-Chat</strong></td>
+            <td style="text-align:center"><strong>66.4</strong></td>
+            <td style="text-align:center"><strong>75.2</strong></td>
+            <td style="text-align:center"><strong>4.11</strong></td>
+        </tr>
+    </tbody>
+</table>
 
-# Step-Audio-Chat
+*Note: Moshi are marked with "\*" and should be considered for reference only.
 
-本仓库是Step-Audio中的多模态大型语言模型(LLM)部分。它是一个 1300 亿参数的多模态大型语言模型 (LLM)，它负责理解和生成人类语音。 该模型经过专门设计，能够无缝集成语音识别、语义理解、对话管理、语音克隆和语音生成等功能。
 
-This repository contains the Multimodal Large Language Model (LLM) component of Step-Audio. It is a 130 billion parameter multimodal LLM that is responsible for understanding and generating human speech. The model is specifically designed to seamlessly integrate functions such as speech recognition, semantic understanding, dialogue management, voice cloning, and speech generation.
+### 2.2 Public Test Set
+
+<table>
+    <thead>
+        <tr>
+            <th>Model</th>
+            <th style="text-align:center">Llama Question</th>
+            <th style="text-align:center">Web Questions</th>
+            <th style="text-align:center">TriviaQA*</th>
+            <th style="text-align:center">ComplexBench</th>
+            <th style="text-align:center">HSK-6</th>
+        </tr>
+    </thead>
+    <tbody>
+        <tr>
+            <td>GLM4-Voice</td>
+            <td style="text-align:center">64.7</td>
+            <td style="text-align:center">32.2</td>
+            <td style="text-align:center">39.1</td>
+            <td style="text-align:center">66.0</td>
+            <td style="text-align:center">74.0</td>
+        </tr>
+        <tr>
+            <td>Moshi</td>
+            <td style="text-align:center">62.3</td>
+            <td style="text-align:center">26.6</td>
+            <td style="text-align:center">22.8</td>
+            <td style="text-align:center">-</td>
+            <td style="text-align:center">-</td>
+        </tr>
+        <tr>
+            <td>Freeze-Omni</td>
+            <td style="text-align:center">72.0</td>
+            <td style="text-align:center">44.7</td>
+            <td style="text-align:center">53.9</td>
+            <td style="text-align:center">-</td>
+            <td style="text-align:center">-</td>
+        </tr>
+        <tr>
+            <td>LUCY</td>
+            <td style="text-align:center">59.7</td>
+            <td style="text-align:center">29.3</td>
+            <td style="text-align:center">27.0</td>
+            <td style="text-align:center">-</td>
+            <td style="text-align:center">-</td>
+        </tr>
+        <tr>
+            <td>MinMo</td>
+            <td style="text-align:center">78.9</td>
+            <td style="text-align:center">55.0</td>
+            <td style="text-align:center">48.3</td>
+            <td style="text-align:center">-</td>
+            <td style="text-align:center">-</td>
+        </tr>
+        <tr>
+            <td>Qwen2-Audio</td>
+            <td style="text-align:center">52.0</td>
+            <td style="text-align:center">27.0</td>
+            <td style="text-align:center">37.3</td>
+            <td style="text-align:center">54.0</td>
+            <td style="text-align:center">-</td>
+        </tr>
+        <tr>
+            <td><strong>Step-Audio-Chat</strong></td>
+            <td style="text-align:center"><strong><i>81.0</i></strong></td>
+            <td style="text-align:center"><strong>75.1</strong></td>
+            <td style="text-align:center"><strong>58.0</strong></td>
+            <td style="text-align:center"><strong>74.0</strong></td>
+            <td style="text-align:center"><strong>86.0</strong></td>
+        </tr>
+    </tbody>
+</table>
+
+*Note: Results marked with "\*" on TriviaQA dataset are considered for reference only.*
+
+*TriviaQA dataset marked with "\*" indicates results are for reference only.*
+
+### 2.3 Audio instruction following
+<table>
+    <thead>
+        <tr>
+            <th rowspan="2">Category</th>
+            <th colspan="2" style="text-align:center">Instruction Following</th>
+            <th colspan="2" style="text-align:center">Audio Quality</th>
+        </tr>
+        <tr>
+            <th style="text-align:center">GLM-4-Voice</th>
+            <th style="text-align:center">Step-Audio</th>
+            <th style="text-align:center">GLM-4-Voice</th>
+            <th style="text-align:center">Step-Audio</th>
+        </tr>
+    </thead>
+    <tbody>
+        <tr>
+            <td>Languages</td>
+            <td style="text-align:center">1.9</td>
+            <td style="text-align:center">3.8</td>
+            <td style="text-align:center">2.9</td>
+            <td style="text-align:center">3.3</td>
+        </tr>
+        <tr>
+            <td>Role-playing</td>
+            <td style="text-align:center">3.8</td>
+            <td style="text-align:center">4.2</td>
+            <td style="text-align:center">3.2</td>
+            <td style="text-align:center">3.6</td>
+        </tr>
+        <tr>
+            <td>Singing / RAP</td>
+            <td style="text-align:center">2.1</td>
+            <td style="text-align:center">2.4</td>
+            <td style="text-align:center">2.4</td>
+            <td style="text-align:center">4</td>
+        </tr>
+        <tr>
+            <td>Voice Control</td>
+            <td style="text-align:center">3.6</td>
+            <td style="text-align:center">4.4</td>
+            <td style="text-align:center">3.3</td>
+            <td style="text-align:center">4.1</td>
+        </tr>
+    </tbody>
+</table>
 
 
-更多信息请参考我们的仓库： [Step-Audio](https://github.com/stepfun-ai/Step-Audio).
+## 3. More information
 
 For more information, please refer to our repository: [Step-Audio](https://github.com/stepfun-ai/Step-Audio).