Update README.md
Browse files
README.md
CHANGED
|
@@ -44,7 +44,7 @@ mistral-chat $HOME/mistral_models/mamba-codestral-7B-v0.1 --instruct --max_toke
|
|
| 44 |
```
|
| 45 |
|
| 46 |
## Evaluation
|
| 47 |
-
We evaluate Codestral Mamba
|
| 48 |
| Benchmarks | HumanEval | MBPP | Spider | CruxE | HumanEval C++ | HumanEvalJava |HumanEvalJS |HumanEval Bash |
|
| 49 |
| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
|
| 50 |
| CodeGemma 1.1 7B | 61.0% | 67.7% | 46.3% | 50.4% | 49.1% | 41.8% | 52.2% | 9.4% |
|
|
|
|
| 44 |
```
|
| 45 |
|
| 46 |
## Evaluation
|
| 47 |
+
We evaluate Codestral Mamba and open-weight models of the similar size on industry-standard benchmarks.
|
| 48 |
| Benchmarks | HumanEval | MBPP | Spider | CruxE | HumanEval C++ | HumanEvalJava |HumanEvalJS |HumanEval Bash |
|
| 49 |
| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
|
| 50 |
| CodeGemma 1.1 7B | 61.0% | 67.7% | 46.3% | 50.4% | 49.1% | 41.8% | 52.2% | 9.4% |
|