license:unknown region:us
---
license: unknown
---
## Competitors
This Section shows how to **use** or **get** *ONNX Models* and *Younger Instances* in Competitors Benchmarks.


Competitors We Provide:
 - MLPerf v4.1
 - MLPerf v0.5
 - Phoronix pts/onnx-1.19.0



### Use ONNX Models Directly
This repository provides ONNX Models and Younger Instances about various Competitors.
The structure of the directory `competitors` is shown as below:

```
competitors/
│
├── mlperf_v4.1/
│   ├── README.md
│   ├── instances/
│   │   ├── instance_1/
│   │   ├── instance_2/
│   │   └── ... (more instances)
│   │
│   └── onnxs/
│       ├── onnx_1/
│       ├── onnx_2/
│       └── ... (more onnx models)
│
├── mlperf_v0.5/
│   └── ...
│
├── phoronix/
│   ├── instances/
│   │   ├── instance_1/
│   │   ├── instance_2/
│   │   └── ... (more instances)
│   │
│   └── onnxs/
│       ├── downloads.xml
│       ├── onnx_1/
│       ├── onnx_2/
│       └── ... (more onnx models)
└── ... (more competitors)
```

You can directly download these resources for your need.

1. Set shell env variable `BENCH_VER` as 'mlperf_v4.1', 'mlperf_v0.5', or 'phoronix'.
```
BENCH_VER=mlperf_v4.1
```

2. Run the commands:
```
younger benchmarks prepare --bench-dirpath ./${BENCH_VER}/onnxs/ --dataset-dirpath ./${BENCH_VER}/instances --version ${BENCH_VER} --direct 'both'
```


### How To Convert ONNX into Instance By Yourself

#### MLPerf

Set shell env variable `BENCH_VER` as 'mlperf_v4.1' or 'mlperf_v0.5'.

All ONNX link or conversion commands are placed in file `./${BENCH_VER}/README.md`

##### Manually
1. Go To [Official Website](https://github.com/mlcommons/inference)

2. Directly download all ONNX Models provided by MLCommons.

3. In version *MLPerf Inference v4.1*, **gpt-j** and **stable-diffusion-xl** should be converted through `optimum` command:
  - Stable Diffusion XL 1.0 [See](https://github.com/mlcommons/inference/tree/master/text_to_image)
  `python -m optimum.exporters.onnx -m stabilityai/stable-diffusion-xl-base-1.0 --monolith --task stable-diffusion-xl --cache_dir ./cache/hub ./stable-diffusion`
  - GPT-J 6B [See](https://github.com/mlcommons/inference/tree/master/language/gpt-j)
  `python -m optimum.exporters.onnx -m EleutherAI/gpt-j-6b --monolith  --cache_dir ./cache/hub ./gpt-j`

4. Simply use younger package to convert ONNX in to Instance:
```
from younger.datasets.modules import Instance
instance = Instance('/path/to/onnx/model')
```

##### Automatically

1. Run the command:
```
younger benchmarks prepare --bench-dirpath ./${BENCH_VER}/onnxs/ --dataset-dirpath ./${BENCH_VER}/instances --version ${BENCH_VER}
```

2. All ONNX Models will be downloaded into `./${BENCH_VER}/onnxs/`, and converted into Younger Instances and saved into `./${BENCH_VER}/instances`


#### Phoronix

##### Manually
1. Go To [Official Website](https://openbenchmarking.org/test/pts/onnx)

2. Find specific version of XML content in Section 'Revision History'.

3. Download all ONNX Models follow the link in XML.

4. Simply use younger package to convert ONNX in to Instance:
```
from younger.datasets.modules import Instance
instance = Instance('/path/to/onnx/model')
```

##### Automatically

1. Go To [Official Website](https://openbenchmarking.org/test/pts/onnx)

2. Copy specific version of XML content in Section 'Revision History'.

3. Place the XML content in `./phoronix/onnxs/downloads.xml`.

4. Run the command:
```
younger benchmarks prepare --bench-dirpath ./phoronix/onnxs/ --dataset-dirpath ./phoronix/instances --version phoronix
```

4. All ONNX Models will be downloaded into `./phoronix/onnxs/`, and converted into Younger Instances and saved into `./phoronix/instances`
