whitphx HF Staff commited on
Commit
4c19328
·
verified ·
1 Parent(s): a3cfb90

Add/update the quantized ONNX model files and README.md for Transformers.js v3

Browse files

## Applied Quantizations

### ✅ Based on `decoder_model.onnx` *with* slimming

↳ ✅ `fp16`: `decoder_model_fp16.onnx` (added)
↳ ✅ `int8`: `decoder_model_int8.onnx` (added)
↳ ✅ `uint8`: `decoder_model_uint8.onnx` (added)
↳ ✅ `q4`: `decoder_model_q4.onnx` (added)
↳ ✅ `q4f16`: `decoder_model_q4f16.onnx` (added)
↳ ✅ `bnb4`: `decoder_model_bnb4.onnx` (added)

### ✅ Based on `decoder_model.onnx` *with* slimming

↳ ✅ `fp16`: `decoder_model_fp16.onnx` (added)
↳ ✅ `int8`: `decoder_model_int8.onnx` (added)
↳ ✅ `uint8`: `decoder_model_uint8.onnx` (added)
↳ ✅ `q4`: `decoder_model_q4.onnx` (added)
↳ ✅ `q4f16`: `decoder_model_q4f16.onnx` (added)
↳ ✅ `bnb4`: `decoder_model_bnb4.onnx` (added)

### ✅ Based on `encoder_model.onnx` *with* slimming

↳ ✅ `int8`: `encoder_model_int8.onnx` (added)
↳ ✅ `uint8`: `encoder_model_uint8.onnx` (added)
↳ ✅ `q4`: `encoder_model_q4.onnx` (added)
↳ ✅ `q4f16`: `encoder_model_q4f16.onnx` (added)
↳ ✅ `bnb4`: `encoder_model_bnb4.onnx` (added)

### ✅ Based on `encoder_model.onnx` *with* slimming

↳ ✅ `int8`: `encoder_model_int8.onnx` (added)
↳ ✅ `uint8`: `encoder_model_uint8.onnx` (added)
↳ ✅ `q4`: `encoder_model_q4.onnx` (added)
↳ ✅ `q4f16`: `encoder_model_q4f16.onnx` (added)
↳ ✅ `bnb4`: `encoder_model_bnb4.onnx` (added)

### ✅ Based on `decoder_with_past_model.onnx` *with* slimming

↳ ✅ `fp16`: `decoder_with_past_model_fp16.onnx` (added)
↳ ✅ `int8`: `decoder_with_past_model_int8.onnx` (added)
↳ ✅ `uint8`: `decoder_with_past_model_uint8.onnx` (added)
↳ ✅ `q4`: `decoder_with_past_model_q4.onnx` (added)
↳ ✅ `q4f16`: `decoder_with_past_model_q4f16.onnx` (added)
↳ ✅ `bnb4`: `decoder_with_past_model_bnb4.onnx` (added)

### ✅ Based on `decoder_with_past_model.onnx` *with* slimming

↳ ✅ `fp16`: `decoder_with_past_model_fp16.onnx` (added)
↳ ✅ `int8`: `decoder_with_past_model_int8.onnx` (added)
↳ ✅ `uint8`: `decoder_with_past_model_uint8.onnx` (added)
↳ ✅ `q4`: `decoder_with_past_model_q4.onnx` (added)
↳ ✅ `q4f16`: `decoder_with_past_model_q4f16.onnx` (added)
↳ ✅ `bnb4`: `decoder_with_past_model_bnb4.onnx` (added)

### ✅ Based on `decoder_model_merged.onnx` *with* slimming

↳ ✅ `fp16`: `decoder_model_merged_fp16.onnx` (replaced because it was invalid)
↳ ✅ `int8`: `decoder_model_merged_int8.onnx` (added)
↳ ✅ `uint8`: `decoder_model_merged_uint8.onnx` (added)
↳ ✅ `q4`: `decoder_model_merged_q4.onnx` (added)
↳ ✅ `q4f16`: `decoder_model_merged_q4f16.onnx` (added)
↳ ✅ `bnb4`: `decoder_model_merged_bnb4.onnx` (added)

### ✅ Based on `decoder_model_merged.onnx` *with* slimming

↳ ✅ `fp16`: `decoder_model_merged_fp16.onnx` (replaced because it was invalid)
↳ ✅ `int8`: `decoder_model_merged_int8.onnx` (added)
↳ ✅ `uint8`: `decoder_model_merged_uint8.onnx` (added)
↳ ✅ `q4`: `decoder_model_merged_q4.onnx` (added)
↳ ✅ `q4f16`: `decoder_model_merged_q4f16.onnx` (added)
↳ ✅ `bnb4`: `decoder_model_merged_bnb4.onnx` (added)

README.md CHANGED
@@ -6,4 +6,30 @@ pipeline_tag: summarization
6
 
7
  https://huggingface.co/facebook/bart-large-cnn with ONNX weights to be compatible with Transformers.js.
8
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
9
  Note: Having a separate repo for ONNX weights is intended to be a temporary solution until WebML gains more traction. If you would like to make your models web-ready, we recommend converting to ONNX using [🤗 Optimum](https://huggingface.co/docs/optimum/index) and structuring your repo like this one (with ONNX weights located in a subfolder named `onnx`).
 
6
 
7
  https://huggingface.co/facebook/bart-large-cnn with ONNX weights to be compatible with Transformers.js.
8
 
9
+ ## Usage (Transformers.js)
10
+
11
+ If you haven't already, you can install the [Transformers.js](https://huggingface.co/docs/transformers.js) JavaScript library from [NPM](https://www.npmjs.com/package/@huggingface/transformers) using:
12
+ ```bash
13
+ npm i @huggingface/transformers
14
+ ```
15
+
16
+ **Example:** Summarization.
17
+
18
+ ```js
19
+ import { pipeline } from '@huggingface/transformers';
20
+
21
+ const generator = await pipeline('summarization', 'Xenova/bart-large-cnn');
22
+ const text = 'The tower is 324 metres (1,063 ft) tall, about the same height as an 81-storey building, ' +
23
+ 'and the tallest structure in Paris. Its base is square, measuring 125 metres (410 ft) on each side. ' +
24
+ 'During its construction, the Eiffel Tower surpassed the Washington Monument to become the tallest ' +
25
+ 'man-made structure in the world, a title it held for 41 years until the Chrysler Building in New ' +
26
+ 'York City was finished in 1930. It was the first structure to reach a height of 300 metres. Due to ' +
27
+ 'the addition of a broadcasting aerial at the top of the tower in 1957, it is now taller than the ' +
28
+ 'Chrysler Building by 5.2 metres (17 ft). Excluding transmitters, the Eiffel Tower is the second ' +
29
+ 'tallest free-standing structure in France after the Millau Viaduct.';
30
+ const output = await generator(text, {
31
+ max_new_tokens: 100,
32
+ });
33
+ ```
34
+
35
  Note: Having a separate repo for ONNX weights is intended to be a temporary solution until WebML gains more traction. If you would like to make your models web-ready, we recommend converting to ONNX using [🤗 Optimum](https://huggingface.co/docs/optimum/index) and structuring your repo like this one (with ONNX weights located in a subfolder named `onnx`).
onnx/decoder_model_bnb4.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4242eb9ac1e946902d3846e0a660fb1e4470d84f7d0d67d7e5bf475a8fee0ba9
3
+ size 324819020
onnx/decoder_model_fp16.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4715d835ef87d8a0488ac052775039ac54fb6a3d8e8b6a2b8446107da618866e
3
+ size 508696917
onnx/decoder_model_int8.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4d1c3334ad28c9cf4a3d3fa86667407fe1f422a52c3dca32161c16b22714674e
3
+ size 255464148
onnx/decoder_model_merged_bnb4.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7acc932a4e446df67406356f4e33d60c74ec9420c1dba36a7cad4ca8a8263774
3
+ size 325603874
onnx/decoder_model_merged_fp16.onnx CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:379fa182aa6ddf9471d967d155c22e19d6c7e150a819c67710ee587f734d650c
3
- size 509661023
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:26d8b325a227b8c2fa94d0302c56359fbdd4e6d973ec8ffdcff68aa080fb73a3
3
+ size 509382753
onnx/decoder_model_merged_int8.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:224befbbc0d21e64f28fc6298e8c528d814da599899aba1d29bf0ed154dd28fb
3
+ size 256355663
onnx/decoder_model_merged_q4.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:14958a24d7ee062cab34b9f1dd24302e60e032895c4284da4445c5d638cddc34
3
+ size 338184950
onnx/decoder_model_merged_q4f16.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2ca5b682b7efb0b17117e50e8ad8b96973bf167579136a8ac2324b536230b446
3
+ size 220006815
onnx/decoder_model_merged_uint8.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:dd8bbf890cad418aad46062c3e8d0ecdcb0543623d9507ce275a5d7add9ab5ca
3
+ size 256355728
onnx/decoder_model_q4.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:010faa1ca3eb250ca07401026ad5f7a7b6b99ed63c94a92fcd4ce477b1f3df7f
3
+ size 337400960
onnx/decoder_model_q4f16.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:668942228b39e5973bce73a584fd8b603c4dc7f342eaa3e0ce088105874430cb
3
+ size 219307779
onnx/decoder_model_uint8.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:32d9f3129a52cafbc55a37c715d6728902381dc9a2722f8c78d5a7b6f9045659
3
+ size 255464213
onnx/decoder_with_past_model_bnb4.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c35ef9f8db16874a6e5d87f18f9d5123b143ce631345a00c0562f0640018ee04
3
+ size 310431387
onnx/decoder_with_past_model_fp16.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6bcad702a11763a8a05522b084d2e5047b1e7f395fba157131d18ad0e8e79303
3
+ size 458195908
onnx/decoder_with_past_model_int8.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6a45c877415c6a711cd577c20bcf5bb2e09ee3d392cce3b5eecab1aadd5271a4
3
+ size 230046616
onnx/decoder_with_past_model_q4.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bc20ee37f4944b4b1f9045cbf8961db799639462ba3d85f4d8b60ae232091f6a
3
+ size 321440655
onnx/decoder_with_past_model_q4f16.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cbf122686da9f34f4df2259c36eaef948c14ac291d615c50c5c73c1ecc583d51
3
+ size 204979090
onnx/decoder_with_past_model_uint8.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3848e0aef540a3bac35db0bacb83522b907d9c304c423caf58b018b5362d8f82
3
+ size 230046668
onnx/encoder_model_bnb4.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:69ae3940438ea72b4c1bdd0cdcccba9ed5af0a1846b6c98248f6e6d1ef23afb1
3
+ size 295912467
onnx/encoder_model_int8.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4ced420079c1949f45fc7b543cf9a8f5146f94d0154fbdf661a1c09e60f47c34
3
+ size 204472178
onnx/encoder_model_q4.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:992218a50dad66c795b13c1cc71541cfef2e98a8e40e627467a22e64b1a7102a
3
+ size 305349063
onnx/encoder_model_q4f16.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3c805036f49c3088ce619f4afe2dfaf3a8c6fd5a31ec6099f11fec415fb47f4c
3
+ size 190546742
onnx/encoder_model_uint8.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:812dadb73d91873548add060f8a569b7fc07fb93d55e328bacefe0331bc92904
3
+ size 204472217