Marco127 commited on
Commit
b2e0c60
·
verified ·
1 Parent(s): e7fcd35

Add new SentenceTransformer model

Browse files
1_Pooling/config.json ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "word_embedding_dimension": 768,
3
+ "pooling_mode_cls_token": true,
4
+ "pooling_mode_mean_tokens": false,
5
+ "pooling_mode_max_tokens": false,
6
+ "pooling_mode_mean_sqrt_len_tokens": false,
7
+ "pooling_mode_weightedmean_tokens": false,
8
+ "pooling_mode_lasttoken": false,
9
+ "include_prompt": true
10
+ }
README.md ADDED
@@ -0,0 +1,569 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ tags:
3
+ - sentence-transformers
4
+ - sentence-similarity
5
+ - feature-extraction
6
+ - generated_from_trainer
7
+ - dataset_size:3362
8
+ - loss:MultipleNegativesRankingLoss
9
+ base_model: sentence-transformers/multi-qa-mpnet-base-dot-v1
10
+ widget:
11
+ - source_sentence: '
12
+
13
+ Guests are responsible for damages caused to hotel property according to the valid
14
+ legal
15
+
16
+ prescriptions of Hungary.'
17
+ sentences:
18
+ - '
19
+
20
+ Guests are responsible for damages caused to hotel property according to the valid
21
+ legal
22
+
23
+ prescriptions of Hungary.'
24
+ - '
25
+
26
+ We request that guests report any complaints and defects to the hotel reception
27
+ or hotel
28
+
29
+ management in person. Your complaints shall be attended to immediately.'
30
+ - '
31
+
32
+ We do not guarantee that any special requests will be met, but we will use our
33
+ best endeavours to do so as
34
+
35
+ well as using our best endeavours to advise you if that is not the case.'
36
+ - source_sentence: '
37
+
38
+ If we must cancel the reservation due to circumstances beyond our control, the
39
+ entire payment will be
40
+
41
+ refunded to you without any further obligation on our part and you will have no
42
+ further recourse against us.'
43
+ sentences:
44
+ - '
45
+
46
+ We do not guarantee that any special requests will be met, but we will use our
47
+ best endeavours to do so as
48
+
49
+ well as using our best endeavours to advise you if that is not the case.'
50
+ - '
51
+
52
+ A hotel guest may not leave the room to another person, even if the time for which
53
+ he or she has paid has
54
+
55
+ not expired.'
56
+ - '
57
+
58
+ If we must cancel the reservation due to circumstances beyond our control, the
59
+ entire payment will be
60
+
61
+ refunded to you without any further obligation on our part and you will have no
62
+ further recourse against us.'
63
+ - source_sentence: 'For safety reasons it is not permitted to leave children under
64
+ 12 years of age in hotel
65
+
66
+ rooms and other common areas of the hotel without adult supervision, and children
67
+ under
68
+
69
+ 12 years of age may not use the lift without supervision.'
70
+ sentences:
71
+ - 'For safety reasons it is not permitted to leave children under 12 years of age
72
+ in hotel
73
+
74
+ rooms and other common areas of the hotel without adult supervision, and children
75
+ under
76
+
77
+ 12 years of age may not use the lift without supervision.'
78
+ - '
79
+
80
+ I accept personal responsibility for payment of all amounts arising from my party''s
81
+ stay at the Hotel.
82
+
83
+ I/we are obligated to vacate my/our room/s at the designated check-out time, unless
84
+ I have made prior
85
+
86
+ alternative check-out arrangements with the management of the Hotel. My/our failure
87
+ to do so will result in
88
+
89
+ my liability for the costs of an additional night''s accommodation.'
90
+ - '
91
+
92
+ Elevators are to be used for the sole purpose of transporting guests and their
93
+ luggage to the appropriate
94
+
95
+ floor of the hotel. Misuse and horseplay will not be allowed.'
96
+ - source_sentence: '
97
+
98
+ Accommodation in the hotel is permitted only to persons who are not carrying infectious
99
+
100
+ diseases and who are not visibly under the influence of alcohol or drugs.'
101
+ sentences:
102
+ - '
103
+
104
+ Animals may not be allowed onto beds or other furniture, which serves for
105
+
106
+ guests. It is not permitted to use baths, showers or washbasins for bathing or
107
+
108
+ washing animals.'
109
+ - '
110
+
111
+ Accommodation in the hotel is permitted only to persons who are not carrying infectious
112
+
113
+ diseases and who are not visibly under the influence of alcohol or drugs.'
114
+ - '
115
+
116
+ The pets can not be left without supervision if there is a risk of causing any
117
+
118
+ damage or might disturb other guests.'
119
+ - source_sentence: '
120
+
121
+ A hotel guest may not leave the room to another person, even if the time for which
122
+ he or she has paid has
123
+
124
+ not expired.'
125
+ sentences:
126
+ - '
127
+
128
+ A hotel guest may not leave the room to another person, even if the time for which
129
+ he or she has paid has
130
+
131
+ not expired.'
132
+ - '
133
+
134
+ There is no running, shouting, roughhousing or horseplay accepted while on the
135
+ hotel property. This
136
+
137
+ includes hallways, lobby areas, stairways, elevators, food service areas and guest
138
+ rooms.'
139
+ - 'Orders for accommodation services made in writing or by other means, which have
140
+ been
141
+
142
+ confirmed by the hotel and have not been cancelled by the customer in a timely
143
+ manner, are
144
+
145
+ mutually binding. The front office manager keeps a record of all received and
146
+ confirmed
147
+
148
+ orders.'
149
+ pipeline_tag: sentence-similarity
150
+ library_name: sentence-transformers
151
+ metrics:
152
+ - dot_accuracy
153
+ - dot_accuracy_threshold
154
+ - dot_f1
155
+ - dot_f1_threshold
156
+ - dot_precision
157
+ - dot_recall
158
+ - dot_ap
159
+ - dot_mcc
160
+ model-index:
161
+ - name: SentenceTransformer based on sentence-transformers/multi-qa-mpnet-base-dot-v1
162
+ results:
163
+ - task:
164
+ type: binary-classification
165
+ name: Binary Classification
166
+ dataset:
167
+ name: Unknown
168
+ type: unknown
169
+ metrics:
170
+ - type: dot_accuracy
171
+ value: 0.667063020214031
172
+ name: Dot Accuracy
173
+ - type: dot_accuracy_threshold
174
+ value: 48.93047332763672
175
+ name: Dot Accuracy Threshold
176
+ - type: dot_f1
177
+ value: 0.49865951742627346
178
+ name: Dot F1
179
+ - type: dot_f1_threshold
180
+ value: 33.95234298706055
181
+ name: Dot F1 Threshold
182
+ - type: dot_precision
183
+ value: 0.33253873659118
184
+ name: Dot Precision
185
+ - type: dot_recall
186
+ value: 0.9964285714285714
187
+ name: Dot Recall
188
+ - type: dot_ap
189
+ value: 0.31258772254817324
190
+ name: Dot Ap
191
+ - type: dot_mcc
192
+ value: 0.0
193
+ name: Dot Mcc
194
+ ---
195
+
196
+ # SentenceTransformer based on sentence-transformers/multi-qa-mpnet-base-dot-v1
197
+
198
+ This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [sentence-transformers/multi-qa-mpnet-base-dot-v1](https://huggingface.co/sentence-transformers/multi-qa-mpnet-base-dot-v1). It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
199
+
200
+ ## Model Details
201
+
202
+ ### Model Description
203
+ - **Model Type:** Sentence Transformer
204
+ - **Base model:** [sentence-transformers/multi-qa-mpnet-base-dot-v1](https://huggingface.co/sentence-transformers/multi-qa-mpnet-base-dot-v1) <!-- at revision 4633e80e17ea975bc090c97b049da26062b054d3 -->
205
+ - **Maximum Sequence Length:** 512 tokens
206
+ - **Output Dimensionality:** 768 dimensions
207
+ - **Similarity Function:** Dot Product
208
+ <!-- - **Training Dataset:** Unknown -->
209
+ <!-- - **Language:** Unknown -->
210
+ <!-- - **License:** Unknown -->
211
+
212
+ ### Model Sources
213
+
214
+ - **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
215
+ - **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
216
+ - **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
217
+
218
+ ### Full Model Architecture
219
+
220
+ ```
221
+ SentenceTransformer(
222
+ (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: MPNetModel
223
+ (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
224
+ )
225
+ ```
226
+
227
+ ## Usage
228
+
229
+ ### Direct Usage (Sentence Transformers)
230
+
231
+ First install the Sentence Transformers library:
232
+
233
+ ```bash
234
+ pip install -U sentence-transformers
235
+ ```
236
+
237
+ Then you can load this model and run inference.
238
+ ```python
239
+ from sentence_transformers import SentenceTransformer
240
+
241
+ # Download from the 🤗 Hub
242
+ model = SentenceTransformer("Marco127/Base_Test1_")
243
+ # Run inference
244
+ sentences = [
245
+ '\nA hotel guest may not leave the room to another person, even if the time for which he or she has paid has\nnot expired.',
246
+ '\nA hotel guest may not leave the room to another person, even if the time for which he or she has paid has\nnot expired.',
247
+ 'Orders for accommodation services made in writing or by other means, which have been\nconfirmed by the hotel and have not been cancelled by the customer in a timely manner, are\nmutually binding. The front office manager keeps a record of all received and confirmed\norders.',
248
+ ]
249
+ embeddings = model.encode(sentences)
250
+ print(embeddings.shape)
251
+ # [3, 768]
252
+
253
+ # Get the similarity scores for the embeddings
254
+ similarities = model.similarity(embeddings, embeddings)
255
+ print(similarities.shape)
256
+ # [3, 3]
257
+ ```
258
+
259
+ <!--
260
+ ### Direct Usage (Transformers)
261
+
262
+ <details><summary>Click to see the direct usage in Transformers</summary>
263
+
264
+ </details>
265
+ -->
266
+
267
+ <!--
268
+ ### Downstream Usage (Sentence Transformers)
269
+
270
+ You can finetune this model on your own dataset.
271
+
272
+ <details><summary>Click to expand</summary>
273
+
274
+ </details>
275
+ -->
276
+
277
+ <!--
278
+ ### Out-of-Scope Use
279
+
280
+ *List how the model may foreseeably be misused and address what users ought not to do with the model.*
281
+ -->
282
+
283
+ ## Evaluation
284
+
285
+ ### Metrics
286
+
287
+ #### Binary Classification
288
+
289
+ * Evaluated with [<code>BinaryClassificationEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.BinaryClassificationEvaluator)
290
+
291
+ | Metric | Value |
292
+ |:-----------------------|:-----------|
293
+ | dot_accuracy | 0.6671 |
294
+ | dot_accuracy_threshold | 48.9305 |
295
+ | dot_f1 | 0.4987 |
296
+ | dot_f1_threshold | 33.9523 |
297
+ | dot_precision | 0.3325 |
298
+ | dot_recall | 0.9964 |
299
+ | **dot_ap** | **0.3126** |
300
+ | dot_mcc | 0.0 |
301
+
302
+ <!--
303
+ ## Bias, Risks and Limitations
304
+
305
+ *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
306
+ -->
307
+
308
+ <!--
309
+ ### Recommendations
310
+
311
+ *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
312
+ -->
313
+
314
+ ## Training Details
315
+
316
+ ### Training Dataset
317
+
318
+ #### Unnamed Dataset
319
+
320
+ * Size: 3,362 training samples
321
+ * Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>label</code>
322
+ * Approximate statistics based on the first 1000 samples:
323
+ | | sentence1 | sentence2 | label |
324
+ |:--------|:------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|:------------------------------------------------|
325
+ | type | string | string | int |
326
+ | details | <ul><li>min: 11 tokens</li><li>mean: 48.75 tokens</li><li>max: 156 tokens</li></ul> | <ul><li>min: 11 tokens</li><li>mean: 48.75 tokens</li><li>max: 156 tokens</li></ul> | <ul><li>0: ~69.20%</li><li>1: ~30.80%</li></ul> |
327
+ * Samples:
328
+ | sentence1 | sentence2 | label |
329
+ |:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------|
330
+ | <code>Hotel guests may receive visits in their hotel rooms from guests not staying in the hotel.<br>Visitors must present a personal document at the hotel reception and register in the visitors'<br>book. These visits can last for only a maximum of 2 hours and must finish until 10:00 pm.</code> | <code>Hotel guests may receive visits in their hotel rooms from guests not staying in the hotel.<br>Visitors must present a personal document at the hotel reception and register in the visitors'<br>book. These visits can last for only a maximum of 2 hours and must finish until 10:00 pm.</code> | <code>0</code> |
331
+ | <code><br>We do not guarantee that any special requests will be met, but we will use our best endeavours to do so as<br>well as using our best endeavours to advise you if that is not the case.</code> | <code><br>We do not guarantee that any special requests will be met, but we will use our best endeavours to do so as<br>well as using our best endeavours to advise you if that is not the case.</code> | <code>0</code> |
332
+ | <code><br>Pool and Fitness Room hours and guidelines are provided at check in. All rules and times will be enforced to<br>allow efficient operation of the hotel and for the comfort and safety of all guests.</code> | <code><br>Pool and Fitness Room hours and guidelines are provided at check in. All rules and times will be enforced to<br>allow efficient operation of the hotel and for the comfort and safety of all guests.</code> | <code>1</code> |
333
+ * Loss: [<code>MultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters:
334
+ ```json
335
+ {
336
+ "scale": 20.0,
337
+ "similarity_fct": "cos_sim"
338
+ }
339
+ ```
340
+
341
+ ### Evaluation Dataset
342
+
343
+ #### Unnamed Dataset
344
+
345
+ * Size: 841 evaluation samples
346
+ * Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>label</code>
347
+ * Approximate statistics based on the first 841 samples:
348
+ | | sentence1 | sentence2 | label |
349
+ |:--------|:-----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|:------------------------------------------------|
350
+ | type | string | string | int |
351
+ | details | <ul><li>min: 11 tokens</li><li>mean: 48.1 tokens</li><li>max: 156 tokens</li></ul> | <ul><li>min: 11 tokens</li><li>mean: 48.1 tokens</li><li>max: 156 tokens</li></ul> | <ul><li>0: ~66.71%</li><li>1: ~33.29%</li></ul> |
352
+ * Samples:
353
+ | sentence1 | sentence2 | label |
354
+ |:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------|
355
+ | <code>In the case of fire, guests are obliged to notify the reception without hesitation, either<br>directly, or on the phone (0) and may use a portable fire extinguisher located at the corridors<br>of each floor to extinguish the flames. The use of the elevator in case of fire is prohibited!</code> | <code>In the case of fire, guests are obliged to notify the reception without hesitation, either<br>directly, or on the phone (0) and may use a portable fire extinguisher located at the corridors<br>of each floor to extinguish the flames. The use of the elevator in case of fire is prohibited!</code> | <code>0</code> |
356
+ | <code><br>Children should be accompanied in locations such as stairways etc.<br> The rooms are for accommodation service. Each individual staying in a room<br>must be registered at the reception.</code> | <code><br>Children should be accompanied in locations such as stairways etc.<br> The rooms are for accommodation service. Each individual staying in a room<br>must be registered at the reception.</code> | <code>0</code> |
357
+ | <code><br>Towels for the Fitness Room and Pool are located in those areas. Towels from guest rooms are not to be<br>taken to the Pool or Fitness Room.</code> | <code><br>Towels for the Fitness Room and Pool are located in those areas. Towels from guest rooms are not to be<br>taken to the Pool or Fitness Room.</code> | <code>0</code> |
358
+ * Loss: [<code>MultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters:
359
+ ```json
360
+ {
361
+ "scale": 20.0,
362
+ "similarity_fct": "cos_sim"
363
+ }
364
+ ```
365
+
366
+ ### Training Hyperparameters
367
+ #### Non-Default Hyperparameters
368
+
369
+ - `eval_strategy`: steps
370
+ - `per_device_train_batch_size`: 16
371
+ - `per_device_eval_batch_size`: 16
372
+ - `learning_rate`: 2e-05
373
+ - `num_train_epochs`: 5
374
+ - `warmup_ratio`: 0.1
375
+ - `fp16`: True
376
+ - `batch_sampler`: no_duplicates
377
+
378
+ #### All Hyperparameters
379
+ <details><summary>Click to expand</summary>
380
+
381
+ - `overwrite_output_dir`: False
382
+ - `do_predict`: False
383
+ - `eval_strategy`: steps
384
+ - `prediction_loss_only`: True
385
+ - `per_device_train_batch_size`: 16
386
+ - `per_device_eval_batch_size`: 16
387
+ - `per_gpu_train_batch_size`: None
388
+ - `per_gpu_eval_batch_size`: None
389
+ - `gradient_accumulation_steps`: 1
390
+ - `eval_accumulation_steps`: None
391
+ - `torch_empty_cache_steps`: None
392
+ - `learning_rate`: 2e-05
393
+ - `weight_decay`: 0.0
394
+ - `adam_beta1`: 0.9
395
+ - `adam_beta2`: 0.999
396
+ - `adam_epsilon`: 1e-08
397
+ - `max_grad_norm`: 1.0
398
+ - `num_train_epochs`: 5
399
+ - `max_steps`: -1
400
+ - `lr_scheduler_type`: linear
401
+ - `lr_scheduler_kwargs`: {}
402
+ - `warmup_ratio`: 0.1
403
+ - `warmup_steps`: 0
404
+ - `log_level`: passive
405
+ - `log_level_replica`: warning
406
+ - `log_on_each_node`: True
407
+ - `logging_nan_inf_filter`: True
408
+ - `save_safetensors`: True
409
+ - `save_on_each_node`: False
410
+ - `save_only_model`: False
411
+ - `restore_callback_states_from_checkpoint`: False
412
+ - `no_cuda`: False
413
+ - `use_cpu`: False
414
+ - `use_mps_device`: False
415
+ - `seed`: 42
416
+ - `data_seed`: None
417
+ - `jit_mode_eval`: False
418
+ - `use_ipex`: False
419
+ - `bf16`: False
420
+ - `fp16`: True
421
+ - `fp16_opt_level`: O1
422
+ - `half_precision_backend`: auto
423
+ - `bf16_full_eval`: False
424
+ - `fp16_full_eval`: False
425
+ - `tf32`: None
426
+ - `local_rank`: 0
427
+ - `ddp_backend`: None
428
+ - `tpu_num_cores`: None
429
+ - `tpu_metrics_debug`: False
430
+ - `debug`: []
431
+ - `dataloader_drop_last`: False
432
+ - `dataloader_num_workers`: 0
433
+ - `dataloader_prefetch_factor`: None
434
+ - `past_index`: -1
435
+ - `disable_tqdm`: False
436
+ - `remove_unused_columns`: True
437
+ - `label_names`: None
438
+ - `load_best_model_at_end`: False
439
+ - `ignore_data_skip`: False
440
+ - `fsdp`: []
441
+ - `fsdp_min_num_params`: 0
442
+ - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
443
+ - `fsdp_transformer_layer_cls_to_wrap`: None
444
+ - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
445
+ - `deepspeed`: None
446
+ - `label_smoothing_factor`: 0.0
447
+ - `optim`: adamw_torch
448
+ - `optim_args`: None
449
+ - `adafactor`: False
450
+ - `group_by_length`: False
451
+ - `length_column_name`: length
452
+ - `ddp_find_unused_parameters`: None
453
+ - `ddp_bucket_cap_mb`: None
454
+ - `ddp_broadcast_buffers`: False
455
+ - `dataloader_pin_memory`: True
456
+ - `dataloader_persistent_workers`: False
457
+ - `skip_memory_metrics`: True
458
+ - `use_legacy_prediction_loop`: False
459
+ - `push_to_hub`: False
460
+ - `resume_from_checkpoint`: None
461
+ - `hub_model_id`: None
462
+ - `hub_strategy`: every_save
463
+ - `hub_private_repo`: None
464
+ - `hub_always_push`: False
465
+ - `gradient_checkpointing`: False
466
+ - `gradient_checkpointing_kwargs`: None
467
+ - `include_inputs_for_metrics`: False
468
+ - `include_for_metrics`: []
469
+ - `eval_do_concat_batches`: True
470
+ - `fp16_backend`: auto
471
+ - `push_to_hub_model_id`: None
472
+ - `push_to_hub_organization`: None
473
+ - `mp_parameters`:
474
+ - `auto_find_batch_size`: False
475
+ - `full_determinism`: False
476
+ - `torchdynamo`: None
477
+ - `ray_scope`: last
478
+ - `ddp_timeout`: 1800
479
+ - `torch_compile`: False
480
+ - `torch_compile_backend`: None
481
+ - `torch_compile_mode`: None
482
+ - `dispatch_batches`: None
483
+ - `split_batches`: None
484
+ - `include_tokens_per_second`: False
485
+ - `include_num_input_tokens_seen`: False
486
+ - `neftune_noise_alpha`: None
487
+ - `optim_target_modules`: None
488
+ - `batch_eval_metrics`: False
489
+ - `eval_on_start`: False
490
+ - `use_liger_kernel`: False
491
+ - `eval_use_gather_object`: False
492
+ - `average_tokens_across_devices`: False
493
+ - `prompts`: None
494
+ - `batch_sampler`: no_duplicates
495
+ - `multi_dataset_batch_sampler`: proportional
496
+
497
+ </details>
498
+
499
+ ### Training Logs
500
+ | Epoch | Step | Training Loss | Validation Loss | dot_ap |
501
+ |:------:|:----:|:-------------:|:---------------:|:------:|
502
+ | -1 | -1 | - | - | 0.3126 |
503
+ | 0.4739 | 100 | 0.0011 | 0.0001 | - |
504
+ | 0.9479 | 200 | 0.0002 | 0.0000 | - |
505
+ | 1.4218 | 300 | 0.0 | 0.0000 | - |
506
+ | 1.8957 | 400 | 0.0001 | 0.0000 | - |
507
+ | 2.3697 | 500 | 0.0 | 0.0000 | - |
508
+ | 2.8436 | 600 | 0.0 | 0.0000 | - |
509
+ | 3.3175 | 700 | 0.0 | 0.0000 | - |
510
+ | 3.7915 | 800 | 0.0 | 0.0000 | - |
511
+ | 4.2654 | 900 | 0.0 | 0.0000 | - |
512
+ | 4.7393 | 1000 | 0.0 | 0.0000 | - |
513
+
514
+
515
+ ### Framework Versions
516
+ - Python: 3.11.11
517
+ - Sentence Transformers: 3.4.1
518
+ - Transformers: 4.48.3
519
+ - PyTorch: 2.5.1+cu124
520
+ - Accelerate: 1.3.0
521
+ - Datasets: 3.2.0
522
+ - Tokenizers: 0.21.0
523
+
524
+ ## Citation
525
+
526
+ ### BibTeX
527
+
528
+ #### Sentence Transformers
529
+ ```bibtex
530
+ @inproceedings{reimers-2019-sentence-bert,
531
+ title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
532
+ author = "Reimers, Nils and Gurevych, Iryna",
533
+ booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
534
+ month = "11",
535
+ year = "2019",
536
+ publisher = "Association for Computational Linguistics",
537
+ url = "https://arxiv.org/abs/1908.10084",
538
+ }
539
+ ```
540
+
541
+ #### MultipleNegativesRankingLoss
542
+ ```bibtex
543
+ @misc{henderson2017efficient,
544
+ title={Efficient Natural Language Response Suggestion for Smart Reply},
545
+ author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
546
+ year={2017},
547
+ eprint={1705.00652},
548
+ archivePrefix={arXiv},
549
+ primaryClass={cs.CL}
550
+ }
551
+ ```
552
+
553
+ <!--
554
+ ## Glossary
555
+
556
+ *Clearly define terms in order to be accessible across audiences.*
557
+ -->
558
+
559
+ <!--
560
+ ## Model Card Authors
561
+
562
+ *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
563
+ -->
564
+
565
+ <!--
566
+ ## Model Card Contact
567
+
568
+ *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
569
+ -->
config.json ADDED
@@ -0,0 +1,24 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "sentence-transformers/multi-qa-mpnet-base-dot-v1",
3
+ "architectures": [
4
+ "MPNetModel"
5
+ ],
6
+ "attention_probs_dropout_prob": 0.1,
7
+ "bos_token_id": 0,
8
+ "eos_token_id": 2,
9
+ "hidden_act": "gelu",
10
+ "hidden_dropout_prob": 0.1,
11
+ "hidden_size": 768,
12
+ "initializer_range": 0.02,
13
+ "intermediate_size": 3072,
14
+ "layer_norm_eps": 1e-05,
15
+ "max_position_embeddings": 514,
16
+ "model_type": "mpnet",
17
+ "num_attention_heads": 12,
18
+ "num_hidden_layers": 12,
19
+ "pad_token_id": 1,
20
+ "relative_attention_num_buckets": 32,
21
+ "torch_dtype": "float32",
22
+ "transformers_version": "4.48.3",
23
+ "vocab_size": 30527
24
+ }
config_sentence_transformers.json ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "__version__": {
3
+ "sentence_transformers": "3.4.1",
4
+ "transformers": "4.48.3",
5
+ "pytorch": "2.5.1+cu124"
6
+ },
7
+ "prompts": {},
8
+ "default_prompt_name": null,
9
+ "similarity_fn_name": "dot"
10
+ }
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fd0d91289f33955eb9fd0d649fced0a1477fc32687dd5a2746aadbf6af795ac9
3
+ size 437967672
modules.json ADDED
@@ -0,0 +1,14 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [
2
+ {
3
+ "idx": 0,
4
+ "name": "0",
5
+ "path": "",
6
+ "type": "sentence_transformers.models.Transformer"
7
+ },
8
+ {
9
+ "idx": 1,
10
+ "name": "1",
11
+ "path": "1_Pooling",
12
+ "type": "sentence_transformers.models.Pooling"
13
+ }
14
+ ]
sentence_bert_config.json ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ {
2
+ "max_seq_length": 512,
3
+ "do_lower_case": false
4
+ }
special_tokens_map.json ADDED
@@ -0,0 +1,51 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "bos_token": {
3
+ "content": "<s>",
4
+ "lstrip": false,
5
+ "normalized": false,
6
+ "rstrip": false,
7
+ "single_word": false
8
+ },
9
+ "cls_token": {
10
+ "content": "<s>",
11
+ "lstrip": false,
12
+ "normalized": false,
13
+ "rstrip": false,
14
+ "single_word": false
15
+ },
16
+ "eos_token": {
17
+ "content": "</s>",
18
+ "lstrip": false,
19
+ "normalized": false,
20
+ "rstrip": false,
21
+ "single_word": false
22
+ },
23
+ "mask_token": {
24
+ "content": "<mask>",
25
+ "lstrip": true,
26
+ "normalized": false,
27
+ "rstrip": false,
28
+ "single_word": false
29
+ },
30
+ "pad_token": {
31
+ "content": "<pad>",
32
+ "lstrip": false,
33
+ "normalized": false,
34
+ "rstrip": false,
35
+ "single_word": false
36
+ },
37
+ "sep_token": {
38
+ "content": "</s>",
39
+ "lstrip": false,
40
+ "normalized": false,
41
+ "rstrip": false,
42
+ "single_word": false
43
+ },
44
+ "unk_token": {
45
+ "content": "[UNK]",
46
+ "lstrip": false,
47
+ "normalized": false,
48
+ "rstrip": false,
49
+ "single_word": false
50
+ }
51
+ }
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer_config.json ADDED
@@ -0,0 +1,73 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "added_tokens_decoder": {
3
+ "0": {
4
+ "content": "<s>",
5
+ "lstrip": false,
6
+ "normalized": false,
7
+ "rstrip": false,
8
+ "single_word": false,
9
+ "special": true
10
+ },
11
+ "1": {
12
+ "content": "<pad>",
13
+ "lstrip": false,
14
+ "normalized": false,
15
+ "rstrip": false,
16
+ "single_word": false,
17
+ "special": true
18
+ },
19
+ "2": {
20
+ "content": "</s>",
21
+ "lstrip": false,
22
+ "normalized": false,
23
+ "rstrip": false,
24
+ "single_word": false,
25
+ "special": true
26
+ },
27
+ "3": {
28
+ "content": "<unk>",
29
+ "lstrip": false,
30
+ "normalized": true,
31
+ "rstrip": false,
32
+ "single_word": false,
33
+ "special": true
34
+ },
35
+ "104": {
36
+ "content": "[UNK]",
37
+ "lstrip": false,
38
+ "normalized": false,
39
+ "rstrip": false,
40
+ "single_word": false,
41
+ "special": true
42
+ },
43
+ "30526": {
44
+ "content": "<mask>",
45
+ "lstrip": true,
46
+ "normalized": false,
47
+ "rstrip": false,
48
+ "single_word": false,
49
+ "special": true
50
+ }
51
+ },
52
+ "bos_token": "<s>",
53
+ "clean_up_tokenization_spaces": false,
54
+ "cls_token": "<s>",
55
+ "do_lower_case": true,
56
+ "eos_token": "</s>",
57
+ "extra_special_tokens": {},
58
+ "mask_token": "<mask>",
59
+ "max_length": 250,
60
+ "model_max_length": 512,
61
+ "pad_to_multiple_of": null,
62
+ "pad_token": "<pad>",
63
+ "pad_token_type_id": 0,
64
+ "padding_side": "right",
65
+ "sep_token": "</s>",
66
+ "stride": 0,
67
+ "strip_accents": null,
68
+ "tokenize_chinese_chars": true,
69
+ "tokenizer_class": "MPNetTokenizer",
70
+ "truncation_side": "right",
71
+ "truncation_strategy": "longest_first",
72
+ "unk_token": "[UNK]"
73
+ }
vocab.txt ADDED
The diff for this file is too large to render. See raw diff