Demircan12 commited on
Commit
6c7090b
·
verified ·
1 Parent(s): 66e49e6

Add new SentenceTransformer model

Browse files
.gitattributes CHANGED
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ tokenizer.json filter=lfs diff=lfs merge=lfs -text
1_Pooling/config.json ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "word_embedding_dimension": 768,
3
+ "pooling_mode_cls_token": false,
4
+ "pooling_mode_mean_tokens": true,
5
+ "pooling_mode_max_tokens": false,
6
+ "pooling_mode_mean_sqrt_len_tokens": false,
7
+ "pooling_mode_weightedmean_tokens": false,
8
+ "pooling_mode_lasttoken": false,
9
+ "include_prompt": true
10
+ }
README.md ADDED
@@ -0,0 +1,412 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ tags:
3
+ - sentence-transformers
4
+ - sentence-similarity
5
+ - feature-extraction
6
+ - generated_from_trainer
7
+ - dataset_size:1535
8
+ - loss:MultipleNegativesRankingLoss
9
+ base_model: nomic-ai/nomic-embed-text-v2-moe
10
+ widget:
11
+ - source_sentence: İhale Yönetmeliği Madde 19'a göre doğrudan temin için mal/hizmet/tarife/seyahat
12
+ alımlarının bedel sınırı nedir?
13
+ sentences:
14
+ - 'Week9 (Nov 25): Condition Variables, Deadlocks / Project Phase II: Scheduler
15
+ - @Nov 28, Preliminary Report Due: Dec 23, How to write Project Report, Phase
16
+ 1- DEMO'
17
+ - ç) Üniversitenin tahmini bedeli bir önceki hesap dönemi toplam giderlerinin TÜFE
18
+ oranına göre güncellenecek üçyüzsekizbinüçyüzyetmişdört TL’ye karşılık gelen tutardaki
19
+ bedeli aşmayacak olan mal ve hizmet alımları, tarifeli alımlar ile seyahat alımları.
20
+ - MADDE 9 – (1) Yabancı uyruklu öğrenciler hakkında ilgili mevzuat hükümleri ile
21
+ Senato tarafından belirlenen esaslar uygulanır.
22
+ - source_sentence: Madde 8'e göre Komisyon Başkanı kime karşı sorumludur?
23
+ sentences:
24
+ - Komisyon Başkanı, Rektöre karşı sorumludur.
25
+ - Komisyon Başkanı, çalışma birimlerinin faaliyetlerini izler ve denetler.
26
+ - Alınacak pedagojik formasyon dersleri öğrencinin dönemlik ders yükünün üzerinde
27
+ olması halinde genel not ortalaması 3.00 ve üzeri olan öğrencilerden ücret alınmayacak...
28
+ - source_sentence: What is the grading breakdown for CSE 462?
29
+ sentences:
30
+ - 'Grading Breakdown: 30% Midterm
31
+
32
+ 45% Final
33
+
34
+ 25% Assignments'
35
+ - Teklifler iadeli taahhütlü olarak da gönderilebilir.
36
+ - Any form of cheating will be reported to the faculty's relevant administrative
37
+ body for further action.
38
+ - source_sentence: Resmi Yazışma Yönergesi Madde 16'ya göre paragraflar nasıl başlar
39
+ ve hizalanır?
40
+ sentences:
41
+ - (8)Yazışma birim kodu olmayan veya verilmeyen hiçbir birim yazışma yapamaz.
42
+ - (3) Zorunlu hâllerde veya olağanüstü durumlarda hazırlanan olur yazılarında “OLUR”
43
+ ibaresinden sonra tarih ve imza için uygun boş satır bırakılarak ilk satırda imzalayanın
44
+ adı ve soyadına, ikinci satırda ise unvan bilgilerine yer verilir (Örnek 18).
45
+ - (4) Paragrafa 1,25 cm içeriden başlanır ve metin iki yana hizalanır.
46
+ - source_sentence: CSE 447 (Ozkaya) sınavında kopya çekme girişimi nasıl değerlendirilir?
47
+ sentences:
48
+ - 'Week-10 AVL tree
49
+
50
+ Week-11 IPR tree
51
+
52
+ Week-12 B tree
53
+
54
+ Week-13 B+ tree'
55
+ - 'Alan Eğitimcisi : Doktorasını ve/veya doçentliğini, ilgili alan eğitiminde (fizik
56
+ eğitimi, kimya eğitimi, biyoloji eğitimi, matematik eğitimi, tarih eğitimi, din
57
+ eğitimi, Türkçe eğitimi vb.) almış öğretim üyesini,'
58
+ - 'Exam Cheating Policy: Any attempt at cheating during the midterm and final exams
59
+ will be treated seriously.'
60
+ pipeline_tag: sentence-similarity
61
+ library_name: sentence-transformers
62
+ ---
63
+
64
+ # SentenceTransformer based on nomic-ai/nomic-embed-text-v2-moe
65
+
66
+ This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [nomic-ai/nomic-embed-text-v2-moe](https://huggingface.co/nomic-ai/nomic-embed-text-v2-moe) on the json dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
67
+
68
+ ## Model Details
69
+
70
+ ### Model Description
71
+ - **Model Type:** Sentence Transformer
72
+ - **Base model:** [nomic-ai/nomic-embed-text-v2-moe](https://huggingface.co/nomic-ai/nomic-embed-text-v2-moe) <!-- at revision 1066b6599d099fbb93dfcb64f9c37a7c9e503e85 -->
73
+ - **Maximum Sequence Length:** 512 tokens
74
+ - **Output Dimensionality:** 768 dimensions
75
+ - **Similarity Function:** Cosine Similarity
76
+ - **Training Dataset:**
77
+ - json
78
+ <!-- - **Language:** Unknown -->
79
+ <!-- - **License:** Unknown -->
80
+
81
+ ### Model Sources
82
+
83
+ - **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
84
+ - **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
85
+ - **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
86
+
87
+ ### Full Model Architecture
88
+
89
+ ```
90
+ SentenceTransformer(
91
+ (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: NomicBertModel
92
+ (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
93
+ (2): Normalize()
94
+ )
95
+ ```
96
+
97
+ ## Usage
98
+
99
+ ### Direct Usage (Sentence Transformers)
100
+
101
+ First install the Sentence Transformers library:
102
+
103
+ ```bash
104
+ pip install -U sentence-transformers
105
+ ```
106
+
107
+ Then you can load this model and run inference.
108
+ ```python
109
+ from sentence_transformers import SentenceTransformer
110
+
111
+ # Download from the 🤗 Hub
112
+ model = SentenceTransformer("Demircan12/nomic-embed-text-v2-moe-YeditepeFT")
113
+ # Run inference
114
+ sentences = [
115
+ 'CSE 447 (Ozkaya) sınavında kopya çekme girişimi nasıl değerlendirilir?',
116
+ 'Exam Cheating Policy: Any attempt at cheating during the midterm and final exams will be treated seriously.',
117
+ 'Week-10 AVL tree\nWeek-11 IPR tree\nWeek-12 B tree\nWeek-13 B+ tree',
118
+ ]
119
+ embeddings = model.encode(sentences)
120
+ print(embeddings.shape)
121
+ # [3, 768]
122
+
123
+ # Get the similarity scores for the embeddings
124
+ similarities = model.similarity(embeddings, embeddings)
125
+ print(similarities.shape)
126
+ # [3, 3]
127
+ ```
128
+
129
+ <!--
130
+ ### Direct Usage (Transformers)
131
+
132
+ <details><summary>Click to see the direct usage in Transformers</summary>
133
+
134
+ </details>
135
+ -->
136
+
137
+ <!--
138
+ ### Downstream Usage (Sentence Transformers)
139
+
140
+ You can finetune this model on your own dataset.
141
+
142
+ <details><summary>Click to expand</summary>
143
+
144
+ </details>
145
+ -->
146
+
147
+ <!--
148
+ ### Out-of-Scope Use
149
+
150
+ *List how the model may foreseeably be misused and address what users ought not to do with the model.*
151
+ -->
152
+
153
+ <!--
154
+ ## Bias, Risks and Limitations
155
+
156
+ *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
157
+ -->
158
+
159
+ <!--
160
+ ### Recommendations
161
+
162
+ *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
163
+ -->
164
+
165
+ ## Training Details
166
+
167
+ ### Training Dataset
168
+
169
+ #### json
170
+
171
+ * Dataset: json
172
+ * Size: 1,535 training samples
173
+ * Columns: <code>anchor</code> and <code>positive</code>
174
+ * Approximate statistics based on the first 1000 samples:
175
+ | | anchor | positive |
176
+ |:--------|:-----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|
177
+ | type | string | string |
178
+ | details | <ul><li>min: 11 tokens</li><li>mean: 22.23 tokens</li><li>max: 43 tokens</li></ul> | <ul><li>min: 6 tokens</li><li>mean: 38.09 tokens</li><li>max: 247 tokens</li></ul> |
179
+ * Samples:
180
+ | anchor | positive |
181
+ |:--------------------------------------------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
182
+ | <code>Are the Fall (Regular) and Spring (Irregular) programs in the Faculty of Law different according to MADDE 5?</code> | <code>Güz (Regular) ve Bahar (Irregular) programları başlangıç zamanı dışında her açıdan birbirine denktir.</code> |
183
+ | <code>According to MADDE 6, who can take the Postgraduate Proficiency Exam?</code> | <code>(2) Yeterlik Sınavı’na, yeni kayıtlı öğrencilerle birlikte halen hazırlık programında öğrenimine devam eden öğrenciler de girebilirler.</code> |
184
+ | <code>What is the purpose of the Horizontal/Vertical Transfer Adaptation Principles (Madde 1)?</code> | <code>Madde 1- (1) Yatay/Dikey Geçiş İntibak Esaslarının amacı, Yeditepe Üniversitesine yatay geçiş veya dikey geçiş ile kabul edilen öğrencilerin intibak işlemlerine ilişkin esas ve usulleri belirlemektir.</code> |
185
+ * Loss: [<code>MultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters:
186
+ ```json
187
+ {
188
+ "scale": 20.0,
189
+ "similarity_fct": "cos_sim"
190
+ }
191
+ ```
192
+
193
+ ### Evaluation Dataset
194
+
195
+ #### json
196
+
197
+ * Dataset: json
198
+ * Size: 220 evaluation samples
199
+ * Columns: <code>anchor</code> and <code>positive</code>
200
+ * Approximate statistics based on the first 220 samples:
201
+ | | anchor | positive |
202
+ |:--------|:-----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|
203
+ | type | string | string |
204
+ | details | <ul><li>min: 12 tokens</li><li>mean: 21.82 tokens</li><li>max: 39 tokens</li></ul> | <ul><li>min: 9 tokens</li><li>mean: 36.3 tokens</li><li>max: 160 tokens</li></ul> |
205
+ * Samples:
206
+ | anchor | positive |
207
+ |:----------------------------------------------------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
208
+ | <code>CSE 439 dersinin notlandırma dağılımı nasıldır?</code> | <code>Grading Breakdown: Midterm: 30%\nFinal: 35%\nHomeworks and Quizzes: 15%\nTerm Project: 20%</code> |
209
+ | <code>Hukuk Fakültesi Yönetmeliği Madde 13'e göre dersler öğrencilerin hangi yeteneklerini geliştirmeye yönelik yürütülür?</code> | <code>(3) Dersler öğrencilerin muhakeme ve sözlü-yazılı anlatım yeteneklerinin geliştirilmesine katkı sağlayacak şekilde yürütülür.</code> |
210
+ | <code>Yönetmelik Madde 18'e göre hangi sınav Yönetmeliğine göre başarılı olanlar dil sınavından muaf tutulur?</code> | <code>b) Öğretim dilinin anadil olarak konuşulduğu ülkelerde yabancıların yükseköğrenim görebilmeleri için aranan asgari yabancı dil seviyesinin tespiti amacına yönelik olarak yapılan sınavlarda ve 25/9/2013 tarihli ve 28776 sayılı Resmî Gazete’de yayımlanan Yeditepe Üniversitesi Yabancı Diller Hazırlık Programı Eğitim-Öğretim ve Sınav Yönetmeliği hükümlerine göre başarılı olanlar.</code> |
211
+ * Loss: [<code>MultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters:
212
+ ```json
213
+ {
214
+ "scale": 20.0,
215
+ "similarity_fct": "cos_sim"
216
+ }
217
+ ```
218
+
219
+ ### Training Hyperparameters
220
+ #### Non-Default Hyperparameters
221
+
222
+ - `eval_strategy`: steps
223
+ - `per_device_train_batch_size`: 16
224
+ - `per_device_eval_batch_size`: 16
225
+ - `num_train_epochs`: 5
226
+ - `warmup_ratio`: 0.1
227
+ - `batch_sampler`: no_duplicates
228
+
229
+ #### All Hyperparameters
230
+ <details><summary>Click to expand</summary>
231
+
232
+ - `overwrite_output_dir`: False
233
+ - `do_predict`: False
234
+ - `eval_strategy`: steps
235
+ - `prediction_loss_only`: True
236
+ - `per_device_train_batch_size`: 16
237
+ - `per_device_eval_batch_size`: 16
238
+ - `per_gpu_train_batch_size`: None
239
+ - `per_gpu_eval_batch_size`: None
240
+ - `gradient_accumulation_steps`: 1
241
+ - `eval_accumulation_steps`: None
242
+ - `torch_empty_cache_steps`: None
243
+ - `learning_rate`: 5e-05
244
+ - `weight_decay`: 0.0
245
+ - `adam_beta1`: 0.9
246
+ - `adam_beta2`: 0.999
247
+ - `adam_epsilon`: 1e-08
248
+ - `max_grad_norm`: 1.0
249
+ - `num_train_epochs`: 5
250
+ - `max_steps`: -1
251
+ - `lr_scheduler_type`: linear
252
+ - `lr_scheduler_kwargs`: {}
253
+ - `warmup_ratio`: 0.1
254
+ - `warmup_steps`: 0
255
+ - `log_level`: passive
256
+ - `log_level_replica`: warning
257
+ - `log_on_each_node`: True
258
+ - `logging_nan_inf_filter`: True
259
+ - `save_safetensors`: True
260
+ - `save_on_each_node`: False
261
+ - `save_only_model`: False
262
+ - `restore_callback_states_from_checkpoint`: False
263
+ - `no_cuda`: False
264
+ - `use_cpu`: False
265
+ - `use_mps_device`: False
266
+ - `seed`: 42
267
+ - `data_seed`: None
268
+ - `jit_mode_eval`: False
269
+ - `use_ipex`: False
270
+ - `bf16`: False
271
+ - `fp16`: False
272
+ - `fp16_opt_level`: O1
273
+ - `half_precision_backend`: auto
274
+ - `bf16_full_eval`: False
275
+ - `fp16_full_eval`: False
276
+ - `tf32`: None
277
+ - `local_rank`: 0
278
+ - `ddp_backend`: None
279
+ - `tpu_num_cores`: None
280
+ - `tpu_metrics_debug`: False
281
+ - `debug`: []
282
+ - `dataloader_drop_last`: False
283
+ - `dataloader_num_workers`: 0
284
+ - `dataloader_prefetch_factor`: None
285
+ - `past_index`: -1
286
+ - `disable_tqdm`: False
287
+ - `remove_unused_columns`: True
288
+ - `label_names`: None
289
+ - `load_best_model_at_end`: False
290
+ - `ignore_data_skip`: False
291
+ - `fsdp`: []
292
+ - `fsdp_min_num_params`: 0
293
+ - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
294
+ - `tp_size`: 0
295
+ - `fsdp_transformer_layer_cls_to_wrap`: None
296
+ - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
297
+ - `deepspeed`: None
298
+ - `label_smoothing_factor`: 0.0
299
+ - `optim`: adamw_torch
300
+ - `optim_args`: None
301
+ - `adafactor`: False
302
+ - `group_by_length`: False
303
+ - `length_column_name`: length
304
+ - `ddp_find_unused_parameters`: None
305
+ - `ddp_bucket_cap_mb`: None
306
+ - `ddp_broadcast_buffers`: False
307
+ - `dataloader_pin_memory`: True
308
+ - `dataloader_persistent_workers`: False
309
+ - `skip_memory_metrics`: True
310
+ - `use_legacy_prediction_loop`: False
311
+ - `push_to_hub`: False
312
+ - `resume_from_checkpoint`: None
313
+ - `hub_model_id`: None
314
+ - `hub_strategy`: every_save
315
+ - `hub_private_repo`: None
316
+ - `hub_always_push`: False
317
+ - `gradient_checkpointing`: False
318
+ - `gradient_checkpointing_kwargs`: None
319
+ - `include_inputs_for_metrics`: False
320
+ - `include_for_metrics`: []
321
+ - `eval_do_concat_batches`: True
322
+ - `fp16_backend`: auto
323
+ - `push_to_hub_model_id`: None
324
+ - `push_to_hub_organization`: None
325
+ - `mp_parameters`:
326
+ - `auto_find_batch_size`: False
327
+ - `full_determinism`: False
328
+ - `torchdynamo`: None
329
+ - `ray_scope`: last
330
+ - `ddp_timeout`: 1800
331
+ - `torch_compile`: False
332
+ - `torch_compile_backend`: None
333
+ - `torch_compile_mode`: None
334
+ - `include_tokens_per_second`: False
335
+ - `include_num_input_tokens_seen`: False
336
+ - `neftune_noise_alpha`: None
337
+ - `optim_target_modules`: None
338
+ - `batch_eval_metrics`: False
339
+ - `eval_on_start`: False
340
+ - `use_liger_kernel`: False
341
+ - `eval_use_gather_object`: False
342
+ - `average_tokens_across_devices`: False
343
+ - `prompts`: None
344
+ - `batch_sampler`: no_duplicates
345
+ - `multi_dataset_batch_sampler`: proportional
346
+
347
+ </details>
348
+
349
+ ### Training Logs
350
+ | Epoch | Step | Training Loss | Validation Loss |
351
+ |:------:|:----:|:-------------:|:---------------:|
352
+ | 1.0417 | 100 | 0.1199 | 0.0659 |
353
+ | 2.0833 | 200 | 0.0236 | 0.0524 |
354
+ | 3.125 | 300 | 0.0145 | 0.0578 |
355
+ | 4.1667 | 400 | 0.0102 | 0.0617 |
356
+
357
+
358
+ ### Framework Versions
359
+ - Python: 3.11.12
360
+ - Sentence Transformers: 4.1.0
361
+ - Transformers: 4.51.3
362
+ - PyTorch: 2.6.0+cu124
363
+ - Accelerate: 1.6.0
364
+ - Datasets: 3.6.0
365
+ - Tokenizers: 0.21.1
366
+
367
+ ## Citation
368
+
369
+ ### BibTeX
370
+
371
+ #### Sentence Transformers
372
+ ```bibtex
373
+ @inproceedings{reimers-2019-sentence-bert,
374
+ title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
375
+ author = "Reimers, Nils and Gurevych, Iryna",
376
+ booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
377
+ month = "11",
378
+ year = "2019",
379
+ publisher = "Association for Computational Linguistics",
380
+ url = "https://arxiv.org/abs/1908.10084",
381
+ }
382
+ ```
383
+
384
+ #### MultipleNegativesRankingLoss
385
+ ```bibtex
386
+ @misc{henderson2017efficient,
387
+ title={Efficient Natural Language Response Suggestion for Smart Reply},
388
+ author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
389
+ year={2017},
390
+ eprint={1705.00652},
391
+ archivePrefix={arXiv},
392
+ primaryClass={cs.CL}
393
+ }
394
+ ```
395
+
396
+ <!--
397
+ ## Glossary
398
+
399
+ *Clearly define terms in order to be accessible across audiences.*
400
+ -->
401
+
402
+ <!--
403
+ ## Model Card Authors
404
+
405
+ *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
406
+ -->
407
+
408
+ <!--
409
+ ## Model Card Contact
410
+
411
+ *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
412
+ -->
config.json ADDED
@@ -0,0 +1,73 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "activation_function": "gelu",
3
+ "add_pooling_layer": false,
4
+ "architectures": [
5
+ "NomicBertModel"
6
+ ],
7
+ "attn_pdrop": 0.0,
8
+ "auto_map": {
9
+ "AutoConfig": "nomic-ai/nomic-bert-2048--configuration_hf_nomic_bert.NomicBertConfig",
10
+ "AutoModel": "nomic-ai/nomic-bert-2048--modeling_hf_nomic_bert.NomicBertModel",
11
+ "AutoModelForMaskedLM": "nomic-ai/nomic-bert-2048--modeling_hf_nomic_bert.NomicBertForPreTraining",
12
+ "AutoModelForMultipleChoice": "nomic-ai/nomic-bert-2048--modeling_hf_nomic_bert.NomicBertForMultipleChoice",
13
+ "AutoModelForQuestionAnswering": "nomic-ai/nomic-bert-2048--modeling_hf_nomic_bert.NomicBertForQuestionAnswering",
14
+ "AutoModelForSequenceClassification": "nomic-ai/nomic-bert-2048--modeling_hf_nomic_bert.NomicBertForSequenceClassification",
15
+ "AutoModelForTokenClassification": "nomic-ai/nomic-bert-2048--modeling_hf_nomic_bert.NomicBertForTokenClassification"
16
+ },
17
+ "bos_token_id": null,
18
+ "causal": false,
19
+ "dense_seq_output": true,
20
+ "embd_pdrop": 0.1,
21
+ "eos_token_id": null,
22
+ "expert_choice_router": false,
23
+ "ffn_div": 1,
24
+ "fused_bias_fc": true,
25
+ "fused_dropout_add_ln": true,
26
+ "initializer_range": 0.02,
27
+ "layer_norm_epsilon": 1e-05,
28
+ "max_trained_positions": 2048,
29
+ "mlp_fc1_bias": true,
30
+ "mlp_fc2_bias": true,
31
+ "model_type": "nomic_bert",
32
+ "moe_every_n_layers": 2,
33
+ "moe_impl": "megablocks",
34
+ "moe_normalize_expert_weights": false,
35
+ "moe_resid_pdrop": 0.0,
36
+ "moe_top_k": 2,
37
+ "n_embd": 768,
38
+ "n_head": 12,
39
+ "n_inner": 3072,
40
+ "n_layer": 12,
41
+ "n_positions": 2048,
42
+ "num_experts": 8,
43
+ "num_shared_experts": 0,
44
+ "pad_token_id": 1,
45
+ "pad_vocab_size_multiple": 64,
46
+ "parallel_block": false,
47
+ "parallel_block_tied_norm": false,
48
+ "prenorm": false,
49
+ "qkv_proj_bias": true,
50
+ "reorder_and_upcast_attn": false,
51
+ "resid_pdrop": 0.0,
52
+ "rotary_emb_base": 10000,
53
+ "rotary_emb_fraction": 1.0,
54
+ "rotary_emb_interleaved": false,
55
+ "rotary_emb_scale_base": null,
56
+ "rotary_scaling_factor": null,
57
+ "router_aux_loss_coef": 0.1,
58
+ "scale_attn_by_inverse_layer_idx": false,
59
+ "scale_attn_weights": true,
60
+ "summary_activation": null,
61
+ "summary_first_dropout": 0.1,
62
+ "summary_proj_to_labels": true,
63
+ "summary_type": "cls_index",
64
+ "summary_use_proj": true,
65
+ "torch_dtype": "float32",
66
+ "transformers_version": "4.51.3",
67
+ "type_vocab_size": 1,
68
+ "use_cache": true,
69
+ "use_flash_attn": true,
70
+ "use_rms_norm": null,
71
+ "use_xentropy": true,
72
+ "vocab_size": 250048
73
+ }
config_sentence_transformers.json ADDED
@@ -0,0 +1,20 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "__version__": {
3
+ "sentence_transformers": "4.1.0",
4
+ "transformers": "4.51.3",
5
+ "pytorch": "2.6.0+cu124"
6
+ },
7
+ "prompts": {
8
+ "query": "search_query: ",
9
+ "passage": "search_document: ",
10
+ "Classification": "classification: ",
11
+ "MultilabelClassification": "classification: ",
12
+ "Clustering": "clustering: ",
13
+ "PairClassification": "classification: ",
14
+ "STS": "classification: ",
15
+ "Summarization": "classification: ",
16
+ "Speed": "search_document: "
17
+ },
18
+ "default_prompt_name": null,
19
+ "similarity_fn_name": "cosine"
20
+ }
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0c72a21fd4c5bdd24e9f095ecc125bf8d4154cd557cf7ba62fdc75b5a4a74c06
3
+ size 1901187232
modules.json ADDED
@@ -0,0 +1,20 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [
2
+ {
3
+ "idx": 0,
4
+ "name": "0",
5
+ "path": "",
6
+ "type": "sentence_transformers.models.Transformer"
7
+ },
8
+ {
9
+ "idx": 1,
10
+ "name": "1",
11
+ "path": "1_Pooling",
12
+ "type": "sentence_transformers.models.Pooling"
13
+ },
14
+ {
15
+ "idx": 2,
16
+ "name": "2",
17
+ "path": "2_Normalize",
18
+ "type": "sentence_transformers.models.Normalize"
19
+ }
20
+ ]
sentence_bert_config.json ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ {
2
+ "max_seq_length": 512,
3
+ "do_lower_case": false
4
+ }
sentencepiece.bpe.model ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cfc8146abe2a0488e9e2a0c56de7952f7c11ab059eca145a0a727afce0db2865
3
+ size 5069051
special_tokens_map.json ADDED
@@ -0,0 +1,51 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "bos_token": {
3
+ "content": "<s>",
4
+ "lstrip": false,
5
+ "normalized": false,
6
+ "rstrip": false,
7
+ "single_word": false
8
+ },
9
+ "cls_token": {
10
+ "content": "<s>",
11
+ "lstrip": false,
12
+ "normalized": false,
13
+ "rstrip": false,
14
+ "single_word": false
15
+ },
16
+ "eos_token": {
17
+ "content": "</s>",
18
+ "lstrip": false,
19
+ "normalized": false,
20
+ "rstrip": false,
21
+ "single_word": false
22
+ },
23
+ "mask_token": {
24
+ "content": "<mask>",
25
+ "lstrip": true,
26
+ "normalized": false,
27
+ "rstrip": false,
28
+ "single_word": false
29
+ },
30
+ "pad_token": {
31
+ "content": "<pad>",
32
+ "lstrip": false,
33
+ "normalized": false,
34
+ "rstrip": false,
35
+ "single_word": false
36
+ },
37
+ "sep_token": {
38
+ "content": "</s>",
39
+ "lstrip": false,
40
+ "normalized": false,
41
+ "rstrip": false,
42
+ "single_word": false
43
+ },
44
+ "unk_token": {
45
+ "content": "<unk>",
46
+ "lstrip": false,
47
+ "normalized": false,
48
+ "rstrip": false,
49
+ "single_word": false
50
+ }
51
+ }
tokenizer.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:883b037111086fd4dfebbbc9b7cee11e1517b5e0c0514879478661440f137085
3
+ size 17082987
tokenizer_config.json ADDED
@@ -0,0 +1,55 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "added_tokens_decoder": {
3
+ "0": {
4
+ "content": "<s>",
5
+ "lstrip": false,
6
+ "normalized": false,
7
+ "rstrip": false,
8
+ "single_word": false,
9
+ "special": true
10
+ },
11
+ "1": {
12
+ "content": "<pad>",
13
+ "lstrip": false,
14
+ "normalized": false,
15
+ "rstrip": false,
16
+ "single_word": false,
17
+ "special": true
18
+ },
19
+ "2": {
20
+ "content": "</s>",
21
+ "lstrip": false,
22
+ "normalized": false,
23
+ "rstrip": false,
24
+ "single_word": false,
25
+ "special": true
26
+ },
27
+ "3": {
28
+ "content": "<unk>",
29
+ "lstrip": false,
30
+ "normalized": false,
31
+ "rstrip": false,
32
+ "single_word": false,
33
+ "special": true
34
+ },
35
+ "250001": {
36
+ "content": "<mask>",
37
+ "lstrip": true,
38
+ "normalized": false,
39
+ "rstrip": false,
40
+ "single_word": false,
41
+ "special": true
42
+ }
43
+ },
44
+ "bos_token": "<s>",
45
+ "clean_up_tokenization_spaces": true,
46
+ "cls_token": "<s>",
47
+ "eos_token": "</s>",
48
+ "extra_special_tokens": {},
49
+ "mask_token": "<mask>",
50
+ "model_max_length": 512,
51
+ "pad_token": "<pad>",
52
+ "sep_token": "</s>",
53
+ "tokenizer_class": "XLMRobertaTokenizer",
54
+ "unk_token": "<unk>"
55
+ }