Jrinky/model2

SentenceTransformer based on BAAI/bge-m3

This is a sentence-transformers model finetuned from BAAI/bge-m3. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: BAAI/bge-m3
• Maximum Sequence Length: 1024 tokens
• Output Dimensionality: 1024 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 1024, 'do_lower_case': False}) with Transformer model: XLMRobertaModel 
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("Jrinky/mpnet-base-all-nli-triplet")
# Run inference
sentences = [
    'What rights do individuals have regarding their personal data according to European Regulation no. 679/2016',
    'By consulting this site, data relating to identified or identifiable persons may be processed. The consent mechanisms will be evident, brief and easily understandable; if the original conditions for which consent was requested were to be changed, for example if the purpose of data processing changed, further consent will be required pursuant to European Regulation no. 679/2016. All the documents related to consents collected will be kept separate from any other corporate document. Your personal data will not be disclosed and you are granted the exercise of the rights referred to in Articles. 11-20 of the European Regulation n. 679/2016 by writing to Promoviaggi S.p.A., Viale Gian Galeazzo, nr. 3, 20136 Milano (Italy) or by sending an e-mail to [email protected].',
    'The regional economy has benefited from these investments in infrastructure. The project owner has also funded the construction of a local school, which is providing benefits to local children. Project impacts and benefits:\n- The project has generated hydropower plant operation/ maintenance jobs for local people. - 24 operational staff have benefited from six months of capacity building in the form of technical training. - The construction of a new transmission line is reducing electricity loss and increasing the electricity supply in the region. - Former low-quality infrastructure systems in the region have improved, e.g. by upgrading roads, and by building bridges and irrigation canals. - A local school has been built. - The project has provided local farmers with support to broaden their agricultural activities to make them more sustainable (e.g. by implementing aquaculture, which reduces the need for logging for farmland). - The project has reduced the need for wood for heating, cooking, and lighting, thus allowing the forest to regenerate and improving soil conditions, hydrology and biodiversity. - The project has improved regional air quality by reducing the need for diesel generators and wood fires.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 1024]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Training Details

Training Dataset

Unnamed Dataset

• Size: 6,433 training samples
• Columns: anchor and positive
• Approximate statistics based on the first 1000 samples:

  	anchor	positive
  type	string	string
  details	min: 6 tokens mean: 18.1 tokens max: 38 tokens	min: 6 tokens mean: 182.31 tokens max: 1024 tokens

• Samples:

  anchor	positive
  What type of insect is Crambus sudanicola	Crambus sudanicola is a moth in the family Crambidae.
  How can you improve storage capacity in standard-height kitchens with unused wall space	If you have standard-height cabinets with unused wall space above, increasing the cabinets by six inches can improve storage capacity. Utilize cutlery dividers in drawers to organize cooking utensils and tools and keep them off counters.
  What new guidelines has the Library Association issued regarding the sale of rare books and manuscripts	The Library Association has issued new guidelines for the sale of rare books and manuscripts by institutions, writes Kam Patel. The move follows an acrimonious dispute at Keele University over the sale of rare mathematics books for Pounds 1 million. The association said that for an institution to have the authority to sell books, its library should first establish that it has a full legal title to the works.

• Loss: selfloss.Infonce with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim"
  }
  

Evaluation Dataset

Unnamed Dataset

• Size: 804 evaluation samples
• Columns: anchor and positive
• Approximate statistics based on the first 804 samples:

  	anchor	positive
  type	string	string
  details	min: 6 tokens mean: 17.78 tokens max: 37 tokens	min: 4 tokens mean: 190.09 tokens max: 1024 tokens

• Samples:

  anchor	positive
  What does the speaker suggest about the relationship between unarmed civilians and the metaphor of eggs and a wall	One way to read the metaphor, he says, is that unarmed civilians are the eggs, while tanks, guns and white phosphorus shells are the wall. But he also offers a more nuanced interpretation: Each of us is, more or less, an egg.
  When did Reed-Rowe retire from the Foreign Service	Reed-Rowe completed that assignment July 26, 2013, and was succeeded by Amy J. Hyatt. Reed-Rowe then joined the United States Army War College as a member of the Command team which focuses on the development of the next generation of military, interagency and international leaders. She officially retired from the Foreign Service in July 2014. During her career, Reed-Rowe earned several Department of State Meritorious Honor Awards, recognition from the Republic of the Marshall Islands, and the Republic of Palau, and the U.S. Army Superior Civilian Service Award. Personal In addition to English, Reed-Rowe speaks Spanish and French. She has two adult children, Nikkia Rowe and Kevin Anthony Rowe. See also List of ambassadors of the United States References External links US Department of State: Ambassadorial Nomination Statement: Helen Reed-Rowe, Ambassador-Designate to Palau, July 21, 2010 Ambassadors of the United States to Palau African-American diplomats American women ambassadors ...
  When will Star Wars: Galaxy's Edge officially open	Star Wars: Galaxy’s Edge may not officially open until the end of June, but for some fans, it could happen even sooner.

• Loss: selfloss.Infonce with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim"
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 4
• per_device_eval_batch_size: 4
• learning_rate: 2e-05
• num_train_epochs: 1
• warmup_ratio: 0.1
• fp16: True
• batch_sampler: no_duplicates

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 4
• per_device_eval_batch_size: 4
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• learning_rate: 2e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 1
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: True
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: False
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• dispatch_batches: None
• split_batches: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• prompts: None
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional

Training Logs

Epoch	Step	Training Loss	Validation Loss
0.7407	100	0.2167	0.1060

Framework Versions

• Python: 3.12.3
• Sentence Transformers: 3.4.0
• Transformers: 4.42.4
• PyTorch: 2.2.0+cu121
• Accelerate: 1.3.0
• Datasets: 3.2.0
• Tokenizers: 0.19.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

Infonce

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}