Update README.md

README.md

@@ -28,23 +28,53 @@ It achieves the following results on the evaluation set:
 
 ## Model description
 
-More information needed
+# MiniLM: Small and Fast Pre-trained Models for Language Understanding and Generation
+
+MiniLM is a distilled model from the paper "MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers".
 
 ## Intended uses & limitations
 
-More information needed
+This model has been created as a learning guide on:
+- How to implement a text classification model using Hugging Face Transformers in TensorFlow
+- How to handle imbalanced class distribution
+
+# How to use the model
+```
+from transformers import pipeline
+
+model_cpt = "laxsvips/minilm-finetuned-emotion"
+pipe = pipeline("text-classification", model=model_cpt)
+pipe("I am really excited about part 2 of the Hugging Face course")
+predicted_scores = pipe("I am so glad you could help me")
+print(predicted_scores)
+````
+The results:
+```
+[[{'label': 'sadness', 'score': 0.003758953418582678},
+  {'label': 'joy', 'score': 0.9874302744865417},
+  {'label': 'love', 'score': 0.00610917154699564},
+  {'label': 'anger', 'score': 9.696640336187556e-05},
+  {'label': 'fear', 'score': 0.0006420552381314337},
+  {'label': 'surprise', 'score': 0.00196251692250371}]]
+```
 
 ## Training and evaluation data
 
-More information needed
+[Emotion](https://huggingface.co/datasets/emotion)
+
+Emotion is a dataset of English Twitter messages with six basic emotions: anger, fear, joy, love, sadness, and surprise. 
 
 ## Training procedure
 
+Refer to the [Colab](https://colab.research.google.com/github/laxmiharikumar/transformers/blob/main/TF_SimpleTrainingWithTransTrainers.ipynb) notebook
+
 ### Training hyperparameters
 
 The following hyperparameters were used during training:
-- optimizer: {'name': 'Adam', 'weight_decay': None, 'clipnorm': None, 'global_clipnorm': None, 'clipvalue': None, 'use_ema': False, 'ema_momentum': 0.99, 'ema_overwrite_frequency': None, 'jit_compile': True, 'is_legacy_optimizer': False, 'learning_rate': {'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 5e-05, 'decay_steps': 1250, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}}, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False}
-- training_precision: float32
+- optimizer: 'Adam', 
+- learning_rate': 5e-05, 
+- batch_size : 64
+- num_epochs - 5
 
 ### Training results
 
@@ -56,10 +86,22 @@ The following hyperparameters were used during training:
 | 0.1465     | 0.9364         | 0.1838          | 0.9295              | 3     |
 | 0.1168     | 0.9446         | 0.1709          | 0.9350              | 4     |
 
+### Evaluation Metrics
+```
+{'accuracy': 0.935,
+ 'precision': 0.937365614416424,
+ 'recall': 0.935,
+ 'f1_score': 0.9355424419858925}
+```
 
 ### Framework versions
 
 - Transformers 4.26.1
 - TensorFlow 2.11.0
 - Datasets 2.9.0
-- Tokenizers 0.13.2
+- Tokenizers 0.13.2
+
+### References
+1. https://www.youtube.com/watch?v=u--UVvH-LIQ
+2. https://huggingface.co/docs/transformers
+3. https://www.tensorflow.org/api_docs/python/tf