Training BERT #4 - Train With Next Sentence Prediction (NSP)
Next sentence prediction (NSP) is one-half of the training process behind the BERT model (the other being masked-language modeling - MLM).

Although NSP (and MLM) are used to pre-train BERT models, we can use these exact methods to further pre-train our models to better understand the specific style of language in our own use cases.

So, in this video, we'll cover exactly how we take an unstructured body of text, and use it to pre-train a BERT model using NSP.

Meditations data:
https://github.com/jamescalam/transformers/blob/main/data/text/meditations/clean.txt

Jupyter Notebook
https://github.com/jamescalam/transformers/blob/main/course/training/06_nsp_training.ipynb

🤖 70% Discount on the NLP With Transformers in Python course:
https://bit.ly/3DFvvY5

📙 Medium article:
https://towardsdatascience.com/bert-for-next-sentence-prediction-466b67f8226f

🎉 Sign-up For New Articles Every Week on Medium!
https://medium.com/@jamescalam/membership

📖 If membership is too expensive - here's a free link:
https://towardsdatascience.com/bert-for-next-sentence-prediction-466b67f8226f?sk=3595968413abde1c5833e1a96e449673

🕹️ Free AI-Powered Code Refactoring with Sourcery: 
https://sourcery.ai/?utm_source=YouTub&utm_campaign=JBriggs&utm_medium=aff