BERT 101 - State Of The Art NLP Model Explained
britneymuller
# BERT 101 🤗 State Of The Art NLP Model Explained


<script async defer src="https://unpkg.com/medium-zoom-element@0/dist/medium-zoom-element.min.js"></script>

## What is BERT?

BERT, short for Bidirectional Encoder Representations from Transformers, is a Machine Learning (ML) model for natural language processing. It was developed in 2018 by researchers at Google AI Language and serves as a swiss army knife solution to 11+ of the most common language tasks, such as sentiment analysis and named entity recognition. 

Language has historically been difficult for computers to ‘understand’. Sure, computers can collect, store, and read text inputs but they lack basic language _context_.

So, along came Natural Language Processing (NLP): the field of artificial intelligence aiming for computers to read, analyze, interpret and derive meaning from text and spoken words. This practice combines linguistics, statistics, and Machine Learning to assist computers in ‘understanding’ human language.

Individual NLP tasks have traditionally been solved by individual models created for each specific task. That is, until— BERT!

BERT revolutionized the NLP space by solving for 11+ of the most common NLP tasks (and better than previous models) making it the jack of all NLP trades. 

In this guide, you'll learn what BERT is, why it’s different, and how to get started using BERT:

1. [What is BERT used for?](#1-what-is-bert-used-for)
2. [How does BERT work?](#2-how-does-bert-work)
3. [BERT model size & architecture](#3-bert-model-size--architecture)
4. [BERT’s performance on common language tasks](#4-berts-performance-on-common-language-tasks)
5. [Environmental impact of deep learning](#5-enviornmental-impact-of-deep-learning)
6. [The open source power of BERT](#6-the-open-source-power-of-bert)
7. [How to get started using BERT](#7-how-to-get-started-using-bert)
8. [BERT FAQs](#8-bert-faqs)
9. [Conclusion](#9-conclusion)

Let's get started! 🚀


## 1. What is BERT used for?

BERT can be used on a wide variety of language tasks:

-   Can determine how positive or negative a movie’s reviews are. [(Sentiment Analysis)](https://huggingface.co/blog/sentiment-analysis-python)
-   Helps chatbots answer your questions. [(Question answering)](https://huggingface.co/tasks/question-answering)
-   Predicts your text when writing an email (Gmail). [(Text prediction)](https://huggingface.co/tasks/fill-mask)
-   Can write an article about any topic with just a few sentence inputs. [(Text generation)](https://huggingface.co/tasks/text-generation)
-   Can quickly summarize long legal contracts. [(Summarization)](https://huggingface.co/tasks/summarization)
-   Can differentiate words that have multiple meanings (like ‘bank’) based on the surrounding text. (Polysemy resolution)

**There are many more language/NLP tasks + more detail behind each of these.**

***Fun Fact:*** You interact with NLP (and likely BERT) almost every single day! 

NLP is behind Google Translate, voice assistants (Alexa, Siri, etc.), chatbots, Google searches, voice-operated GPS, and more.