Welcome Gemma - Google’s new open LLM
philschmid, osanseviero, pcuenq
> [!NOTE] An update to the Gemma models was released two months after this post, see the latest versions [in this collection](https://huggingface.co/collections/google/gemma-release-65d5efbccdbb8c4202ec078b).

Gemma, a new family of state-of-the-art open LLMs, was released today by Google! It's great to see Google reinforcing its commitment to open-source AI, and we’re excited to fully support the launch with comprehensive integration in Hugging Face.

Gemma comes in two sizes: 7B parameters, for efficient deployment and development on consumer-size GPU and TPU and 2B versions for CPU and on-device applications. Both come in base and instruction-tuned variants.

We’ve collaborated with Google to ensure the best integration into the Hugging Face ecosystem. You can find the 4 open-access models (2 base models & 2 fine-tuned ones) on the Hub. Among the features and integrations being released, we have:

- [Models on the Hub](https://huggingface.co/models?search=google/gemma), with their model cards and licenses
- [🤗 Transformers integration](https://github.com/huggingface/transformers/releases/tag/v4.38.0)
- Integration with Google Cloud
- Integration with Inference Endpoints
- An example of fine-tuning Gemma on a single GPU with 🤗 TRL

## Table of contents

- [What is Gemma?](#what-is-gemma)
  - [Prompt format](#prompt-format)
  - [Exploring the Unknowns](#exploring-the-unknowns)
- [Demo](#demo)
  - [Using 🤗 Transformers](#using-🤗-transformers)
  - [JAX Weights](#jax-weights)
- [Integration with Google Cloud](#integration-with-google-cloud)
- [Integration with Inference Endpoints](#integration-with-inference-endpoints)
- [Fine-tuning with 🤗 TRL](#fine-tuning-with-🤗-trl)
- [Additional Resources](#additional-resources)
- [Acknowledgments](#acknowledgments)


## What is Gemma?

Gemma is a family of 4 new LLM models by Google based on Gemini. It comes in two sizes: 2B and 7B parameters, each with base (pretrained) and instruction-tuned versions. All the variants  can be run on various types of consumer hardware, even without quantization, and have a context length of 8K tokens:

- [gemma-7b](https://huggingface.co/google/gemma-7b): Base 7B model.
- [gemma-7b-it](https://huggingface.co/google/gemma-7b-it): Instruction fine-tuned version of the base 7B model.
- [gemma-2b](https://huggingface.co/google/gemma-2b): Base 2B model.
- [gemma-2b-it](https://huggingface.co/google/gemma-2b-it): Instruction fine-tuned version of the base 2B model.

A month after the original release, Google released a new version of the instruct models. This version has better coding capabilities, factuality, instruction following and multi-turn quality. The model also is less prone to begin its with "Sure,".

- [gemma-1.1-7b-it](https://huggingface.co/google/gemma-1.1-7b-it)
- [gemma-1.1-2b-it](https://huggingface.co/google/gemma-1.1-2b-it)

<div class="flex items-center justify-center">
<img src="/blog/assets/gemma/Gemma-logo-small.png" alt="Gemma logo">
</div>

So, how good are the Gemma models? Here’s an overview of the base models and their performance compared to other open models on the [LLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard) (higher scores are better):

| Model                                                                            | License         | Commercial use? | Pretraining size [tokens] | Leaderboard  score ⬇️ |
|