independently-platform/functiongemma-rag

FunctionGemma-270M-IT RAG

This is a fine-tuned derivative of google/functiongemma-270m-it, optimized for lightweight Retrieval-Augmented Generation (RAG) on mobile / edge / low-power devices. The fine-tune specializes the model to **consistently emit a tool call to vector_search**—with a well-formed, high-recall search query—when the user asks a natural-language question that should be answered from a document store.

It’s intended to be used as the “retrieval controller” in a local-first RAG pipeline:
User question → model generates vector_search(query=…) → system retrieves passages → (optional) downstream answer model composes final response.

Base model

• Base: google/functiongemma-270m-it (Gemma 3 270M family), a small model tuned specifically for function calling. (Google AI for Developers)

• Interface & formatting: Uses FunctionGemma’s special control tokens for tool use (e.g., <start_function_call>…<end_function_call>) and the <escape> delimiter for string fields. (Google AI for Developers)

• Context length (base): 32K total input context (and up to 32K output context per request, budget permitting). (Hugging Face)

What’s new in this fine-tune

Primary behavioral change: When asked questions in natural language, the model reliably chooses to call:

• vector_search

• with a single string argument: a retrieval query designed to maximize recall and relevance for downstream passage ranking.

Example behavior (from your eval set):

• Prompt: “Can you compare the political systems of the Roman Republic and the Aztec Empire… succession and social mobility?”
  Output: <start_function_call>call:vector_search{query:<escape>Roman Republic vs Aztec Empire political systems succession social mobility ...<escape>}<end_function_call> ✅

(Additional examples include VAR vs VAR review, journalism ethics across platforms, intrinsic vs extrinsic motivation, bench vs jury trial, Rodin image sources.)

Intended use

Designed for:

• On-device or constrained deployments (mobile apps, embedded, low-cost CPU boxes) that need fast, local routing to retrieval. FunctionGemma is explicitly positioned as a lightweight base for local-first agents and edge workflows. (Google AI for Developers)

• RAG systems where the most important skill is producing the right search query, not writing the final answer.

Not designed for:

• Being the sole “answer model” for complex, high-stakes, or deeply reasoned tasks (it’s small; use it to retrieve, then answer with a stronger model if needed).

• Multi-step tool plans out of the box (FunctionGemma’s training is strongest for single-turn / parallel calls; multi-step chaining isn’t its primary trained workflow). (Google AI for Developers)

Tool contract

This fine-tune assumes a tool with the following conceptual signature:

• Tool name: vector_search

• Arguments:

  • query (string): a search query describing the user’s information need

• Returns: passages/snippets (top-k) with metadata (titles/urls/ids), which are then fed into a downstream step.

Important formatting note: String values in tool blocks must be wrapped in <escape>…<escape> to avoid parsing ambiguity. (Google AI for Developers)

How to use (recommended pattern)

1. Run the model on the user question.

2. If the output contains a vector_search call, execute retrieval.

3. Feed retrieved passages to:

   • either the same model (if you accept lower-quality synthesis), or

   • a larger model for final answer generation.

If you are using the Hugging Face tooling, FunctionGemma models are typically used via chat templates that support tool definitions and function-call decoding. (Hugging Face)