🧮 Model Card for dia-intent-sequencer-calculator-adapter

🧠 Model Details

This model is a parameter-efficient fine-tuning (PEFT) LoRA adapter, designed as a demonstration and testbed to evaluate intent sequencing using the DIA DSL module. It fine-tunes microsoft/Phi-4-mini-instruct to act as an intent sequencer, parsing basic compositional arithmetic expressions (combining integers with the operators +, -, *, /) into structured intent sequences using nested function calls (e.g., multiply(a=add(a=2, b=3), b=4)). See the dataset card for details on the range and distribution of integers used in training.

Specifically, the adapter evaluates the model's capability to:

• handle nested, composable tool calls
• generalize across randomized function and parameter names, emphasizing structural reasoning over memorization

It is provided for demonstration and experimentation purposes only.

• Model type: LoRA adapter (PEFT)
• Language(s): English
• License: MIT
• Finetuned from model: microsoft/Phi-4-mini-instruct

🚀 Uses

This adapter fine-tunes Phi-4-mini-instruct to parse basic compositional arithmetic expressions (combining integers with the operators +, -, *, /) into structured DIA DSL code. It supports nested expressions and randomized function/parameter names to help reduce memorization, encouraging structural understanding.

Example:

$ You are a precise intent sequencer. You parse the user's prompt and split it into atomic intents that match one of the defined intents below:
- intent: add
  description: Add two numbers.
  parameters:
    - name: x
      type: float
      description: first number to add
      optional: False
    - name: y
      type: float
      description: second number to add
      optional: False
  return:
    type: float
    description: the sum of x and y
- intent: multiply
  description: Multiply two numbers.
  parameters:
    - name: x
      type: float
      description: first number
      optional: False
    - name: y
      type: float
      description: second number
      optional: False
  return:
    type: float
    description: the product of x and y
...
> 3 + (2 * 4)
< add(x=3, y=multiply(x=2, y=4))

For an example of runtime usage, see calculator.py.

🏗️ Training Details

Trained on a6188466/dia-intent-sequencer-calculator-dataset using the dsl adapter from fifo-tool-datasets and fine_tune.py.

• Dataset: 2,384 synthetic basic arithmetic expressions with randomized tool/argument names
• Epochs: 5
• Batch size: 1
• Precision: bf16
• Framework: transformers, peft, trl (SFTTrainer)

⚙️ Training Hyperparameters

{
  "num_train_epochs": 5,
  "train_batch_size": 1,
  "learning_rate": 5e-06,
  "lr_scheduler_type": "cosine",
  "warmup_ratio": 0.2,
  "bf16": true,
  "seed": 0
}

📈 Training Results

{
  "mean_token_accuracy": 0.921614208817482,
  "total_flos": 100920531959470080.0,
  "train_loss": 0.49252039038094897,
  "train_runtime": 2808.2227,
  "train_samples_per_second": 4.245,
  "train_steps_per_second": 4.245,
  "final_learning_rate": 1.3566795215025352e-13
}

✅ Evaluation

• Eval set: The tool calculator_eval_performance.py was used to run two tests:
  • A held-out test using a test set containing basic arithmetic expressions with randomized tool and parameter names
  • A randomized test where the tool programmatically generates random examples of basic arithmetic expressions with randomized tool and parameter names
• Metric: Functional equivalence — predicted trees are considered correct if they evaluate to the correct result
• Results:
  • Randomized test (length 2-5):
    • Length 2 (no composition): 100.00%
    • Length 3 (1-level composition): 100.00%
    • Length 4 (2-and-3 level composition): 94.90%
    • Length 5 (up to 4-level composition): 88.60%
  • Held-out test set (length 3-6):
    • Length 3 (1-level composition): 98.73%
    • Length 4 (2-and-3 level composition): 98.77%
    • Length 5 (up to 4-level composition): 85.88%
    • Length 6 (up to 5-level composition): 74.75%

These results suggest that the DIA DSL has potential for composing function calls in structured symbolic tasks. This model is intended solely for demonstration purposes and should not be used beyond that scope.

⚠️ Disclaimer & Limitations

This adapter is intended solely as a demonstration and testbed for the DIA DSL module, showcasing its potential for composing function calls in structured symbolic tasks. It should not be used beyond that scope.

This adapter is provided as is, without warranties or guarantees of any kind. It is intended for demonstration and experimentation only.

🪪 License

MIT License. See LICENSE for details.

📬 Contact

For questions, feedback, or bug reports, please open an issue on GitHub or start a discussion on the Hugging Face Hub.