license:mit region:us
---
license: mit
---

# Malagasy Root Words & Derivatives Dataset

## Overview

This repository provides a dataset of Malagasy root words and their derivatives, scraped from the open-source dictionary resources at motmalgache.org. The data is stored in a SQLite database (`dictionary.db`) containing two main tables:

1.  `root_words`
2.  `derivatives`

Combined, they contain over 40,000 pairs of (inflected/derived form, root form).

The dataset can be leveraged for morphological analysis and related tasks such as lemmatization, stemming, or sequence-to-sequence modeling in the Malagasy language.

## Dataset Structure

### Database Tables

**Table:** `root_words`

**Columns:**

*   `id` (INTEGER, Primary Key, autoincrement)
*   `root` (TEXT): The root word in Malagasy
*   `letter` (TEXT): The initial letter or category from the dictionary site
*   `url` (TEXT): Source link to the dictionary entry
*   `word_type` (TEXT, Nullable): Part-of-speech-like marker (extracted from URL fragment, e.g. `#mg.n`, `#mg.v`, etc.)

**Table:** `derivatives`

**Columns:**

*   `id` (INTEGER, Primary Key, autoincrement)
*   `root_id` (INTEGER, Foreign Key referencing `root_words(id)`)
*   `derivative` (TEXT): The derived or inflected form of the root
*   `url` (TEXT): Source link to the dictionary entry for the derivative
*   `word_type` (TEXT, Nullable): Similar part-of-speech-like marker (if present)

### Relationships

Each row in `derivatives` is linked to exactly one `root_words` entry via `root_id`.

### Statistics

*   Total root words: ~ (Count from your final DB, e.g. ~12,000 unique roots)
*   Total derivatives: ~ (Count from your final DB, e.g. ~44,000 inflected forms)
*   Data coverage: Focused on the Malagasy language, potentially capturing morphological variants, borrowed forms, and partial synonyms.

## How the Dataset was Created

*   **Scraper:** I used a Python-based web scraper (`requests` + `BeautifulSoup`) to crawl the root-word listing pages at motmalgache.org.
*   **Database Setup:** I created the `dictionary.db` with two tables (`root_words` and `derivatives`).
*   **Extraction Logic:**
    *   Each letter (A, B, C, etc.) was fetched from the root listing page.
    *   The root word, along with its link and part-of-speech label, was stored in `root_words`.
    *   For each root, I also captured any derived forms displayed on the same row in the dictionary table, storing them in the `derivatives` table.
*   **Deduplication:** I used unique constraints in the database to avoid inserting duplicate (`root`, `url`) pairs or (`root_id`, `derivative`) pairs.

## Intended Uses & Applications

*   **Morphological Analysis:** Explore how Malagasy words are derived from roots, for linguistic research.
*   **Stemming / Lemmatization:** Build or fine-tune morphological analyzers or morphological generation models.
*   **NLP Experiments:** Use the pairs (`derivative`, `root`) for sequence-to-sequence tasks, especially on character-level transformations.

## Contact

*   **Maintainer:** Tomasz Bawolski/torquenada
*   **GitHub:** [https://github.com/TBawolski](https://github.com/TBawolski)
*   **Hugging Face:** [https://huggingface.co/torquenada](https://huggingface.co/torquenada)
*   **Email:** t.bawolski@student.uw.edu.pl

Feel free to open issues, send pull requests, or reach out if you encounter any problems or have improvements to suggest!