AvinabhDutta-Dev/assamese-sentiment-cnn-bilstm

Assamese Sentiment Analysis — Custom CNN-BiLSTM Baseline

This repository contains a custom deep learning baseline model combining a 1D Convolutional Neural Network with a Bidirectional LSTM (CNN-BiLSTM), optimized for binary sentiment analysis on Assamese textual reviews. It serves as a rigorous architectural baseline to evaluate the performance benefits of pre-trained transformer language models.

---

Model Summary

• Model Architecture: Custom CNN + BiLSTM (Keras / TensorFlow)
• Task: Binary Sentiment Classification
• Language: Assamese (as)
• Developer: Avinabh Dutta
• Primary Framework: TensorFlow & Keras
• Top Performance: 93.05% Accuracy | 0.9238 Macro F1

---

Architecture Details

The custom neural network combines local feature extraction with long-range contextual sequence dependencies through a sequential multi-layer layout:

• Embedding Layer: Transforms input token matrices into dense vector representations using a 300-dimensional embedding space.
• 1D Convolutional Layer: Employs 128 filters with a kernel size of 5 to actively extract local semantic patterns and feature maps.
• Max Pooling Layer: Applied immediately after convolution to compress dimensionality and retain the most informative signals.
• BiLSTM Layer: Utilizes 64 hidden units structured bidirectionally to process sequence dynamics from both forward and backward directions concurrently.
• Dropout Layer: Implements a strict dropout rate of 0.5 directly prior to final classification to mitigate overfitting.
• Output Layer: Features a single Dense layer driven by a Sigmoid activation function for binary sentiment distribution mapping.

---

Performance & Evaluation Metrics

The CNN-BiLSTM baseline yielded robust deep learning results, outperforming traditional machine learning methods and establishing a high threshold before transformer integration:

Metric	Score	Percentage / Value
Accuracy	0.9305	93.05%
Macro F1-Score	0.9238	92.38%
Macro Precision	0.9341	93.41%
Macro Recall	0.9160	91.60%
ROC-AUC	0.9787	97.87%
Cohen's Kappa	0.8477	0.8477

---

Dataset Overview

The dataset consists of localized social media commentaries, media assessments, and audience reviews covering drama (নাটক) and cultural content in Assamese.

• Feature Representation: Tracks linguistic subtext from Assamese textual tokens, including character length, word density, punctuation patterns, and emoji occurrences.
• Target Classes: Binary Classification (0: Negative, 1: Positive).

---

How to Use

Because this is a custom Keras architecture, you can load and use the saved model weights using TensorFlow:

import tensorflow as tf
from tensorflow.keras.models import load_model

# Load the saved model file (.h5 or SavedModel format)
model = load_model("assamese_sentiment_cnn_bilstm.h5")

# Example inference workflow requires tokenization/padding pipeline matching training setup
# predictions = model.predict(padded_sequences)

---

Training Hyperparameters

Training optimization and regularizations were tightly managed across the deep learning baseline execution sequence:

Parameter	Value
Max Epochs	15 (with Early Stopping enabled)
Batch Size	32
Loss Function	Binary Crossentropy
Activation (Output)	Sigmoid
Optimization Strategy	Early Stopping on Validation Loss / Macro F1

---

Limitations & Edge Cases

• Lack of Pre-trained Weights: Unlike transformer models (MuRIL, IndicBERT, XLM-R) which leverage massive pre-existing language knowledge, this model must learn all semantic distributions entirely from scratch using only the local training dataset.
• Complex Semantic Shifts: While it captures local phrases and explicit sentiment words effectively, it exhibits limited capacity for deep contextual abstraction, struggling with complex sentence structures, implicit sentiment reversals, and subtle contextual shifts compared to transformer-based alternatives.

---

Citation & Contact

If you use this model or refer to this work in your research, please link back to this Hugging Face repository or the GitHub project repository. If you use this model, please cite the associated preprint:

Dev, C., Baishya, B., Dey, N., Dutta, A., & Dutta, S. (2026). Sentiment Analysis of Assamese Reviews using Transfer Learning Models. Preprint. https://doi.org/10.2139/ssrn.6357982