README.md

# CSI-BERT2

The description is generated by Grok3.

## Model Details

- **Model Name**: CSI-BERT2

- **Model Type**: BERT-inspired transformer for CSI prediction and classification

- **Version**: 2.0

- **Release Date**: August 2025

- **Developers**: Zijian Zhao

- **Organization**: SRIBD, SYSU

- **License**: Apache License 2.0

- **Paper**: [CSI-BERT2: A BERT-inspired Framework for Efficient CSI Prediction and Classification in Wireless Communication and Sensing](https://arxiv.org/abs/2412.06861)

- **Citation:**

  ```
  @article{zhao2024mining,
    title={CSI-BERT2: A BERT-inspired Framework for Efficient CSI Prediction and Classification in Wireless Communication and Sensing},
    author={Zhao, Zijian and Meng, Fanyi and Lyu, Zhonghao and Li, Hang and Li, Xiaoyang and Zhu, Guangxu},
    journal={arXiv preprint arXiv:2412.06861},
    year={2024}
  }
  ```

- **Contact**: [email protected]

- **Repository**: https://github.com/RS2002/CSI-BERT2

- **Previous Version**: [CSI-BERT](https://github.com/RS2002/CSI-BERT)

## Model Description

CSI-BERT2 is an upgraded BERT-inspired transformer model for Channel State Information (CSI) prediction and classification in wireless communication and sensing. It improves upon [CSI-BERT](https://github.com/RS2002/CSI-BERT) with optimized model and code structure, supporting tasks like CSI recovery, prediction, gesture recognition, fall detection, people identification, and people number estimation. The model processes CSI amplitude data and supports adversarial training with a GAN-based discriminator.

- **Architecture**: BERT-based transformer with optional GAN discriminator
- **Input Format**: CSI amplitude (batch_size, length, receiver_num * carrier_dim), attention mask (batch_size, length), optional timestamp (batch_size, length)
- **Output Format**: Hidden states of dimension [batch_size, length, hidden_dim]
- **Hidden Size**: 128
- **Training Objective**: MLM pre-training with GAN (optional) and task-specific fine-tuning
- **Tasks Supported**: CSI recovery, CSI prediction, CSI classification

## Training Data

The model was trained on the following datasets:

- **Public Datasets:**
  - [WiGesture](http://www.sdp8.net/Dataset?id=5d4ee7ca-d0b0-45e3-9510-abb6e9cdebf9): Gesture recognition, people identification
  - [WiFall](https://github.com/RS2002/KNN-MMD/tree/main/WiFall): Action recognition, fall detection, people identification
- **Proposed Dataset:**
  - [WiCount]([CSI-BERT2/WiCount at main · RS2002/CSI-BERT2](https://github.com/RS2002/CSI-BERT2/tree/main/WiCount)): People number estimation
- **Data Structure:**
  - **Amplitude**: (batch_size, length, receiver_num * carrier_dim)
  - **Timestamp**: (batch_size, length) (optional)
  - **Label**: (batch_size)
- **Note**: Refer to [CSI-BERT](https://github.com/RS2002/CSI-BERT) for data preparation details. Custom dataloaders may be needed for specific tasks.

## Usage

### Installation

```shell
git clone https://huggingface.co/RS2002/CSI-BERT2
```

### Example Code

```python
import torch
from model import CSI_BERT2
model = CSI_BERT2.from_pretrained("RS2002/CSI-BERT2")
csi = torch.rand((2, 100, 52))
time_stamp = torch.rand((2, 100))
attention_mask = torch.zeros((2, 100))
y = model(csi,time_stamp,attention_mask)
print(y.shape) # dim: [2,100,128] (batch_size,length,hidden_dim)
```