task_categories:image-classification task_categories:text-to-image license:cc region:us art anime
---
license: cc
tags:
- art
- anime
pretty_name: LFAnime
task_categories:
- image-classification
- text-to-image
---

# Dataset Card for LFANIME

A dataset of anime frames collected by KaraKaraWitch.

## Dataset Details

### Dataset Description

LFANIME, or Low-Framerate Anime, comprises frames from Japanese animation. The dataset serves dual purposes—facilitating fine-tuning of image diffusion models and functioning as a pre-training resource. Moreover, we anticipate its utilization in image classification.

Important Note: LFAnime is not intended for watching anime. To discourage this application, we have intentionally lowered the frame rate and excluded audio from the dataset.

- **Curated by:** KaraKaraWitch
- **Funded by [optional]:** N/A
- **Shared by [optional]:** N/A
- **Language(s) (NLP):** Nil. Primarily japanese, but no audio is included.
- **License:** CC

## Uses

A tar file compresses each "Episode," encompassing sequential anime frames. The dataset also incorporates chapters for episodes that have them. It's important to note that certain frame numbers may be absent intentionally.

### Direct Use

<!-- This section describes suitable use cases for the dataset. -->

We release this dataset for free in the hopes that it could be used for text to image generation and/or image classification.

### Out-of-Scope Use

Technically speaking, this dataset could be used to watch anime. However we do not recommend as such.  
Additionally there could be unforseen usage that the author does not intend.

<!-- This section addresses misuse, malicious use, and uses that the dataset will not work well for. -->

## Dataset Structure

<!-- This section provides a description of the dataset fields, and additional information about the dataset structure such as criteria used to create the splits, relationships between data points, etc. -->

Each tar file should generally follow this format `LFAnime-[T(Test),A(Alpha),B(Beta),R(Release)]-[Sequential Index]-[AnilistID]-[Episode]`

Each tar file should contain:
```
frame_[XXXX]_[detection_type]_[seconds (float)].jpg
kframes.log (scxvid keyframe log)
metadata.json (Selected frames + Detection metrics + Mode)
```

`detection_type` can be one of the following:
```
- key (KeyFrame)
- p_key (Previous Frame from Key Frame)
- inter (Inter frame)
```

## Dataset Creation

### Curation Rationale

<!-- Motivation for the creation of this dataset. -->

The emphasis has been on developing models for generating images from text, particularly in the realm of creating "anime"-style visuals.  
Examples of such models include Waifu Diffusion and NovelAI's SD 1.x models. Regrettably, these models tend to converge, resulting in a consistent aesthetic.  
While this aesthetic may appeal to many users, it poses a challenge when attempting to diverge from or fine-tune the ingrained visual style of most SD 1.x models.

### Source Data

<!-- This section describes the source data (e.g. news text and headlines, social media posts, translated sentences, ...). -->

#### Data Collection and Processing

<!-- This section describes the data collection and processing process such as data selection criteria, filtering and normalization methods, tools and libraries used, etc. -->

We've opted not to reveal the specific origins of the anime to establish a level of separation between the producers and this dataset.  
Nevertheless, we can outline the processing steps as follows:

1. Extract frames from the mkv file, sampling every 10 frames per second.
2. Utilize scxvid to generate a timecode for identifying scene cuts.
3. Exclude frames that precede or follow a scene cut (considering potential inclusion of 1/2 frames at each scene cut).
4. Save the processed frames to a tar file.

#### Who are the source data producers?

We have decided not to disclose the exact sources.

## Bias, Risks, and Limitations

<!-- This section is meant to convey both technical and sociotechnical limitations. -->

As this dataset is a personal collection from KaraKaraWitch, it will have tendencies to generally not "Shonen" anime and will have female protagonists in general.

### Recommendations

<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->

Users should be made aware of the risks, biases and limitations of the dataset.

## Citation [optional]

```
@misc{lfanime,
  title         = {LFAnime: A Low Framerate anime dataset.},
  author        = {KaraKaraWitch},
  year          = {2023},
  howpublished  = {\url{https://huggingface.co/datasets/RyokoExtra/LFANIME}},
}
```


## Glossary [optional]

<!-- If relevant, include terms and calculations in this section that can help readers understand the dataset or dataset card. -->

Anime:

> Anime (Japanese: アニメ, IPA: [aꜜɲime]) is hand-drawn and computer-generated animation originating from Japan. Outside Japan and in English, anime refers specifically to animation produced in Japan.[1] However, in Japan and in Japanese, anime (a term derived from a shortening of the English word animation) describes all animated works, regardless of style or origin. Many works of animation with a similar style to Japanese animation are also produced outside Japan. Video games sometimes also feature themes and artstyles that can be considered as "anime".  
> - Wikipedia

### Contributions

- [@KaraKaraWitch (Twitter)](https://twitter.com/KaraKaraWitch) for gathering this dataset.
- [ChatGPT](https://chat.openai.com) rewording sentences in this datacard.