task_categories:image-to-text task_categories:text-to-image language:en license:mit size_categories:1K<n<10K format:json modality:text library:datasets library:pandas library:mlcroissant library:polars region:us
---
license: mit
task_categories:
- image-to-text
- text-to-image
language:
- en
pretty_name: simons ARC (abstraction & reasoning corpus) solve symmetry version 28
size_categories:
- 1K<n<10K
configs:
- config_name: default
  data_files:
  - split: train
    path: data.jsonl
---

# Version 1

ARC-AGI Tasks where the job is to transform symmetric images.

example count: 2-4.

test count: 1-2.

image size: 2-3.

symmetry types: hstack2, hstack3, vstack2, vstack3, grid2x2.

# Version 2

image size: 2-4.

# Version 3

Added `HSTACK4`, `VSTACK4`.

# Version 4

Added `HSTACK5`, `VSTACK5`.

# Version 5

Added `ImageSymmetrySquare`, so images can be rotated by 90 degrees, and flipped over the diagonals.

# Version 6

Only exercising `ImageSymmetrySquare` with `HSTACK2`, `VSTACK2`. 

Added random padding around the input images, so the job is to extract the content.

# Version 7

Only exercising `ImageSymmetrySquare` with `GRID2X2`. 

# Version 8

Only exercising `ImageSymmetrySquare` with `HSTACK3`, `VSTACK3`. 

# Version 9

Only exercising `ImageSymmetrySquare` with `HSTACK4`, `HSTACK5`, `VSTACK4`, `VSTACK5`. 

# Version 10

Only exercising `ImageSymmetrySquare` with all the pattern types enabled.

# Version 11

Only exercising `ImageSymmetrySquare` with `HSTACK4`, `VSTACK4`. 

# Version 12

Only exercising `ImageSymmetrySquare` with `HSTACK5`, `VSTACK5`.

# Version 13

Only exercising `ImageSymmetrySquare` with all the pattern types enabled.

# Version 14

Enabled `ImageSymmetryRect`, and added padding.

Enabled `ImageSymmetrySquare`.

# Version 15

Focus only on the new `generate_task_with_symmetry_line`.

pattern size: 2-4.

padding size: 0-3.

# Version 16

Focus only on the new `generate_task_with_symmetry_line`.

pattern size: 2-5.

padding size: 0-4.

# Version 17

Focus only on the new `generate_task_with_symmetry_line`.

pattern size: 2-5.

padding size: 0-4.

Invert colors of pattern.

Invert colors of background and wall.

# Version 18

Focus only on the new `generate_task_with_symmetry_line`.

pattern size: 2-5.

padding size: 0-4.

Invert colors of pattern.

Invert colors of background and wall.

Sporadic insert decoy pixel inside the pattern that is either background color or wall color.

Random thickness of wall.

wall size: 1-2.

# Version 19

Enabled all the task generators.

Slightly smaller patterns for the `generate_task_with_symmetry_line`, since the big patterns seems to be something that the model struggles grokking.

# Version 20

Turned up parameters so the tasks have a length of just below 1024 context length. Previously I aimed for 512 tokens.

# Version 21

Focus only on the new `generate_task_with_symmetry_line`.

pattern size: 2-7.

padding size: 0-10.

wall size: 1-3.

# Version 22

Enabled all transformations.

Smaller images, so they can fit inside a context length 1024.

I forgot to enable it: Earlier predictions added to some of the rows.

# Version 23

Even smaller images. The model struggles grokking the previous verison. 

I forgot to enable it: Earlier predictions added to some of the rows.

# Version 24

Focus only on `generate_task_with_symmetry_line`.

I forgot to enable it: Earlier predictions added to some of the rows.

# Version 25

Enabled all transformations.

Now I have enabled it, oh. Earlier predictions added to some of the rows.

# Version 26

Added fields: `arc_task`, `test_index`, `earlier_output`.

# Version 27

Replaced RLE compressed response with raw pixel response.

Reduced padding size from 8 to 3 pixels.

# Version 28

max image size: 8.
