task_categories:image-to-text task_categories:text-to-image language:en license:mit size_categories:100K<n<1M format:json modality:text library:datasets library:pandas library:mlcroissant library:polars region:us
---
license: mit
task_categories:
- image-to-text
- text-to-image
language:
- en
pretty_name: simons ARC (abstraction & reasoning corpus) image version 54
size_categories:
- 10K<n<100K
configs:
- config_name: default
  data_files:
  - split: train
    path: data.jsonl
---

# Version 1

Have dataset items that are somewhat evenly of each type. The LLM learned some of the types fine. However rotated images are causing problems.
The image sizes are between 1 and 10 pixels.

# Version 2

Here the majority of dataset items are rotated images. Since this is what my LLM is struggling with.
Smaller images. Here the image sizes are between 1 and 5 pixels.
This helped a lot on the validation loss.

# Version 3

Main focus is now on `count_same_color_as_center_with_8neighbors_nowrap` and image size 1-6.
Which the LLM has struggeld with in the past, maybe due to too big image sizes.
Struggles somewhat with the `count_same_color_as_center_with_8neighbors_nowrap`.

# Version 4

I'm trying smaller images again. Here the image sizes are between 1 and 5 pixels.
Added `same_color_inside_3x3_area_nowrap` that checks if all surrounding pixels agree on the same color, 
maybe that have some synergy with the `count_same_color_as_center_with_8neighbors_nowrap`.
It helped a little, but it's still not as good at counting neighbors as I would like.

# Version 5

I have added a `pixels_with_k_matching_neighbors` with a k parameter between 1-8.
This may help improve on counting the number of neighboring pixels.
The image size 1-6.
This did indeed help on counting the number of surrounding pixels.

# Version 6

Same weight to all the transformations.
Image size 1-11.

# Version 7

Focus on histogram and k-nearest neighbors. image size 1-12.
It seems like the LLM has gotten the hang of it.

# Version 8

Focus on histogram and k-nearest neighbors. image size 5-20.

# Version 9

Focus on histogram and k-nearest neighbors. image size 10-30.

# Version 10

Same weight to all the transformations.
image width 10-30. image height 2-5.

# Version 11

Same weight to all the transformations.
image width 2-5. image height 10-30.

# Version 12

Focus on k-nearest neighbors.
image width 2-5. image height 10-30.

# Version 13

Focus on `compres_x`, `compres_y`, `compres_xy`.
image size is 1-10.

# Version 14

Focus on histograms and k-nearest-neighbors.
image size 5-20.

# Version 15

Focus on histograms and k-nearest-neighbors.
image size 10-30.

# Version 16

Focus on k-nearest-neighbors.
image size 10-25.

# Version 17

Disabled k-nearest-neighbors, I suspect this is the reason why it converges so slowly.
image size 15-30.

# Version 18

Disabled k-nearest-neighbors, and compression.
image size 15-25.

# Version 19

Translate x/y by plus/minus 1.
Disabled rotation and transpose.
image size 22-30.

# Version 20

Focus on k-nearest-neighbors.
image size 5-15.

# Version 21

Focus on k-nearest-neighbors.
image size 8-18.

# Version 22

Same weight to all the transformations.
image size 8-20.

# Version 23

Same weight to all the transformations.
image size 5-30.

The LLM is struggling learning this. I'm going to try with small images.

# Version 24

Focus on rotate cw, rotate ccw, transpose.
image size 2-10.

The LLM is struggling learning this. Despite being small images. I'm going to try with even small images.

# Version 25

Focus on rotate cw, rotate ccw, transpose.
image size 2-5.

The LLM is struggling learning this. Despite being small images. I'm going to try with even small images.

# Version 26

Focus on rotate cw, rotate ccw, transpose, k-nearest-neighbors. 
image size 1-3.

The LLM is struggling learning this. Despite being small images.

# Version 27

Focus on rotate cw, rotate ccw, transpose. 
image size 1-4.

The LLM is struggling learning this. Despite being small images.

# Version 28

Focus on rotate cw, rotate ccw, transpose. 
image size 1-5.

# Version 29

Focus on rotate cw, rotate ccw, transpose. 
image size 1-6.

# Version 30

Focus on rotate cw, rotate ccw, transpose. 
image size 1-8.

# Version 31

Focus on rotate cw, rotate ccw, transpose. 
image size 1-10.

# Version 32

Focus on rotate cw, rotate ccw, transpose. 
image size 1-12.

# Version 33

Focus on rotate cw, rotate ccw, transpose. 
image size 1-14.
The `serialize` items were using fewer names to identify the dataset, now uses the same names as `deserialize`.

# Version 34

Focus only on `rotate cw`. All other operations have been disabled.
image size 1-30.

# Version 35

Focus only on `rotate ccw`. All other operations have been disabled.
image size 1-30.

# Version 36

Focus only on `rotate cw` and `rotate ccw`. All other operations have been disabled.
image size 1-30.

# Version 37

Focus only on `rotate cw` and `rotate ccw`. All other operations have been disabled.
image size 1-30.
Using the same image and apply both `rotate cw` and `rotate ccw`.
My hypothesis is that it will learn better to distinguish between the two rotation types.

# Version 38

Argh, the validation loss was seriously bad on this one. 

I guess what happened is that `rotate cw` always was followed by `rotate ccw`, causing the model to be biased, always expecting the opposite transformation.
Now I have suffled the entire dataset. So there are still 50% of each operation, in random order.

Using the same image and apply both `rotate cw` and `rotate ccw`.
I still think my hypothesis is sound, that it will learn better to distinguish between the two rotation types when it's the same image.

Focus only on `rotate cw` and `rotate ccw`. All other operations have been disabled.
image size 1-30.

# Version 39

The validation loss is not improving.

I'm going back to small image sizes.

Focus only on `rotate cw` and `rotate ccw`. All other operations have been disabled.
image size 1-10.

This helped on the validation loss. Yay.

# Version 40

Focus only on `rotate cw` and `rotate ccw`. All other operations have been disabled.
image size 1-13.

# Version 41

Focus only on `rotate cw` and `rotate ccw`. All other operations have been disabled.
image size 1-16.

# Version 42

Focus only on `rotate cw` and `rotate ccw`. All other operations have been disabled.
image size 1-19.

Jumping to image size 19 was a bit too optimistic, causing a terrible validation loss. So I have to go with a lower image size.

# Version 43

Focus only on `rotate cw` and `rotate ccw`. All other operations have been disabled.
image size 1-17.

# Version 44

Focus only on `rotate cw` and `rotate ccw`. All other operations have been disabled.
image size 1-18.

# Version 45

Focus only on `rotate cw` and `rotate ccw`. All other operations have been disabled.
image size 1-19.

# Version 46

Focus only on `rotate cw` and `rotate ccw`. All other operations have been disabled.
image size 1-20.
This is something that the LLM struggles with. I'm going to make a dataset with another random seed, with the same size 1-20, to see if it improves or worsens.

# Version 47

Focus only on `rotate cw` and `rotate ccw`. All other operations have been disabled.
image size 1-20. Same size as in previous version.

# Version 48

Focus only on `rotate cw` and `rotate ccw`. All other operations have been disabled.
image size 1-21.
This is something that the LLM struggles with. I'm going to make a dataset with another random seed, with the same size 1-21, to see if it improves or worsens.

# Version 49

Focus only on `rotate cw` and `rotate ccw`. All other operations have been disabled.
image size 1-21. Same size as in previous version.
This is something that the LLM struggles with. Not improving.

# Version 50

Focus only on `get_row_as_list` and `get_column_as_list`. All other operations have been disabled.
The `get_column_as_list` is related to rotating the image.
image size 1-21.
This was first time the LLM tried this, so terrible validation loss. I'm going to try smaller images.

# Version 51

Focus only on `get_row_as_list` and `get_column_as_list`. All other operations have been disabled.
The `get_column_as_list` is related to rotating the image.
image size 1-5.
Excellent validation loss.

# Version 52

Focus only on `get_row_as_list` and `get_column_as_list` and `rotate cw` and `rotate ccw`. All other operations have been disabled.
image size 1-10.
No problems for the LLM to learn that. I'm not going to train it to the end.

# Version 53

Focus only on `get_row_as_list` and `get_column_as_list` and `rotate cw` and `rotate ccw`. All other operations have been disabled.
image size 1-15.

# Version 54

Focus only on `get_row_as_list` and `get_column_as_list` and `rotate cw` and `rotate ccw`. All other operations have been disabled.
image size 1-20.

