## Inspiration

Virtually every classroom has a projector, whiteboard, and sticky notes. With OpenCV and Python being more accessible than ever, we wanted to create an augmented reality entertainment platform that any enthusiast could learn from and bring to their own place of learning. StickyAR is just that, with a super simple interface that can anyone can use to produce any tile-based Numpy game. Our first offering is *StickyJump* , a 2D platformer whose layout can be changed on the fly by placement of sticky notes. We want to demystify computer science in the classroom, and letting students come face to face with what's possible is a task we were happy to take on.

## What it does

StickyAR works by using OpenCV's Contour Recognition software to recognize the borders of a projector image and the position of human placed sticky notes. We then use a matrix transformation scheme to ensure that the positioning of the sticky notes align with the projector image so that our character can appear as if he is standing on top of the sticky notes. We then have code for a simple platformer that uses the sticky notes as the platforms our character runs, jumps, and interacts with!

## How we built it

We split our team of four into two sections, one half that works on developing the OpenCV/Data Transfer part of the project and the other half who work on the game side of the project. It was truly a team effort.

## Challenges we ran into

The biggest challenges we ran into were that a lot of our group members are not programmers by major. We also had a major disaster with Git that almost killed half of our project. Luckily we had some very gracious mentors come out and help us get things sorted out! We also first attempted to the game half of the project in unity which ended up being too much of a beast to handle.

## Accomplishments that we're proud of

That we got it done! It was pretty amazing to see the little square pop up on the screen for the first time on top of the spawning block. As we think more deeply about the project, we're also excited about how extensible the platform is for future games and types of computer vision features.

## What we learned

A whole ton about python, OpenCV, and how much we regret spending half our time working with Unity. Python's general inheritance structure came very much in handy, and its networking abilities were key for us when Unity was still on the table. Our decision to switch over completely to Python for both OpenCV and the game engine felt like a loss of a lot of our work at the time, but we're very happy with the end-product.

## What's next for StickyAR

StickyAR was designed to be as extensible as possible, so any future game that has colored tiles as elements can take advantage of the computer vision interface we produced. We've already thought through the next game we want to make - *StickyJam*. It will be a music creation app that sends a line across the screen and produces notes when it strikes the sticky notes, allowing the player to vary their rhythm by placement and color.
## Inspiration

Although each of us came from different backgrounds, we each share similar experiences/challenges during our high school years: it was extremely hard to visualize difficult concepts, much less understand the the various complex interactions. This was most prominent in chemistry, where 3D molecular models were simply nonexistent, and 2D visualizations only served to increase confusion. Sometimes, teachers would use a combination of Styrofoam balls, toothpicks and pens to attempt to demonstrate, yet despite their efforts, there was very little effect. Thus, we decided to make an application which facilitates student comprehension by allowing them to take a picture of troubling text/images and get an interactive 3D augmented reality model.

## What it does

The app is split between two interfaces: one for text visualization, and another for diagram visualization. The app is currently functional solely with Chemistry, but can easily be expanded to other subjects as well.

If the text visualization is chosen, an in-built camera pops up and allows the user to take a picture of the body of text. We used Google's ML-Kit to parse the text on the image into a string, and ran a NLP algorithm (Rapid Automatic Keyword Extraction) to generate a comprehensive flashcard list. Users can click on each flashcard to see an interactive 3D model of the element, zooming and rotating it so it can be seen from every angle. If more information is desired, a Wikipedia tab can be pulled up by swiping upwards.

If diagram visualization is chosen, the camera remains perpetually on for the user to focus on a specific diagram. An augmented reality model will float above the corresponding diagrams, which can be clicked on for further enlargement and interaction.

## How we built it

Android Studio, Unity, Blender, Google ML-Kit

## Challenges we ran into

Developing and integrating 3D Models into the corresponding environments.
Merging the Unity and Android Studio mobile applications into a single cohesive interface.

## What's next for Stud\_Vision

The next step of our mobile application is increasing the database of 3D Models to include a wider variety of keywords. We also aim to be able to integrate with other core scholastic subjects, such as History and Math.
## Inspiration

We took inspiration from our experience of how education can be hard. Studies conducted by EdX show that classes that teach quantitative subjects like Mathematics and Physics tend to receive lower ratings from students in terms of engagement and educational capacity than their qualitative counterparts. Of all advanced placement tests, AP Physics 1 receives on average the lowest scores year after year, according to College Board statistics. The fact is, across the board, many qualitative subjects are just more difficult to teach, a fact that is compounded by the isolation that came with remote working, as a result of the COVID-19 pandemic. So, we would like to find a way to promote learning in a fun way.
In keeping with the theme of Ctrl + Alt + Create, we took inspiration from another educational game from the history of computing. In 1991, Microsoft released a programming language and environment called QBASIC to teach first time programmers how to code. One of the demo programs they released with this development environment was a game called Gorillas, an artillery game where two players can guess the velocity and angle in order to try to hit their opponents. We decided to re-imagine this iconic little program from the 90s into a modern networked webgame, designed to teach students kinematics and projectile motion.

## What it does

The goal of our project was to create an educational entertainment game that allows students to better engage in qualitative subjects. We wanted to provide a tool for instructors for both in-classroom and remote education and provide a way to make education more accessible for students attending remotely. Specifically, we focused on introductory high school physics, one of the most challenging subjects to tackle. Similar to Kahoot, teachers can setup a classroom or lobby for students to join in from their devices. Students can join in either as individuals, or as a team. Once a competition begins, students use virtual tape measures to find distances in their surroundings, determining how far their opponent is and the size of obstacles that they need to overcome. Based on these parameters, they can then try out an appropriate angle and calculate an initial velocity to fire their projectiles. Although there is no timer, students are incentivized to work quickly in order to fire off their projectiles before their opponents. Students have a limited number of shots as well, incentivizing them to double-check their work wisely.

## How we built it

We built this web app using HTML, CSS, and Javascript. Our team split up into a Graphics Team and Logics Team. The Logics Team implemented the Kinematics and the game components of this modern recreation of QBASIC Gorillas. The Graphics Team created designs and programmed animations to represent the game logic as well as rendering the final imagery. The two teams came together to make sure everything worked well together.

## Challenges we ran into

We ran into many challenges which include time constraints and our lack of knowledge about certain concepts. We later realized we should have spent more time on planning and designing the game before splitting into teams because it caused problems in miscommunication between the teams about certain elements of the game. Due to time constraints, we did not have time to implement a multiplayer version of the game.

## Accomplishments that we're proud of

The game logically works in single player game. We are proud that we were able to logically implement the entire game, as well as having all the necessary graphics to show its functionality.

## What we learned

We learned the intricacies of game design and game development. Most of us have usually worked with more information-based websites and software technologies. We learned how to make a webapp game from scratch. We also improved our HTML/CSS/Javascript knowledge and our concepts of MVC.

## What's next for Gorillamatics

First we would like to add networking to this game to better meet the goals of increasing connectivity in the classroom as well as sparking a love for Physics in a fun way. We would also like to have better graphics. For the long term, we are planning on adding different obstacles to make different kinematics problems.