## Inspiration of Punch My Professors

As McGillians, we juggle from one final to another, drowning in projects that seem to have no end, all, while tackling some mind-bending assignments that leave you scratching your head. It's like you're constantly drowning, swamped with stress, wishing for an epic way to blow off some steam. That's the whole vibe of this game – it's a cool, offbeat escape to channel all that university tension. It's not about getting physical; it's more about flipping all that intense study stress into a good chuckle and a light-hearted jab.

## What is it about

"Punch My Professors" is a game that utilizes OpenCV hand recognition to detect a player's hand movements, translating real-life actions into the digital world. Players can form a fist and simulate a boxing punch towards a 3D model of their professor in the game. The game translates the player's physical punches into virtual ones, allowing them to interact with the professor's 3D model in a fun and harmless boxing match setup.

## How we built it

We built "Punch My Professors" by integrating several technologies:

**Mediapipe Hand Recognition**: We used Mediapipe's advanced hand recognition algorithms to detect and track the player's hand movements in real-time.

**2D to 3D Model Conversion AI**: We leveraged cutting-edge AI to convert 2D images of professors into detailed 3D models, ensuring a realistic and engaging virtual representation.

**Unity Game Development**: The core of the game was developed in Unity, allowing us to craft an immersive 3D environment and ensuring smooth gameplay and physics interactions.

**Custom Physics Engine**: We developed a custom physics engine tailored to simulate the impact of the punches on the 3D models, providing a satisfying feedback loop to the player's actions.

## Challenges we ran into

**Hand Recognition Accuracy**: Achieving high accuracy and low latency in hand recognition was challenging, especially in varying lighting conditions and backgrounds.

**3D Model Fidelity**: Ensuring the 3D models accurately resembled the professors while maintaining performance was a fine balance to strike.

**Physics Simulation**: Creating a realistic and responsive physics simulation that accurately represented the impact of punches on the 3D models required meticulous tweaking and testing.

## Accomplishments that we're proud of

**Seamless Integration**: Successfully integrating OpenCV with Unity and ensuring smooth real-time hand recognition in the game environment.

**High-Fidelity 3D Models**: The AI-powered 2D to 3D conversion resulted in highly realistic models of the professors, exceeding our initial expectations.

**Engaging Gameplay**: Crafting an engaging and fun game that has been well-received by players, providing them with a unique way to de-stress and have fun.

**Expanding the Roster**: Including more characters and scenarios, potentially allowing custom uploads of images to be converted into 3D models.

## What we learned

**Advanced Computer Vision**: Delving deep into OpenCV's capabilities taught us a lot about hand recognition and real-time image processing.

**AI-Powered Modeling**: We gained insights into the latest advancements in AI for 3D modeling, exploring the potential of neural networks in creating lifelike 3D representations from 2D images.

**Game Physics**: We enhanced our understanding of physics in game development, learning how to simulate realistic interactions between virtual objects.

## What's next for Box my Professor

**Multiplayer Functionality**: Introduce a multiplayer mode where players can box against each other's professors in a friendly competitive setup.

**Enhanced Interactivity**: Implement more complex hand gestures and interactions, allowing for a more varied and rich gameplay experience.

**Cross-Platform Support**: Develop the game for multiple platforms, making it accessible to a wider audience.

"Punch My Professors" started as a fun concept but has grown into a project that pushes the boundaries of AI and game development, and we're excited to see how far it can go!
## What is 'Titans'?

VR gaming shouldn't just be a lonely, single-player experience. We believe that we can elevate the VR experience by integrating multiplayer interactions.

We imagined a mixed VR/AR experience where a single VR player's playing field can be manipulated by 'Titans' -- AR players who can plan out the VR world by placing specially designed tiles-- blocking the VR player from reaching the goal tile.

## How we built it

We had three streams of development/design to complete our project: the design, the VR experience, and the AR experience.

For design, we used Adobe Illustrator and Blender to create the assets that were used in this project. We had to be careful that our tile designs were recognizable by both human and AR standards, as the tiles would be used by the AR players to lay our the environment the VR players would be placed in. Additionally, we pursued a low-poly art style with our 3D models, in order to reduce design time in building intricate models and to complement the retro/pixel-style of our eventual AR environment tiles.

For building the VR side of the project, we selected to build a Unity VR application targeting Windows and Mac with the Oculus Rift. One of our most notable achievements here is a custom terrain tessellation and generation engine that mimics several environmental biomes represented in our game as well as integrating a multiplayer service powered by Google Cloud Platform.

The AR side of the project uses Google's ARCore and Google Cloud Anchors API to seamlessly stream anchors (the tiles used in our game) to other devices playing in the same area.

## Challenges we ran into

Hardware issues were one of the biggest time-drains in this project. Setting up all the programs-- Unity and its libraries, blender, etc...-- took up the initial hours following the brainstorming session. The biggest challenge was our Alienware MLH laptop resetting overnight. This was a frustrating moment for our team, as we were in the middle of testing our AR features such as testing the compatibility of our environment tiles.

## Accomplishments that we're proud of

We're proud of the consistent effort and style that went into the game design, from the physical environment tiles to the 3D models, we tried our best to create a pleasant-to-look at game style. Our game world generation is something we're also quite proud of. The fact that we were able to develop an immersive world that we can explore via VR is quite surreal. Additionally, we were able to accomplish some form of AR experience where the phone recognizes the environment tiles.

## What we learned

All of our teammates learned something new: multiplayer in unity, ARCore, Blender, etc... Most importantly we learned the various technical and planning challenges involved in AR/VR game development

## What's next for Titans AR/VR

We hope to eventually connect the AR portion and VR portion of the project together the way we envisioned: where AR players can manipulate the virutal world of the VR player.
## Inspiration

Remember those Japanese game shows with human Tetris? Giant, moving walls would move towards contestants, and they would have to contort their bodies to fit through the holes in the wall or get knocked into the water! You can check out the format here (Click the photo to get taken to a YouTube video!):

[![IMAGE ALT TEXT HERE](https://img.youtube.com/vi/6ioiMXKpHxI/0.jpg)](https://www.youtube.com/watch?v=6ioiMXKpHxI)

We've been wanting to break into the MLH hardware lab to get our hands on a Leap Motion device for ages, and decided to make a virtual human Tetris for your hands!

## What it does

Player uses our *groundbreaking innovation* (Leap Motion device duct taped to the RBC provided sleep-masks) to attach the leap motion on their forehead. The player must move their hand to match the shape of the wall moving towards them in our unity game. The more your hand overlaps with the wall, the more points you lose. The player with the lowest score wins!

**Hootsuite Social Game Version:**

Player one wears the Leap Motion, and also covers their eyes. Player two must tell them how to move their hand to get through the wall.

## How we built it

The game was built using Unity/C#. The shapes of the wall were created using Blender. We needed to interact with the Leap Motion API to calculate the users score. This was done by calculating the position of all of the bones in the visible hands, and then calculating the amount of overlap with the wall shapes. So a player that only has one finger hit the wall will get a lower penalty than a player that has their entire hand overlap with the wall.

## Challenges we ran into

One of the hardest parts of this hackathon was figuring out how to calculate the score for the user. We had to work together as a team to conceptually figure out how we could calculate the overlap of the hand with the wall object.

Leap motion exposes an engine to support contacts/collisions between the virtual hand objects and other physics objects within Unity, but we had the additional challenge of computing "severity" of overlap. A player with 50% of their hand overlapping the incoming Tetris shape should receive a worse score than one who just has an overlapping pinky. We ended up calculating this severity based on "number of contacting digits", as the Leap API exposes the collison state of each finger/palm segment on the hand rig. In order to measure overlap continuously, we didn't use the visual object, but rather a virtual collision object that extends all the way to the user's hands at all time. We switch this collision objects as new Tetris pieces come into focus.

Another challenge we had came from more of a "game design" perspective, where we had to carefully tune the scale and speed of the Tetris pieces, as well as the positioning and field of view of the game camera to give the perception of depth to the oncoming pieces and offer a comfortable level of challenge.

## Accomplishments that we're proud of

We are really happy with how this app turned out! None of us have any prior experience with the Leap Motion, and 2/3 of us have never done any game development or have any experience with Unity. Getting the scoring working was a close call, so we are glad we managed to finish that in time!

## What we learned

We learned lots about game development, and the possibilities and limitations of working with the Leap Motion device.

## What's next for Leap Tetris

We'd be excited to try to make a version of the Fruit Ninja game using the Leap Motion!