## Inspiration

According to a 2015 study in the American Journal of Infection Control, people touch their faces more than 20 times an hour on average. More concerningly, about 44% of the time involves contact with mucous membranes (e.g. eyes, nose, mouth).

With the onset of the COVID-19 pandemic ravaging our population (with more than 300 million current cases according to the WHO), it's vital that we take preventative steps wherever possible to curb the spread of the virus. Health care professionals are urging us to refrain from touching these mucous membranes of ours as these parts of our face essentially act as pathways to the throat and lungs.

## What it does

Our multi-platform application (a python application, and a hardware wearable) acts to make users aware of the frequency they are touching their faces in order for them to consciously avoid doing so in the future. The web app and python script work by detecting whenever the user's hands reach the vicinity of the user's face and tallies the total number of touches over a span of time. It presents the user with their rate of face touches, images of them touching their faces, and compares their rate with a **global average**!

## How we built it

The base of the application (the hands tracking) was built using OpenCV and tkinter to create an intuitive interface for users. The database integration used CockroachDB to persist user login records and their face touching counts. The website was developed in React to showcase our products. The wearable schematic was written up using Fritzing and the code developed on Arduino IDE. By means of a tilt switch, the onboard microcontroller can detect when a user's hand is in an upright position, which typically only occurs when the hand is reaching up to touch the face. The device alerts the wearer via the buzzing of a vibratory motor/buzzer and the flashing of an LED. The emotion detection analysis component was built using the Google Cloud Vision API.

## Challenges we ran into

After deciding to use opencv and deep vision to determine with live footage if a user was touching their face, we came to the unfortunate conclusion that there isn't a lot of high quality trained algorithms for detecting hands, given the variability of what a hand looks like (open, closed, pointed, etc.).
In addition to this, the CockroachDB documentation was out of date/inconsistent which caused the actual implementation to differ from the documentation examples and a lot of debugging.

## Accomplishments that we're proud of

Despite developing on three different OSes we managed to get our application to work on every platform. We are also proud of the multifaceted nature of our product which covers a variety of use cases. Despite being two projects we still managed to finish on time.
To work around the original idea of detecting overlap between hands detected and faces, we opted to detect for eyes visible and determine whether an eye was covered due to hand contact.

## What we learned

We learned how to use CockroachDB and how it differs from other DBMSes we have used in the past, such as MongoDB and MySQL.
We learned about deep vision, how to utilize opencv with python to detect certain elements from a live web camera, and how intricate the process for generating Haar-cascade models are.

## What's next for Hands Off

Our next steps would be to increase the accuracy of Hands Off to account for specific edge cases (ex. touching hair/glasses/etc.) to ensure false touches aren't reported. As well, to make the application more accessible to users, we would want to port the application to a web app so that it is easily accessible to everyone. Our use of CockroachDB will help with scaling in the future. With our newfound familliarity with opencv, we would like to train our own models to have a more precise and accurate deep vision algorithm that is much better suited to our project's goals.
## Inspiration

snore or get pourd on yo pores
Coming into grade 12, the decision of going to a hackathon at this time was super ambitious. We knew coming to this hackathon we needed to be full focus 24/7. Problem being, we both procrastinate and push things to the last minute, so in doing so we created a project to help us

## What it does

It's a 3 stage project which has 3 phases to get the our attention. In the first stage we use a voice command and text message to get our own attention. If I'm still distracted, we commence into stage two where it sends a more serious voice command and then a phone call to my phone as I'm probably on my phone. If I decide to ignore the phone call, the project gets serious and commences the final stage where we bring out the big guns. When you ignore all 3 stages, we send a command that triggers the water gun and shoots the distracted victim which is myself, If I try to resist and run away the water gun automatically tracks me and shoots me wherever I go.

## How we built it

We built it using fully recyclable materials as the future innovators of tomorrow, our number one priority is the environment. We made our foundation fully off of trash cardboard, chopsticks, and hot glue. The turret was built using our hardware kit we brought from home and used 3 servos mounted on stilts to hold the water gun in the air. We have a software portion where we hacked a MindFlex to read off brainwaves to activate a water gun trigger. We used a string mechanism to activate the trigger and OpenCV to track the user's face.

## Challenges we ran into

Challenges we ran into was trying to multi-thread the Arduino and Python together. Connecting the MindFlex data with the Arduino was a pain in the ass, we had come up with many different solutions but none of them were efficient. The data was delayed, trying to read and write back and forth and the camera display speed was slowing down due to that, making the tracking worse. We eventually carried through and figured out the solution to it.

## Accomplishments that we're proud of

Accomplishments we are proud of is our engineering capabilities of creating a turret using spare scraps. Combining both the Arduino and MindFlex was something we've never done before and making it work was such a great feeling. Using Twilio and sending messages and calls is also new to us and such a new concept, but getting familiar with and using its capabilities opened a new door of opportunities for future projects.

## What we learned

We've learned many things from using Twilio and hacking into the MindFlex, we've learned a lot more about electronics and circuitry through this and procrastination. After creating this project, we've learned discipline as we never missed a deadline ever again.

## What's next for You snooze you lose. We dont lose

Coming into this hackathon, we had a lot of ambitious ideas that we had to scrap away due to the lack of materials. Including, a life-size human robot although we concluded with an automatic water gun turret controlled through brain control. We want to expand on this project with using brain signals as it's our first hackathon trying this out
## Inspiration

All three members of the team have had personal experiences with people on the autism spectrum, and seen firsthand their struggles with deciphering emotions. With the recent coronavirus pandemic, the use of online conference programs such as Zoom have exacerbated the issue of reading emotional cues. This is not only a problem for the 75+ million people struggling with ASD worldwide, but also for people with alexithymia, a condition that affects individuals’ ability to understand emotion (which makes up 10% of the world population). Our team decided to solve this problem through the development of Evatone, a tool that provides assistance in emotion identification on video conferencing platforms.

## What it does

Our tool is emotion detection software specifically incorporated for video conferencing. It allows users to see what major expressions they elicit as they speak and identify major emotions expressed by the other participants in the video conference. This appears as labels on video participants’ faces, allowing people with Autism and alexithymia to quickly gauge the emotions of others in the meeting room.

## How we built it

We created a live webcam feed using Typescript (part of our front end) that takes in video input. At set intervals, we send a frame from the video as a JPG image to the Python backend using Flask REST APIs. In our backend, we used the Hume Expression Measurement streaming API, along with a web socket to maintain an open connection, to analyze the facial expression of the frames in real-time, and detect the emotion. We parsed the output of Hume’s list of emotions detected to include only the emotion with the highest score (as this represented the dominant emotion), and then sent this data back to the front end (Typescript) to display it on our live webcam feed. Using cv2’s face detection model paired with div elements part of our html, we then overlay a red box over the face detected in the webcam with the dominant emotion label.

## Challenges we ran into

One challenge we ran into was ensuring that the live webcam feed was running the entire time our application was running. Originally, we built out both the live webcam feed and the backend with Python, using Python’s cv2 library to capture the frames. However, we found that the cv2 webcam frames conflicted with how we were sending frames to Hume’s API. Thus, we transitioned to using Typescript for the webcam and Python for the backend, which allowed us to run the webcam feed the whole time while simultaneously sending frames to our model.

## Accomplishments that we're proud of

We are proud of being able to connect the front end and back end of our code, seamlessly presenting our backend output and data in a more presentable way and with a much better user interface, using front-end code. We are also proud that we were able to learn how to use Hume’s API. On Saturday, we spent hours at Hume’s table debugging our code and learning how web sockets work to allow real-time continuous detection of emotion in facial expressions. We’re really proud that, while we ran into challenges with using Hume’s streaming API, we pushed through, asked questions, and got our final output!

## What we learned

We learned how to work with APIs, as we navigated the Hume API to incorporate it into our code for facial emotion detection. We learned about web sockets and how they allow for a continuous connection. We also learned how to code in TypeScript and how to use Flask as a framework for connecting HTML/CSS/JavaScript with our backend Python code using POST and GET.

## What's next for Evatone

We envision Evatone to be incorporated into online meeting platforms like Zoom, Google Meet, and Microsoft Teams. Furthermore, we see Evatone increasing accessibility for people with autism across the online sphere. We have thought about making a Chrome extension that can scan for any face currently on screen, whether that belongs to a Zoom call or a YouTube video, and similarly detect the emotion of the face, helping people with autism navigate online social interactions with ease.