## Inspiration

We have all had our fair share of hosting gatherings with unwanted guests. We are tired of birthday parties and wedding crashers, causing unnecessary clean-up and drama! We are also broke college students who cannot afford to hire a bouncer and not everybody has a 6’7 friend.

Say hello to getBounced - the automated bouncer bot to suit your security needs! Our product mounts easily to any entranceway and uses facial recognition to decide if a guest is allowed through or if they’ll receive a disciplinary smack!

## What it does

Before the event, the host will input event information such as the name, location, and start and end time. This information will generate a link to a form the guests will use to register. In addition to providing their contact information, the guests will be prompted to take a 10-second clip of their faces. These clips will be fed to train our computer vision model.

On the day of the event, our smart camera will be set up at the entrance of the event and it will determine if the person is a homie or a nobody. If the latter, then they’ll getBounced! A swift and demure sweep of the blade (trust us, it’s safe) and the alerts that the smart camera will fire will, hopefully, deter the uninvited away.

## How we built it

The webapp was built with typescript and sandboxed with expo. We also made a quick backend in Flask, which helps us deal with the controller logic to preprocess image data, while enabling us to gather analytics on the behaviours of our party attendants, like when they arrive, leave, how long they've stayed at the party, and who they've interacted with. We leveraged several machine learning algorithms to segment and predict the identities of those that show up to our front door. Using mtcnn, we were able to isolate the faces from other noise in images, while using FaceNet (by Google) to generate embedding vectors of said faces. Finally, we trained an SVM and Logistic Regression model (choosing the latter at the end due to increased performance) to classify the vectors into users. We were able to use data preprocessing and image handling algorithms to reduce latency in facial analysis and recognition!

For the hardware component, we used a Jetson Nano to run our machine learning and computer vision algorithms, getting live guest video data from a Tapo Wifi Camera. We then connected the Jetson to an Arduino, which is used to control the servo motor upon which our deterrent weapon is mounted. The Jetson sends a signal through serial communication to the Arduino whenever an intruder is detected.

## Challenges we ran into

One challenge we ran into was our lack of initial hardware materials such as power supplies and mounting equipment. This led to many wild goose chases around campus and visits to the hackathon hardware desk begging them to “check if they have any extras in the back”. Eventually though, after a lot of phone calls and a lot of kind souls willing to lend us parts from their collection, we were able to source everything we needed for our amazing bot.

Another challenge we ran into was not being able to control the stepper and servo motors properly using the Jetson Nano. This is because we were trying to use GPIO pins to do PWM control, which they are not meant to do. This led to a lot of wacky behaviour from our motors. To solve this problem, we decided to use an Arduino to control our servo motor, as there are many servo libraries built for Arduino.

## Accomplishments that we're proud of

The biggest thing we are proud of is how we were able to integrate many different fields and skills in engineering together to create a cool and useful product! We were able to integrate computer vision with machine learning and IoT embedded systems motors and app development and even hot glue and cardboard!

## What we learned

On the hardware side, we learned that it is best practice to use specially made microcontrollers to control electronics, especially power ones like motors. This way, our main system on a chip can be used primarily for algorithm running and communication.

## What's next for getBounced

One next step would be to create “hot-swappable” punishments for unwanted guests. Currently, we only have a sword-smacking mechanism but in the future, we would elevate our product to include other options such as water gun spraying, trap door, ice bucket dump, and many more devious things.

Another next step would be to connect our product to a mobile app which can notify the host in real-time whenever any unwanted guests arrive and capture video evidence of their embarrassing downfall!
## Inspiration

The inspiration for this project was from our personal experiences living with bad roommates, who overstuffed the fridge with a lot of food, that used to be expired or took our food. We wanted to tackle this problem and make a solution to manage the inventory of the fridge or a shared closet.

## What it does

The solution uses cameras to monitor, who is placing and removing items to/from the Fridge and what they are placing/removing. It achieves this using computer vision, technology to recognize the user from the registered set of users for a certain device.

## How we built it

The solution was primarily split into two parts, the machine learning side and the user interface. The machine learning side was completely built in python, from client to server, using opencv,flask, tensorflow, and using the pre trained model inception v3. The half the project also controlled the devices cameras to detect movement and detect objects to be recognized. The user interface side of the project was built on react, using bootstrap and semantic ui. The interface allowed the users to checkout the inventory of their devices, and what items each user had.

The two sides were bridged using Node.js and mongodb, which acted as the central data storage and communication point from the machine learning and the user interface.

## Challenges we ran into

The computer vision side of the project was extremely challenging in the limited time, especially detecting objects in the frame without using Deep Learning to keep the client processing load at a minimum, Furthermore, the data-sets had to curated and specially customized to fit this challenge.was

In terms of the back-end, it was difficult to connect the node server to the python machine learning scripts, so we went with a micro service solution, where the machine learning was served on two flask endpoints to the node server which handled all the further processing.

Finally, in terms of the front-end, there were challenges in terms of changing state of semantic-ui components, which finally required us to switch to bootstrap in the middle of the hack. The front-end also posed a greater time challenge, as the primary front-end developer had to catch a flight the night before the hack ended, and therefore was limited in the time she had to develop the user interface

## Accomplishments that we're proud of

The solution overall worked, in the sense that there was a minimal viable product, where all the complex moving pieces came together to actually recognize an object and the person that added the object.

The computer vision aspect of the project was quite successful for the short time, we had to develop the solution. Especially the object recognition, which had a much higher confidence than was expected for such a short hack.

## What we learned

There were a lot of takeaways from this hack, especially in terms of devops, which posted a major challenge. It made us think a lot more about how difficult deploying IoT Solutions are, and the devop complexity they create. Furthermore, the project had a lot of learning to be had from its ambitious scope, which challenged us to use a lot of new technologies which we weren't extremely comfortable with prior to the hack

## What's next for Koelkast

We hope to keep training the algorithms to get them to a higher confidence level, one that would be more production ready. Furthermore, we want to take a look at perfboards, which are custom made to our specific requirements, These boards would be cheaper to product and deploy, which would be needed for the project to be take to production
## Inspiration

In the wake of a COVID-altered reality, where the lack of resources continuously ripples throughout communities, Heckler.AI emerges as a beacon for students, young professionals, those without access to mentorship and individuals seeking to harness vital presentation skills. Our project is a transformative journey, empowering aspiring leaders to conquer their fears. In a world, where connections faltered, Heckler.AI fills the void, offering a lifeline for growth and social resurgence. We're not just cultivating leaders; we're fostering a community of resilient individuals, breaking barriers, and shaping a future where every voice matters. Join us in the journey of empowerment, where opportunities lost become the stepping stones for a brighter tomorrow. Heckler.AI: Where anxiety transforms into confidence, and leaders emerge from every corner.

## What it does

Heckler.AI is your personalized guide to mastering the art of impactful communication. Our advanced system meticulously tracks hand and arm movements, decodes facial cues from smiles to disinterest, and assesses your posture. This real-time analysis provides actionable feedback, helping you refine your presentation skills. Whether you're a student, young professional, or seeking mentorship, Heckler.AI is the transformative tool that not only empowers you to convey confidence but also builds a community of resilient communicators. Join us in this journey where your every move becomes a step towards becoming a compelling and confident leader. Heckler.AI: Precision in every gesture, resonance in every presentation.

## How we built it

We used OpenCV and the MediaPipe framework to detect movements and train machine learning models. The resulting model could identify key postures that affect your presentation skills and give accurate feedback to fast-track your soft skills development. To detect pivotal postures in the presentation, we used several Classification Models, such as Logistic Regression, Ridge Classifier, Random Forest Classifier, and Gradient Boosting Classifier. Afterwards, we select the best performing model, and use the predictions personalized to our project's needs.

## Challenges we ran into

Implementing webcam into taipy was difficult since there was no pre-built library. Our solution was to use a custom GUI component, which is an extension mechanism that allows developers to add their own visual elements into Taipy's GUI. In the beginning we also wanted to use a websocket to provide real-time feedback, but we deemed it too difficult to build with our limited timeline. Incorporating a custom webcam into Taipy was full of challenges, and each developer's platform was different and required different setups. In hindsight, we could've also used Docker to containerize the images for a more efficient process.

Furthermore, we had difficulties deploying Taipy to our custom domain name, "hecklerai.tech". Since Taipy is built of Flask, we tried different methods, such as deploying on Vercel, using Heroku, and Gunicorn, but our attempts were in vain. A potential solution would be deploying on a virtual machine through cloud hosting platforms like AWS, which could be implemented in the future.

## Accomplishments that we're proud of

We are proud of ourselves for making so much progress in such little time with regards to a project that we thought was too ambitious. We were able to clear most of the key milestones of the project.

## What we learned

We learnt how to utilize mediapipe and use it as the core technology for our project. We learnt about how to manipulate the different points of the body to accurately use these quantifiers to assess the posture and other key metrics of presenters, such as facial expression and hand gestures. We also took the time to learn more about taipy and use it to power our front end, building a beautiful user friendly interface that displays a video feed through the user's webcam.

## What's next for Heckler.ai

At Heckler.AI, we're committed to continuous improvement. Next on our agenda is refining the model's training, ensuring unparalleled accuracy in analyzing hand and arm movements, facial cues, and posture. We're integrating a full-scale application that not only addresses your presentation issues live but goes a step further. Post-video analysis will be a game-changer, offering in-depth insights into precise moments that need improvement. Our vision is to provide users with a comprehensive tool, empowering them not just in the moment but guiding their growth beyond, creating a seamless journey from identifying issues to mastering the art of compelling communication. Stay tuned for a Heckler.AI that not only meets your needs but anticipates and addresses them before you even realize. Elevate your communication with Heckler.AI: Your growth, our commitment.