## Inspiration

<https://www.pewresearch.org/social-trends/?attachment_id=34762>
Pew Research Center lists barbers as among the groups of workers whose day-to-day job has the least exposure to AI.

To solve this problem, we propose an AI-powered mobile app which would allow users to preview potential haircuts together with a barber from the palm of their hands.

## Background

We searched *arxiv* for papers involving hair strand reconstruction, pose alignment, and haircut transfers.
We read and drew inspiration from the following papers:
[Neural Haircut: Prior-Guided Strand-Based Hair Reconstruction](https://arxiv.org/pdf/2306.05872.pdf)
[Style Your Hair: Latent Optimization for Pose-Invariant Hairstyle Transfer via Local-Style-Aware Hair Alignment](https://arxiv.org/pdf/2208.07765.pdf)
[Barbershop: GAN-based Image Compositing using Segmentation Masks](https://arxiv.org/pdf/2106.01505.pdf)

Ultimately, we decided to work with the code from the "Barbershop" and "Style Your Hair" paper to develop our backend.

## What is *kapper*?

**kapper** is a cutting-edge solution which utilizes generative artificial intelligence to create virtual haircuts for users. Users may simply upload a photo of themselves and upload another image of their desired haircut style. The models developed by the authors of "Barbershop" and "Style Your Hair" then process the images, identifying the hair, and user poses, and transplants the desired hairstyle onto the user's photo, creating a realistic and personalized virtual haircut.

## How we built it

*kapper* was developed using Python Flask as a backend, and React Native for frontend. *kapper*'s backend is hosted on Google Cloud Platform. *kapper*'s GAN models are written in Pytorch. We used React Native to create a user-friendly mobile app which allows users to easily upload their photos, select styles, and view their virtual haircuts.

## Challenges we ran into

One of the major challenges we encountered during the development of KapperAI was deploying the machine learning model on Google Cloud.

## Accomplishments that we're proud of

We are incredibly proud of several accomplishments in this project. Firstly, we successfully developed a full-stack application that empowers users, especially barbers, with the power of a GAN within the palms of their hands. We hope that in the future,

## What we learned

Throughout the development of KapperAI, our team gained valuable insights into various aspects of app development and machine learning integration. We deepened our understanding of React Native, which allowed us to create an intuitive user interface. Moreover, we learned how to utilize various components of a complex application, including backend machine learning algorithms and frontend user experiences.

## What's next for KapperAI

Our vision for *kapper* is to scale and expand its features. In the long future, we plan to incorporate more advanced machine learning techniques to offer even more realistic virtual haircuts. We also aim to collaborate with professional hairstylists to provide expert recommendations and styling tips to our users. Ultimately, our goal is for *kapper* to become a the goto tool for barber clients to find their next haircut, and to interact with their barbers.
## SafeWatch

*Elderly Patient Fall-Detection | Automated First Responder Information Relay*

Considering the increasing number of **elderly patient falls**, SafeWatch automates the responsibilities of senior resident caregivers thus relieving them of substantial time commitments.

SafeWatch is a **motion detection software** which recognizes collapsed persons and losses of balance. It is coupled with an instantaneous alert system which can notify building security, off-location loved ones or first responders. Easy integration into pre-existing surveillance camera systems allows for **low implementation costs**. It is a technology which allows us to continuously keep a watchful eye on our loved ones in their old age.

**Future applications** of this software include expansion into public areas for rapid detection of car crashes, physical violence, and illicit activity.
## Inspiration

We all deal with nostalgia. Sometimes we miss our loved ones or places we visited and look back at our pictures. But what if we could revolutionize the way memories are shown? What if we said you can relive your memories and mean it literally?

## What it does

retro.act takes in a user prompt such as "I want uplifting 80s music" and will then use sentiment analysis and Cohere's chat feature to find potential songs out of which the user picks one. Then the user chooses from famous dance videos (such as by Michael Jackson). Finally, we will either let the user choose an image from their past or let our model match images based on the mood of the music and implant the dance moves and music into the image/s.

## How we built it

We used Cohere classify for sentiment analysis and to filter out songs whose mood doesn't match the user's current state. Then we use Cohere's chat and RAG based on the database of filtered songs to identify songs based on the user prompt. We match images to music by first generating a caption of the images using the Azure computer vision API doing a semantic search using KNN and Cohere embeddings and then use Cohere rerank to smooth out the final choices. Finally we make the image come to life by generating a skeleton of the dance moves using OpenCV and Mediapipe and then using a pretrained model to transfer the skeleton to the image.

## Challenges we ran into

This was the most technical project any of us have ever done and we had to overcome huge learning curves. A lot of us were not familiar with some of Cohere's features such as re rank, RAG and embeddings. In addition, generating the skeleton turned out to be very difficult. Apart from simply generating a skeleton using the standard Mediapipe landmarks, we realized we had to customize which landmarks we are connecting to make it a suitable input for the pertained model. Lastly, understanding and being able to use the model was a huge challenge. We had to deal with issues such as dependency errors, lacking a GPU, fixing import statements, deprecated packages.

## Accomplishments that we're proud of

We are incredibly proud of being able to get a very ambitious project done. While it was already difficult to get a skeleton of the dance moves, manipulating the coordinates to fit our pre trained model's specifications was very challenging. Lastly, the amount of experimentation and determination to find a working model that could successfully take in a skeleton and output an "alive" image.

## What we learned

We learned about using media pipe and manipulating a graph of coordinates depending on the output we need, We also learned how to use pre trained weights and run models from open source code. Lastly, we learned about various new Cohere features such as RAG and re rank.

## What's next for retro.act

Expand our database of songs and dance videos to allow for more user options, and get a more accurate algorithm for indexing to classify iterate over/classify the data from the db. We also hope to make the skeleton's motions more smooth for more realistic images. Lastly, this is very ambitious, but we hope to make our own model to transfer skeletons to images instead of using a pretrained one.