## Inspiration

As a group of skincare enthusiasts, our inspiration came with the shared experience of spending lots of money on skincare, just to not use it if it it didn't match well with our skin. With SkinSync, it is not only helpful for users that are shopping and on the go, but it also allows people to efficiently find skincare that is right for them in a cost-effective way.

## What it does

SkinSync is an app for on-the-go help for users to decide if their current skincare product is right for them. For first time users, SkinSync will ask users a variety of questions asking about their skin type and will save this information. The main functionality of SkinSync is object detection where users can scan the current skincare product. We extract the text presented in an uploaded picture of a skincare product and analyze the product's ingredients with their saved skin type to see if this product is compatible with the user's skin type.

## How we built it

For our backend, we built our database storing user information and preferences utilizing Firestore and Firebase. We also utilized Tesseract OCR packages for text extraction and OpenAI for ingredient analysis. Our front end consisted of utilizing Typescript and React components.

## Challenges we ran into

Some challenges we ran into is utilizing the object detection and connecting our website with OpenAI

## Accomplishments that we're proud of

Accomplishments we are proud of is successfully implementing Tesseract OCR packages and incorporating that into various interactive modals throughout the website.

## What we learned

Through this project we learned how flexible an app must be for it to accommodate to most users. Especially in the realm of skincare, which is different for individual, we had to learn and make design decisions to keep this website functional and useful.

## What's next for SkinSync

Currently SkinSync is a web-application and we would like to deploy it as a mobile app. Although many users do shop online these days, we would also like to make an app version for users on the go. Another thing we would like to implement for SkinSync is to create more personalized user questions that is catered to acne and wrinkles to broaden our target audience.
### Friday 7PM: Setting Things into Motion 🚶

>
> *Blast to the past - for everyone!*
>
>
>

ECHO enriches the lives of those with memory-related issues through reminiscence therapy. By recalling beloved memories from their past, those with dementia, Alzheimer’s and other cognitive conditions can restore their sense of continuity, rebuild neural pathways, and find fulfillment in the comfort of nostalgia. ECHO enables an AI-driven analytical approach to find insights into a patient’s emotions and recall, so that caregivers and family are better-equipped to provide.

### Friday 11PM: Making Strides 🏃‍♂️

>
> *The first step, our initial thoughts*
>
>
>

When it came to wrangling the frontend, we kept our users in mind and knew our highest priority was creating an application that was intuitive and easy to understand. We designed with the idea that ECHO could be seamlessly integrated into everyday life in mind.

### Saturday 9AM: Tripping 🤺

>
> *Whoops! Challenges and pitfalls*
>
>
>

As with any journey, we faced our fair share of obstacles and roadblocks on the way. While there were no issues finding the right APIs and tools to accomplish what we wanted, we had to scour different forums and tutorials to figure out how we could integrate those features. We built ECHO with Next.js and deployed on Vercel (and in the process, spent quite a few credits spamming a button while the app was frozen..!).

Backend was fairly painless, but frontend was a different story. Our vision came to life on Figma and was implemented with HTML/CSS on the ol’ reliable, VSC. We were perhaps a little too ambitious with the mockup and so removed a couple of the bells and whistles.

### Saturday 4PM: Finding Our Way 💪

>
> *One foot in front of the other - learning new things*
>
>
>

From here on out, we were in entirely uncharted territory and had to read up on documentation. Our AI, the Speech Prosody model from Hume, allowed us to take video input from a user and analyze a user’s tone and face in real-time. We learned how to use websockets for streaming APIs for those quick insights, as opposed to a REST API which (while more familiar to us) would have been more of a handful due to our real-time analysis goals.

### Saturday 10PM: What Brand Running Shoes 👟

>
> *Our tech stack*
>
>
>

Nikes.

Apart from the tools mentioned above, we have to give kudos to the platforms that we used for the safe-keeping of assets. To handle videos, we linked things up to Cloudinary so that users can play back old memories and reminisce, and used Postgres for data storage.

### Sunday 7AM: The Final Stretch 🏁

>
> *The power of friendship*
>
>
>

As a team composed of two UWaterloo CFM majors and a WesternU Engineering major, we had a lot of great ideas between us. When we put our heads together, we combined powers and developed ECHO.

Plus, Ethan very graciously allowed us to marathon this project at his house! Thank you for the dumplings.

### Sunday Onward: After Sunrise 🌅

>
> *Next horizons*
>
>
>

With this journey concluded, ECHO’s next great adventure will come in the form of adding cognitive therapy activities to stimulate the memory in a different way, as well as AI transcript composition (along with word choice analysis) for our recorded videos.
## Inspiration

With more people working at home due to the pandemic, we felt empowered to improve healthcare at an individual level. Existing solutions for posture detection are expensive, lack cross-platform support, and often require additional device purchases. We sought to remedy these issues by creating Upright.

## What it does

Upright uses your laptop's camera to analyze and help you improve your posture. Register and calibrate the system in less than two minutes, and simply keep Upright open in the background and continue working. Upright will notify you if you begin to slouch so you can correct it. Upright also has the Upright companion iOS app to view your daily metrics.

Some notable features include:

* Smart slouch detection with ML
* Little overhead - get started in < 2 min
* Native notifications on any platform
* Progress tracking with an iOS companion app

## How we built it

We created Upright’s desktop app using Electron.js, an npm package used to develop cross-platform apps. We created the individual pages for the app using HTML, CSS, and client-side JavaScript. For the onboarding screens, users fill out an HTML form which signs them in using Firebase Authentication and uploads information such as their name and preferences to Firestore. This data is also persisted locally using NeDB, a local JavaScript database. The menu bar addition incorporates a camera through a MediaDevices web API, which gives us frames of the user’s posture. Using Tensorflow’s PoseNet model, we analyzed these frames to determine if the user is slouching and if so, by how much. The app sends a desktop notification to alert the user about their posture and also uploads this data to Firestore. Lastly, our SwiftUI-based iOS app pulls this data to display metrics and graphs for the user about their posture over time.

## Challenges we ran into

We faced difficulties when managing data throughout the platform, from the desktop app backend to the frontend pages to the iOS app. As this was our first time using Electron, our team spent a lot of time discovering ways to pass data safely and efficiently, discussing the pros and cons of different solutions. Another significant challenge was performing the machine learning on the video frames. The task of taking in a stream of camera frames and outputting them into slouching percentage values was quite demanding, but we were able to overcome several bugs and obstacles along the way to create the final product.

## Accomplishments that we're proud of

We’re proud that we’ve come up with a seamless and beautiful design that takes less than a minute to setup. The slouch detection model is also pretty accurate, something that we’re pretty proud of. Overall, we’ve built a robust system that we believe outperforms other solutions using just the webcamera of your computer, while also integrating features to track slouching data on your mobile device.

## What we learned

This project taught us how to combine multiple complicated moving pieces into one application. Specifically, we learned how to make a native desktop application with features like notifications built-in using Electron. We also learned how to connect our backend posture data with Firestore to relay information from our Electron application to our OS app. Lastly, we learned how to integrate a machine learning model in Tensorflow within our Electron application.

## What's next for Upright

The next step is improving the posture detection model with more training data, tailored for each user. While the posture detection model we currently use is pretty accurate, by using more custom-tailored training data, it would take Upright to the next level. Another step for Upright would be adding Android integration for our mobile app, which currently only supports iOS as of now.