## Inspiration

The idea for Flexi stemmed from a desire to address the growing health crisis related to poor posture and inactivity. With the rise of sedentary lifestyles, especially in desk-bound jobs, we noticed an increasing number of people suffering from bad posture and its negative effects. We wanted to create something that would help people correct their posture and improve their overall fitness in a personalized way, without the need for expensive trainers or gym memberships. The rise of AI technology gave us the perfect tool to make fitness more accessible and tailored to each individual.

## What it does

Flexi uses cutting-edge AI to analyze users' workout videos, providing instant, personalized feedback on posture and form. Users simply record themselves exercising, and our AI evaluates their movements, offering suggestions for improvement. Whether you're a beginner or a seasoned athlete, Flexi adapts to your fitness level and gives you the insights you need to exercise smarter and reduce the risk of injury. It’s like having a personal trainer in your pocket, guiding you every step of the way.

## How we built it

We built the backend using Flask, with Fetch.AI's uAgents at the core of our decentralized multi-agent system, which is a key innovation of this project. For intelligent retrieval, we integrated Gemini and RAG powered by Chroma. OpenCV was used to process videos, extract key frames, and compress file sizes for improved efficiency. Additionally, we developed an exercise knowledge base using Chroma, enhancing the RAG system's capabilities. To enable voice interaction, we integrated Deepgram for speech recognition and user feedback.

## Challenges we ran into

One of the key challenges we faced was designing a robust decentralized multi-agent system with Fetch.AI’s uAgents, ensuring seamless communication between agents while maintaining system efficiency. Training the AI to provide accurate feedback across various body types and exercise forms also required extensive testing and optimization. Additionally, processing large video files in real-time using OpenCV, while keeping performance smooth, posed a significant technical hurdle. Finally, integrating diverse technologies like RAG, Chroma, and Deepgram into a cohesive user experience without compromising functionality was a complex task that required careful coordination.

## Accomplishments that we're proud of

We're incredibly proud of how Flexi can adapt to different users' needs, providing personalized feedback that feels like having a real trainer. The accuracy of our AI in detecting poor posture and giving relevant, easy-to-follow recommendations is a huge achievement. We’ve managed to build a system that not only helps users improve their form but also contributes to long-term health benefits, which was a key goal from the start. Seeing the app come to life and provide real value to users has been incredibly rewarding.

## What we learned

Throughout the development of Flexi, we learned a lot about balancing advanced technology with simplicity in design. It was important to make sure the AI's capabilities were not only powerful but also accessible and understandable to users. We also gained valuable insights into human movement and posture correction, which helped us refine the feedback provided by the app. Moreover, managing and processing video data in real-time while maintaining a seamless user experience taught us a great deal about optimizing both the backend and frontend systems.

## What's next for Flexi

Next, we’re looking to expand Flexi’s capabilities by adding more types of exercises and integrating deeper AI insights. We want to personalize the fitness journey even further by incorporating features like long-term progress tracking, customizable workout plans, and even nutrition advice. Additionally, we plan to explore partnerships with fitness trainers and health professionals to provide more comprehensive support to our users. Ultimately, our vision for Flexi is to become a full-fledged, AI-powered fitness assistant that not only helps with posture correction but also motivates users to achieve their overall fitness goals in a fun and engaging way.
## Inspiration

The pandemic has affected the health of billions worldwide, and not just through COVID-19. Studies have shown a worrying decrease in physical activity due to quarantining and the closure of clubs, sports, and gyms. Not only does this discourage an active lifestyle, but it can also lead to serious injuries from working out alone at home. Without a gym partner or professional trainer to help spot and correct errors in movements, one can continue to perform inefficient and often damaging exercises without even being aware themselves.

## What it does

Our solution to this problem is **GymLens**, a virtual gym trainer that allows anyone to workout at home with *personal rep tracking* and *correct posture guidance*. During the development of our Minimum Viable Product, we implemented a pose tracker using TensorFlow to track the movement of the person’s key body points. The posture during exercises such as pushups can then be evaluated by processing the data points through a linear regression algorithm. Based on the severity of the posture, a hint is provided on the screen to re-correct the posture.

## How we built it

We used a Tensorflow MoveNet model to detect the positions of body parts. These positions were used as inputs for our machine learning algorithm, which we trained to identify specific stances. Using this, we were able to identify repetitions between each pose.

## Challenges we ran into

From the beginning, our team had to navigate the code editor of Sublime Text and Floobits, which proved to be more difficult than we imagined since members could not log in and sign in to Github. Our front end members who were coding with HTML and CSS ran into problems with margins and paddings with divs and buttons. Aligning and making sure the elements were proportional caused a lot of frustration, but with moral support and many external sources, we were able to get a sleek website with which we could host our project.

Incompatibilities between machine learning tools and the library used for pose detection were a major hurdle. We were able to solve this issue by using our own custom-coded machine learning library with a simple feed forward neural network.

Lastly, our struggles with Floobits ended up being one of our biggest setbacks. It turned out, our entire team soon realized, that when two people were on the same file, the lines of code would severely glitch out, causing uncontrollable chaos when typing. Due to the separate nature of front end and back end programmers, it was inevitable that members would step on each others’ toes on the same file and accidentally undo, delete, or add too many characters into one line of code. We ended up having to code cautiously in the fear of deleting valuable code, but we had many laughs over the numerous errors that transpired due to this glitch. Furthermore, Floobits’ ability to overwrite code turned out to be an asset and liability. Although we were able to work on the same files in real-time, destruction from one member of the team turned out to be collateral. On the last evening of the hackathon, one of our team members accidentally overwrote the remote files that the rest of the team had worked hours on instead of the local ones. In a frantic effort to get our code back, our group tried to press ctrl-z to get to the point where the deletion occurred, but it was too late. Unfortunately, there was nothing we could do to get about 3 hours of work back. Luckily, with our excellent team morale, we separated into groups to repair what had been lost. However, our problems with this code editor did not end here. Nearing the end of the hackathon, our front end and back end duos came together triumphantly as we presented our accomplishments to each other. This final step turned out to be an unsuspecting hurdle once more. As the back end merged their final product into the website, many errors with Github pushes and the integration with Floobits became apparent. Progress had not been saved from branch to branch, and the front end code ended up being set back another 2 hours. Having dealt with this problem before, our team put our heads down, pushed away the frustration of restarting, and began to mend the lost progress once more.

## Accomplishments that we're proud of

One significant milestone within our project was the successful alignment of the canvas-drawn posture overlay with the body of a user. Its occurrence brought the team to a video call, where we offered congratulations while hiding our faces behind our freshly made overlay. Its successful tracking later became the main highlight of multiple demonstrations of exercises and jokes surrounding bad posture.

The front end development team enjoyed the challenge of coding in unfamiliar territory. Their encounters with unfamiliar functions as well as their first attempts at Javascript to create a stopwatch with working buttons all resulted in greater feelings of pride as they were incorporated into the site. They are proud to have designed a self-reassured, visually-appealing website by applying the knowledge they’ve gained over the last 36 hours.

Personally, we're proud to have stayed awake for 21 hours.

## What we learned

This hackathon became a giant learning experience for all of our members, despite the range of coding abilities and experience. Our more experienced back end members tackled Tensorflow’s MoveNet pose detection library for the first time. Meanwhile, the members responsible for the user interface, design, and website navigated Floobits and Sublime text as beginners. Our understanding of the different css functions greatly increased, most notably with layering and element positioning with inspect element. Additionally, members ventured into the world of Javascript for the first time, and realized its handiness with HTML and CSS.

Overall, our team surprised ourselves with our mental fortitude when numerous obstacles were thrown our way, and our ability to learn different functions, languages, software, and platforms so quickly. We worked cohesively and efficiently, all the while discovering the true extent of our capabilities under time pressure and during late nights calls.

## What's next for GymLens

In the future, we hope to expand the functionality of our project by making greater use of its movement correction and movement recognition capabilities. We plan to offer a greater library of exercises sourced from online trainers’ videos used as training data. This extra variety will allow users to take total command of their workout regimes, all while ensuring the safety and efficacy of all exercises.
## Inspiration

Since the pandemic, millions of people worldwide have turned to online alternatives to replace public fitness facilities and other physical activities. At-home exercises have become widely acknowledged, but the problem is that there is no way of telling whether people are doing the exercises accurately and whether they notice potentially physically damaging bad habits they may have developed. Even now, those habits may continuously affect and damage their bodies if left unnoticed. That is why we created **Yudo**.

## What it does

Yudo is an exercise web app that uses **TensorFlow AI**, a custom-developed exercise detection algorithm, and **pose detection** to help users improve their form while doing various exercises.

Once you open the web app, select your desired workout and Yudo will provide a quick exercise demo video. The closer your form matches the demo, the higher your accuracy score will be. After completing an exercise, Yudo will provide feedback generated via **ChatGPT** to help users identify and correct the discrepancies in their form.

## How we built it

We first developed the connection between **TensorFlow** and streaming Livestream Video via **BlazePose** and **JSON**. We used the video's data and sent it to TensorFlow, which returned back a JSON object of the different nodes and coordinates which we used to draw the nodes onto a 2D canvas that updates every single frame and projected this on top of the video element. The continuous flow of JSON data from Tensorflow helped create a series of data sets of what different planks forms would look like. We used our own created data sets, took the relative positions of the relevant nodes, and then created mathematical formulas which matched that of the data sets.

After a discussion with Sean, a MLH member, we decided to integrate OpenAI into our project by having it provide feedback based on how well your plank form is. We did so by utilizing the **ExpressJS** back-end to handle requests for the AI-response endpoint. In the process, we also used **Nodaemon**, a process for continuously restarting servers on code change, to help with our development. We also used **Axios** to send data back and forth between the front end and backend

The front end was designed using **Figma** and **Procreate** to create a framework that we could base our **React** components on. Since it was our first time using React and Tensorflow, it took a lot of trial and error to get CSS and HTML elements to work with our React components.

## Challenges we ran into

* Learning and implementing TensorFlow AI and React for the first time during the hackathon
* Creating a mathematical algorithm that accurately measures the form of a user while performing a specific exercise
* Making visual elements appear and move smoothly on a live video feed

## Accomplishments that we're proud of

* This is our 2nd hackathon (except Darryl)
* Efficient and even work distribution between all team members
* Creation of our own data set to accurately model a specific exercise
* A visually aesthetic, mathematically accurate and working application!

## What we learned

* How to use TensorFlow AI and React
* Practical applications of mathematics in computer science algorithms

## What's next for Yudo

* Implementation of more exercises
* Faster and more accurate live video feed and accuracy score calculations
* Provide live feedback during the duration of the exercise
* Integrate a database for users to save their accuracy scores and track their progress