## Inspiration

Because of the busy lives parents don't have enough time to help their little children read. And this can hurt children in longer run. More often when child is left unsupervised they might skip reading.

## What it does

With the help of "Reaby" parents can assign reading tasks for kids and later get data which can help them analyse where the child is lagging. The data provided by "Learn Baby Learn" are :-

1. Amount of text correctly read by the child.
2. For which all words/sentences the child made mistake.
3. Facial expression data to say how comfortable was the child while reading.

## How I built it

I used Houndify's Speech to Text passage to capture speech of the user and convert it to text.
Then compared the text with original transcript of the story or poem. With which the rating of correctly read was calculated.
Used npm diff library to track the words/parts correctly read by the child.
Used Tensorflow's Facial expression package to monitor child's comfort level.
And integrated all on a web portal.

## Challenges I ran into

* Fetching the input text from Houndify's JSON format.
* few deprecated features of "diff" npm package didn't allow me to use the package comfortably. The code still doesnt work on actual web page when hosted(even locally) but works when ran offline.
* Google vision API didnt work with CocoaPods had to move to tensorflow and web based app.
* fetching facial expression rating, storing it into array at calculate mean of expressions.

## Accomplishments that I'm proud of

I have been planning for TreeHacks from past 4 months. But none of my teammates/friends got an entry. Because I am suffering from mild cold, I did not form a team and participated alone and still have made a Proof of concept worthy project.
I had to change my product thrice and had to use web technology(node.js) after 3 years, struggled learning things again but did not let any of the difficulty come in between competing the Hackathon. Self believe, is my accomplishment.

## What I learned

About Houndify and it's applications. Use of tensowrflow on web.

## What's next for Learn Baby Learn

* fetching facial expression rating, storing it into array at calculate mean of expressions.
* Storing all the output data for analytics purpose.
* Improve tech behind catching the mistakes made while reading, capturing such words for further revision.
* With voice modulation understand where all the child stumbled upon while reading and finding irregularities in pronunciation, reading etc.
* Portal/social network to share such videos of kids reading for other kids to inspire from and for us adults to enjoy watching little kids make mistakes while reading and improving.
## Inspiration

Our inspiration for this project was our personal experience of struggling to prepare for public speaking. Whether through previous hackathons, class projects, or poster presentations, we all have had trouble practicing our speeches. The anxiety of public speaking makes it difficult to want to practice with other people and has been a common obstacle preventing us from improving our speech skills. We learned that many people are too shy to seek to rehearse their speeches with others. This motivated us to create an app that provides non-judgmental feedback based on facial expressions and speech patterns, empowering individuals to practice and perfect their public speaking skills in a human-free, non-judgemental environment.

## What it does

Our project starts with a web app where the user can record their presentation in front of their camera. After their presentation concludes, a Q&A takes place with **hume's EVI** (Empathic Voice Interface) within the context of their speech and a custom prompt. Additionally, we used **Deepgram's voice platform** to transcribe the user's speech. **Groq** then parses this speech to rate the user's speaking ability and provide feedback on what they can do to improve.

## How we built it

The frontend of our web application was created using react.js as well as shadcn, a component library. The backend of our application uses hume ai as the main technology. We use the face model in their expression measurement tool to gather values for different emotions/expressions the user exhibits in their speech. These values are then evaluated by Groq which provides a rating for the user's presentation in a friendly, yet helpful manner. The other component of our project is the Q&A portion where the user is asked questions based on the topic they discussed.

## Challenges we ran into

The biggest challenge we ran into was trying to seamlessly integrate three major technologies into the project; those being hume ai, Deepgram, and Google Gemini. Learning all of these technologies on the spot was challenge enough, but finding a way to connect them all was another level of difficult. It was also a challenge testing the code as the wait times for getting output was rather slow and quickly added up, which meant we had to sparingly re-run our code.

## Accomplishments that we're proud of

We're proud of successfully integrating Hume AI and Deepgram, offering actionable feedback on both emotional expressions and speech content. Bringing together facial expression analysis and a post-speech Q&A with an empathetic voice interface (EVI) is something we believe will make practicing speeches much more interactive and insightful for users.

We're also proud of our ability to transcribe speech in real time and use Google Gemini's NLP to analyze and score it for clarity, structure, and impact. These components, working together, form the basis of a comprehensive feedback system that helps users improve their presentation skills with each use.

## What we learned

One major thing we learned was using GitHub to collaborate with backend and frontend developers simultaenously. With one teammate new to hackathons, it was crucial to learn version control for easy management of our codebase. Despite not ultimately using it, we learned about natural language processing (NLP) with Google Gemini. One more thing we learned was how to integrate an AI model into our project with hume ai's face model. This experience emphasized the critical role that emotional awareness plays in effective communication and taught us to appreciate the nuances that come with empathic interactions.

## What's next for Oratix

Moving forward, we plan to enhance our application by refining the feedback mechanism. We want to introduce real-time analysis of their face expressions as well as prosody, so they can adjust accordingly during the presentation rather than after. Another feature we want to add is combining EVI with a live avatar. This will simulate interacting with a human who might nod along or acknowledge statements from the presenter throughout their talk.
## Inspiration

The inspiration for an app like Memo stems from a desire to address common challenges people face in budgeting and expense tracking. The idea originates from observing the inconvenience of manual data entry or the difficulty in keeping an accurate record of daily expenses. Additionally, the increasing reliance on digital receipts and the need for a more intuitive and efficient solution to manage finances are key motivators.

## What it does

Memo takes snapshots of receipts using the webcam, processing the data and returning insightful feedback on spending habits. Details include time and location of purchases, price of purchases, and name of stores. Memo then displays the information in a visually appealing user interface for users to easily analyze their spending.

## How we built it

We built it with Flask, Python, SQL, Tailwind, CSS, React, tesseract, and opencv.

## Challenges we ran into

We had difficulty setting up the database to store the information from the snapshots of receipts.

## Accomplishments that we're proud of

Even though we had little experience with backend development, we created a functioning app that integrated databases and backend technologies such as flask.

## What we learned

We learned Flask, Python, SQL, Tailwind, CSS, React, tesseract, opencv, and we learned to COLLABORATE.

## What's next for Memo

We plan on improving the data visualization for Memo by leveraging Google Maps API. For a better and more intuitive user experience, Memo will display the users' purchases through a 3D map of the store locations, providing insightful data on the time and dates of each purchase.