## Inspiration

In the near future, everything is going to be automated. This applicaiton is to enabled hands free voice messaging, think of voice mail but with the usability of texting. VoiceTunnel is used to solve problems such as communication to your friend if you are driving.

## What it does

An always listening voice client that knows you and your contacts by name! A server that routes messages from the voice client to another voice client. Speak into you client with your friends name, and you begin a voice to voice conversations!

## How I built it

Python and web sockets for the server, with a python client to be used on a raspberry pi
javascript, client side websockets and jquery for a thin web app client

## Challenges I ran into

asynchronous communication issues
challenges of making a natural user interface for the app to know when to listen, stop listening, send the message, and play messages without having any other inputs.

## Accomplishments that I'm proud of

Having a lot of components to demo with the raspberry pi and web client

## What I learned

websockets and setup of the raspberry pi

## What's next for VoiceTunnel

This should be further built onto existing platforms or ai assistants like amazon alexa, siri, or google assistant to allow quick communication between peers
## Discord contact

* Nand Vinchhi - vinchhi
* Advay Choudhury - pdy.gn
* Rushil Madhu - rushil695

## Inspiration

Over the past few years, social media platforms like Tiktok and Instagram reels have used short-form content to take over the entertainment industry. Though this hyper-optimized form of content is engaging, it has led to students having shortened attention spans and a higher tendency to multi-task. Through Relearn, we aim to modernize studying to match these changing mental abilities.

## What it does

#### Scrolling interface for short videos

* Intuitive tiktok-like interface for scrolling educational short videos
* Complete with engaging music, bold subtitles, and background videos
* Functionality to like videos, indicating that the user has understood that particular concept

#### Content generation AI

* Uses lecture videos from Khanacademy + GPT-4.5 analysis to generate bite-sized educational clips
* Custom video processing for adding elements such as music and subtitles

#### Recommendation algorithm

* Ensures that the user progressively covers all content in a given topic
* Occasional meme videos for a quick study break!

#### Edison AI

* An LLM-based bot account that answers student questions in the comments section
* Understands the context from the current video to provide precise and useful answers

## Additional details for sponsor prizes

#### Best Use of AI in Education

Relearn explores the confluence of cutting-edge AI tech and useful education solutions -

* Our content generation pipeline utilizes LLMs (GPT-4.5) to understand context in Khanacademy lecture videos to create short and engaging clips
* Our AI account, Edison, utilizes LLMs to provide precise and useful answers to questions in the comments of a specific video

#### Best Accessibility Hack sponsored by Fidelity

Our platform makes high-quality content available over the internet. We enable anyone with a mobile phone to improve their study consistency and quality, which would help bridge economic and social gaps in the long term.

#### Most Creative Use of Redis Cloud

We used Redis for the following pivotal aspects of our app -

* High speed caching and access of dynamic data such as likes, comments, and progress
* Caching the latest video that a given user interacted with in, and feeding that into our recommendation algorithm (speeds up the algorithm by a huge margin)
* See this file for our Redis functions - <https://github.com/NandVinchhi/relearn-hackathon/blob/main/backend/dbfunctions.py>

## How we built it

#### Front-end mobile application

* Built with Swift and Swift UI
* Used multiple packages for specific functionalities such as Google Auth, video playback etc.

#### Content generation pipeline

* YT Transcript API + PyTube for scraping Khanacademy videos
* Open AI to retrieve context and divide long lecture videos into bite-sized clips
* MoviePy for automatic video editing - subtitles, music, and background video

#### Back-end

* Python + FastAPI for the server
* Supabase database + cloud buckets for storing short videos + curriculum
* Redis database for storing dynamic like, comment, and progress data
* Server deployment on AWS EC2

#### Recommendation algorithm

* Caching latest 'liked' short videos for each topic and unit in Redis
* Super fast progress retrieval from in order to compute the next video on scroll

#### Edison AI

* Used GPT 4.5 + short video transcripts for context-specific answers
* AI chat endpoints deployed along with server on AWS

## Challenges we ran into

* LLM hallucinations
* Bugs with automated video processing
* Getting the video scrolling interface to work smoothly without any sort of flickering or unwanted behavior
* Getting the whole stack working together and deployed before submission deadline

## Accomplishments that we're proud of

* Implementing a custom video scrolling interface for our short videos
* Automatically generating high-quality short videos
* Designing a clean user interface using Figma before building it out
* Getting back-end integrated with google auth + multiple databases + APIs and deploying to cloud
* All of this in < 36 hours!

## What we learned

* Using Redis to enhance a more conventional tech stack
* Deploying FastAPI to AWS EC2 using Gunicorn workers
* Building complex automated video editing functionality using MoviePy
* Utilizing the newly released GPT-4.5 API to perform context retrieval and create high-quality videos.
* Building custom UI Components in Swift and working with Google Auth in an iOS application

## What's next for Relearn

* Expanding curriculum to cover topics for all ages
* Multi-language support to increase accessibility
* Periodic flash cards and quizzes based on progress
* Opportunity for creators to upload their own content directly to Relearn
## Inspiration

AI voices are stale and impersonal. Chrome extensions like "Free Text To Speech Online" use default voices to read text messages on the web out loud. While these default voices excel in cadence and clarity, they miss the nuance and emotion inherent in human speech. This emotional connection is important for a user, as it helps them feel engaged in online communication. Using personalized speech also helps users with special needs who rely on text-to-speech, as this feature assists them in identifying who is talking when vocalizing the messages.

## What it does

TeleSpeech is a Chrome Extension that converts Telegram messages into custom AI-generated speech, mimicking the distinct voice of each sender. Using the chrome extension and the web app, you can upload anyone's voice and use it to read messages out loud in a Telegram group chat.

## How we built it

We used a Chrome Extension (HTML/CSS, Vanilla JS) to read message data and run the text-to-speech, and a Next.js web app to manage the voices used for text-to-speech.

To use TeleSpeech, a user will first upload their voice on our Next.js web app (<https://telespeakto.us>), which will then use the Eleven-Labs Text-to-Speech API to send the AI-generated voice back to the Chrome extension. All user credentials and voice data are securely stored in a Firebase database.

On the Chrome extension, when a user has the Telegram Web App open, the extension's service worker will collect all the messages in a chat. When the Chrome extension is open and a user logs in, a "Play Sound" button appears. When pressed, the Chrome extension sends the web app all the message text, and the web app returns an audio file with an AI-generated voice saying the text data.

## Challenges we ran into

We struggled the most with communicating between the Chrome extension and the web app. Using Vanilla JS with the extension's strict CSP policies made it hard to transfer data between the 2. We also struggled with learning how to use the Eleven-Labs API because we'd never used it before. Finally, two of the members of our team didn't know typescript as well had a decently steep learning curve as we headed into the projects.

## Accomplishments that we're proud of

When we were first able to get one teammate's voice to come out of the speakers reading a message was so incredible. We all thought we could do this project before that happened, but after that, it felt so much more real and attainable. Another is that we built a fully functioning project despite it being our first time at a Hackathon.

## What we learned

Two of the members in the group did not know a lot of JavaScript or typescript going in. The short time was not enough to completely prepare them. But, over the last 36 hours, they were able to figure it out to a higher degree than thought. The other two members learned a lot about how to use Chrome extensions, such as how to use service workers and how to have it communicate with a web app. Besides coding, the four of us also learned a lot about accessibility on screens.

## What's next for TeleSpeech

The next big thing for TeleSpeech is for it to work for multiple platforms, not just Telegram. We want to expand it to WhatsApp, Instagram, and Facebook. It would also be nice if we could use it for news articles, where it would read news articles in the author's voice, or have the articles' quotes be read by the person's voice.