Our smart note taking app revolutionizes the way you take and organize notes. With AI-powered features like auto-summarization and smart semantic search, you can quickly and easily find the information you need. It's perfect for university students and industry professionals alike who want to be more productive and efficient.

## Inspiration

During our lectures, we spend most of our time typing and catching up to what the professor is saying. And after all that typing, we then have to spend time summarising, formating, and cleaning our notes! Not to mention the pain that comes from looking for specific information through dozens of note documents. Taking notes is just way too much time and effort. We wanted to simplify and improve the process of note-taking and give users an easy and efficient way to write, organize, and search through your notes.

## What it does

NoteBuddy allows users to record audio or upload existing audio and auto-generate a transcript for it. You can record lectures, meetings and much more! The transcript is automatically saved to the user's Google Docs and the NoteBuddy platform. The AI-powered auto-summarization feature generates a short summary of the most important points. It also allows for smart semantic text search, which allows you to ask questions and find the answer through all your notes in just a few seconds.

## How we built it

The frontend is built with Vite, JavaScript, and React while the backend is built with python and flask.
We used Appwrite for OAuth 2.0 authentication to use Google's GSuite APIs. We also used Appwrite's database to store data related to the user and their documents.
We used OpenAI's Whisper model to transcribe audio into text and Google's Flan-T5 model to summarize and search through text. Both of these models were deployed to high-performance compute units on the cloud as our machines were not powerful enough.

## Challenges we ran into

The biggest challenge we faced was perfecting semantic document search. To get it to work, we needed to semantically search the documents and answer questions based on the information in the document. We needed to minimize false-positives because we didn't want the user to be directed to a wrong document to get an answer. We were able to overcome false-positives through some clever prompt engineering and model tuning.

## Accomplishments that we're proud of

Seamlessly integrating Google Docs with our platform through Appwrite's OAuth 2.0 service.

## What we learned

We learnt a lot about deploying machine learning models and fine-tuning their parameters to get accurate results to best fit our use case.

## What's next for NoteBuddy

We want to further fine tune the models so they work better for our specific purposes. While we did do some tuning so we could make the models work for our needs, it could still be made more efficient and accurate. We also want to implement more features to allow users to collaborate with each other and make their experience better, such as the ability to import existing documents.
## Inspiration

Our team built an Alexa skill that would engage the user through a simple word game. This project was inspired by a childhood game called First Letter, Last Letter. The objective of this game is for the user to say a word ie.hacker, and for your opponent to say a word that begins with the last letter of the previous word ie.response. As the user continues their success in the game, Alexa will respond faster, prompting the user to increase the speed of their answers.

## What it does

Wake Amazon Alexa and call the skill saying “Alexa, open Word Chain game."
Alexa will explain the instructions of the game, choose a category and start the game with the user.

## How I built it

AWS Lambda, Amazon Alexa Skill, Amazon API Gateway, DynamoDB

## Challenges I ran into

Our team was stuck at the ideation phase for a very long time. We decided to choose to run with creating an Alexa Skill idea on Saturday at 6pm.

## Accomplishments that I'm proud of

We programmed Alexa to have a witty humour.

## What I learned

Don’t stress too much and remember to have fun during a hackathon.

## What's next for Word Chain Game

Publish to Amazon Alexa Skills
## Inspiration

<https://www.youtube.com/watch?v=lxuOxQzDN3Y>
Robbie's story stuck out to me at the endless limitations of technology. He was diagnosed with muscular dystrophy which prevented him from having full control of his arms and legs. He was gifted with a Google home that crafted his home into a voice controlled machine. We wanted to take this a step further and make computers more accessible for people such as Robbie.

## What it does

We use a Google Cloud based API that helps us detect words and phrases captured from the microphone input. We then convert those phrases into commands for the computer to execute. Since the python script is run in the terminal it can be used across the computer and all its applications.

## How I built it

The first (and hardest) step was figuring out how to leverage Google's API to our advantage. We knew it was able to detect words from an audio file but there was more to this project than that. We started piecing libraries to get access to the microphone, file system, keyboard and mouse events, cursor x,y coordinates, and so much more. We build a large (~30) function library that could be used to control almost anything in the computer

## Challenges I ran into

Configuring the many libraries took a lot of time. Especially with compatibility issues with mac vs. windows, python2 vs. 3, etc. Many of our challenges were solved by either thinking of a better solution or asking people on forums like StackOverflow. For example, we wanted to change the volume of the computer using the fn+arrow key shortcut, but python is not allowed to access that key.

## Accomplishments that I'm proud of

We are proud of the fact that we had built an alpha version of an application we intend to keep developing, because we believe in its real-world applications. From a technical perspective, I was also proud of the fact that we were able to successfully use a Google Cloud API.

## What I learned

We learned a lot about how the machine interacts with different events in the computer and the time dependencies involved. We also learned about the ease of use of a Google API which encourages us to use more, and encourage others to do so, too. Also we learned about the different nuances of speech detection like how to tell the API to pick the word "one" over "won" in certain context, or how to change a "one" to a "1", or how to reduce ambient noise.

## What's next for Speech Computer Control

At the moment we are manually running this script through the command line but ideally we would want a more user friendly experience (GUI). Additionally, we had developed a chrome extension that numbers off each link on a page after a Google or Youtube search query, so that we would be able to say something like "jump to link 4". We were unable to get the web-to-python code just right, but we plan on implementing it in the near future.