## Inspiration

At the University of Toronto, accessibility services are always in demand of more volunteer note-takers for students who are unable to attend classes. Video lectures are not always available and most profs either don't post notes, or post very imprecise, or none-detailed notes. Without a doubt, the best way for students to learn is to attend in person, but what is the next best option? That is the problem we tried to tackle this weekend, with notepal. Other applications include large scale presentations such as corporate meetings, or use for regular students who learn better through visuals and audio rather than note-taking, etc.

## What it does

notepal is an automated note taking assistant that uses both computer vision as well as Speech-To-Text NLP to generate nicely typed LaTeX documents. We made a built-in file management system and everything syncs with the cloud upon command. We hope to provide users with a smooth, integrated experience that lasts from the moment they start notepal to the moment they see their notes on the cloud.

## Accomplishments that we're proud of

Being able to integrate so many different services, APIs, and command-line SDKs was the toughest part, but also the part we tackled really well. This was the hardest project in terms of the number of services/tools we had to integrate, but a rewarding one nevertheless.

## What's Next

* Better command/cue system to avoid having to use direct commands each time the "board" refreshes.
* Create our own word editor system so the user can easily edit the document, then export and share with friends.

## See For Your Self

Primary: <https://note-pal.com>
Backup: <https://danielkooeun.lib.id/notepal-api@dev/>
## Inspiration

We're 4 college freshmen that were expecting new experiences with interactive and engaging professors in college; however, COVID-19 threw a wrench in that (and a lot of other plans). As all of us are currently learning online through various video lecture platforms, we found out that these lectures sometimes move too fast or are just flat-out boring. Summaread is our solution to transform video lectures into an easy-to-digest format.

## What it does

"Summaread" automatically captures lecture content using an advanced AI NLP pipeline to automatically generate a condensed note outline. All one needs to do is provide a YouTube link to the lecture or a transcript and the corresponding outline will be rapidly generated for reading. Summaread currently generates outlines that are shortened to about 10% of the original transcript length. The outline can also be downloaded as a PDF for annotation purposes. In addition, our tool uses the Google cloud API to generate a list of Key Topics and links to Wikipedia to encourage further exploration of lecture content.

## How we built it

Our project is comprised of many interconnected components, which we detail below:

**Lecture Detection**

Our product is able to automatically detect when lecture slides change to improve the performance of the NLP model in summarizing results. This tool uses the Google Cloud Platform API to detect changes in lecture content and records timestamps accordingly.

**Text Summarization**

We use the Hugging Face summarization pipeline to automatically summarize groups of text that are between a certain number of words. This is repeated across every group of text previous generated from the Lecture Detection step.

**Post-Processing and Formatting**

Once the summarized content is generated, the text is processed into a set of coherent bullet points and split by sentences using Natural Language Processing techniques. The text is also formatted for easy reading by including “sub-bullet” points that give a further explanation into the main bullet point.

**Key Concept Suggestions**

To generate key concepts, we used the Google Cloud Platform API to scan over the condensed notes our model generates and provide wikipedia links accordingly. Some examples of Key Concepts for a COVID-19 related lecture would be medical institutions, famous researchers, and related diseases.

**Front-End**

The front end of our website was set-up with Flask and Bootstrap. This allowed us to quickly and easily integrate our Python scripts and NLP model.

## Challenges we ran into

1. Text summarization is extremely difficult -- while there are many powerful algorithms for turning articles into paragraph summaries, there is essentially nothing on shortening conversational sentences like those found in a lecture into bullet points.
2. Our NLP model is quite large, which made it difficult to host on cloud platforms

## Accomplishments that we're proud of

1) Making a multi-faceted application, with a variety of machine learning and non-machine learning techniques.

2) Working on an unsolved machine learning problem (lecture simplification)

3) Real-time text analysis to determine new elements

## What we learned

1) First time for multiple members using Flask and doing web development

2) First time using Google Cloud Platform API

3) Running deep learning models makes my laptop run very hot

## What's next for Summaread

1) Improve our summarization model through improving data pre-processing techniques and decreasing run time

2) Adding more functionality to generated outlines for better user experience

3) Allowing for users to set parameters regarding how much the lecture is condensed by
## Inspiration

All of us were talking about how we had our lessons moved online, and how much of a struggle it was to try to keep up with the lecturers to take down notes for every single lesson. We wanted to build something that can automate the task of creating notes for us so that we can save time and focus on understanding the content instead.

## What it does

VD Notes converts all speech from the video/audio file recorded into text that can then be saved and opened using any text editor of choice. It can also translate the file into any language of choice.

## Challenges we ran into

We scrapped our previous idea 8 hours into the hackathon due to time and technical constraints, and barely made it in time for the deadline with VD Notes.

## Accomplishments that we're proud of

The project can be used right now and will be beneficial for everyone watching online videos, not just for students per se. We are also proud that we managed to get the project through, as we thought we might not make it when it was near submission time.

## What we learned

We learnt how to work together under a very short time constraint. We also learnt how to quickly come up with different problem statements and come up with solutions by bouncing ideas between team members as well as utilising each members' strengths to get the project done.

## What's next for VD Notes

Adding more new features like timestamps, organizing text into paragraphs, as well as being able to detect and highlight key phrases.