## Inspiration

Almost all undergraduate students, especially at large universities like the University of California Berkeley, will take a class that has a huge lecture format, with several hundred students listening to a single professor speak. At Berkeley, students (including three of us) took CS61A, the introductory computer science class, alongside over 2000 other students. Besides forcing some students to watch the class on webcasts, the sheer size of classes like these impaired the ability of the lecturer to take questions from students, with both audience and lecturer frequently unable to hear the question and notably the question not registering on webcasts at all. This led us to seek out a solution to this problem that would enable everyone to be heard in a practical manner.

## What does it do?

*Questions?* solves this problem using something that we all have with us at all times: our phones. By using a peer to peer connection with the lecturer’s laptop, a student can speak into their smartphone’s microphone and have that audio directly transmitted to the audio system of the lecture hall. This eliminates the need for any precarious transfer of a physical microphone or the chance that a question will be unheard.
Besides usage in lecture halls, this could also be implemented in online education or live broadcasts to allow participants to directly engage with the speaker instead of feeling disconnected through a traditional chatbox.

## How we built it

We started with a fail-fast strategy to determine the feasibility of our idea. We did some experiments and were then confident that it should work. We split our working streams and worked on the design and backend implementation at the same time. In the end, we had some time to make it shiny when the whole team worked together on the frontend.

## Challenges we ran into

We tried the WebRTC protocol but ran into some problems with the implementation and the available frameworks and the documentation. We then shifted to WebSockets and tried to make it work on mobile devices, which is easier said than done. Furthermore, we had some issues with web security and therefore used an AWS EC2 instance with Nginx and let's encrypt TLS/SSL certificates.

## Accomplishments that we're (very) proud of

With most of us being very new to the Hackathon scene, we are proud to have developed a platform that enables collaborative learning in which we made sure whatever someone has to say, everyone can hear it.
With *Questions?* It is not just a conversation between a student and a professor in a lecture; it can be a discussion between the whole class. *Questions?* enables users’ voices to be heard.

## What we learned

WebRTC looks easy but is not working … at least in our case. Today everything has to be encrypted … also in dev mode. Treehacks 2020 was fun.

## What's next for *Questions?*

In the future, we could integrate polls and iClicker features and also extend functionality for presenters and attendees at conferences, showcases, and similar events. \_ Questions? \_ could also be applied even broader to any situation normally requiring a microphone—any situation where people need to hear someone’s voice.
## Inspiration

As the Coronavirus pandemic continues to impact our lives, students are forced to stay at home and deal with the difficulties that come with online learning.

Personally, we have struggled with connection issues, professors who speak unclearly, and noisy environments. We can only imagine what lectures are like for students who have a language barrier, struggle with hearing impairment, or do not have access to a quiet and comfortable learning environment.

For our hack we wanted to tackle this problem and create a tool that helps improve learning experiences and make classes more accessible for struggling students, with hopes of making a positive social impact by helping people communicate during a time filled with challenges and uncertainty.

## What it does

EasyCC is a chrome extension that provides real-time closed captioning for any audio source running from your computer. EasyCC supports all platforms including Zoom, Collaborate Ultra, Discord, Google Meet, and can even transcribe Youtube videos!

## How we built it

We first prototyped the UI in Figma and developed the front-end for the chrome extension using HTML and CSS. Using Node.js, we then integrated tools that allowed us to capture audio from the desktop and process speech into text using Google Cloud’s Speech-to-text engine. Using socket.io, we relayed the transcripts to our front-end to be displayed in real-time for the user.

## Challenges we ran into

Most of the issues that we ran into were related to the backend and its integration. In particular, setting up our software architecture was challenging because we need to continuously pass large amounts of data from the backend to the frontend, which requires us to have a good understanding of how the web works and how each component interacts with each other. Since calling the Google Speech To Text API must be done in the backend, we had to effectively integrate it to the frontend so that the transcribed messages are displayed approximately in real time. The main hurdle was the lag due to the constant calls between the frontend and the backend, which required us to integrate Socket.io into our codebase, another feat in and of itself. Initially, the audio stream would not record which we discovered to be a permission issue outside of our code, so we had to address that issue in order for the Google Speech To Text API to work. Oftentimes, the documentation for the APIs are hard to understand due to a lack of explanations and examples, so we had to engage in some trial and error and adapt the code to meet our needs in the application.

## Accomplishments that we're proud of

We are proud to have created an application that improves the experience of online lectures using off the shelf technology. We wanted to keep the application straightforward so that we can have it running quickly. Despite having little familiarity with web development and chrome extensions, we managed to create a frontend and backend, and more importantly, link these two together to create a functional application. In the process, we gained exposure to relevant web technologies and picked up researching skills, which is critical to software development. Also, we collaborated effectively to polish our ideas, offer different approaches to solving complex problems, and complement each other’s skills. Finally, we learned how to seek help from mentors effectively, being able to identify issues beyond the scope of our knowledge and research, and using the pointers they provided to devise an effective solution.

## What we learned

Since we all had little experience with web development, this was our first time using the relevant technologies in an integrated manner. In particular, connecting the frontend and the backend is a major challenge that we are proud to have completed, enabling us to better understand the architecture of web applications.

We learned a lot about the services we used, namely Chrome Extensions, Google Speech-To-Text API, Socket.io. This was our first time using these resources, and we are very happy with how we used them in our application.

Since our program is constantly communicating between the frontend and the backend, we decided to use Socket.io to facilitate these interactions as it is designed for instant messaging. This vastly improves the performance when displaying the transcribed message on the overlay compared to constantly making HTTP calls. Error diagnosis is a constant thing we dealt with when developing software, especially when incorporating unfamiliar APIs to our codebase. In particular, although the Google Speech to Text API seemed imposing upon first glance, we are able to read through the documentation, understand what the code is doing, and identify errors preventing the service from running correctly. This was a great experience to us and we have been exposed to several great services during this hackathon.

## What's next for EasyCC

EasyCC has a lot of potential to become a viable captioning service. We hope to add features that will improve our extension and make it even more accessible and useful. For one, we would like to use a translation API, which will connect users all over the world, allowing them to communicate and understand different languages. We could also potentially publish EasyCC onto the Chrome Web Store, so that our service is readily available to anybody.
## Inspiration

How long does class attendance take? 3 minutes? With 60 classes, 4 periods a day, and 180 school days in a year, this program will save a cumulative 72 days every year! Our team recognized that the advent of neural networks yields momentous potential, and one such opportunity is face recognition. We utilized this cutting-edge technology to save time in regards to attendance.

## What it does

The program uses facial recognition to determine who enters and exits the room. With this knowledge, we can keep track of everyone who is inside, everyone who is outside, and the unrecognized people that are inside the room. Furthermore, we can display all of this on a front end html application.

## How I built it

A camera that is mounted by the door sends a live image feed to Raspberry pi, which then transfers that information to Flask. Flask utilizes neural networks and machine learning to study previous images of faces, and when someone enters the room, the program matches the face to a person in the database. Then, the program stores the attendees in the room, the people that are absent, and the unrecognized people. Finally, the front end program uses html, css, and javascript to display the live video feed, the people that are attending or absent, the faces of all unrecognized people.

## Challenges I ran into

When we were using the AWS, we uploaded to the bucket, and that triggered a Lamda. In short, we had too many problematic middle-men, and this was fixed by removing them and communicating directly. Another issue was trying to read from cameras that are not designed for Raspberry pi. Finally, we accidentally pushed the wrong html2 file, causing a huge merge conflict problem.

## Accomplishments that I'm proud of

We were successfully able to integrate neural networks with Flask to recognize faces. We were also able to make everything much more efficient than before.

## What I learned

We learned that it is often better to directly communicate with the needed software. There is no point in having middlemen unless they have a specific use. Furthermore, we also improved our server creating skills and gained many valuable insights. We also taught a team member how to use GIT and how to program in html.

## What's next for Big Brother

We would like to match inputs from external social media sites so that unrecognized attendees could be checked into an event. We also would like to export CSV files that display the attendees, their status, and unrecognized people.