## Inspiration

It’s no secret that the COVID-19 pandemic ruined most of our social lives. ARoom presents an opportunity to boost your morale by supporting you to converse with your immediate neighbors and strangers in a COVID safe environment.

## What it does

Our app is designed to help you bring your video chat experience to the next level. By connecting to your webcam and microphone, ARoom allows you to chat with people living near you virtually. Coupled with an augmented reality system, our application also allows you to view 3D models and images for more interactivity and fun. Want to chat with new people? Open the map offered by ARoom to discover the other rooms available around you and join one to start chatting!

## How we built it

The front-end was created with Svelte, HTML, CSS, and JavaScript. We used Node.js and Express.js to design the backend, constructing our own voice chat API from scratch. We used VS Code’s Live Share plugin to collaborate, as many of us worked on the same files at the same time. We used the A-Frame web framework to implement Augmented Reality and the Leaflet JavaScript library to add a map to the project.

## Challenges we ran into

From the start, Svelte and A-Frame were brand new frameworks for every member of the team, so we had to devote a significant portion of time just to learn them. Implementing many of our desired features was a challenge, as our knowledge of the programs simply wasn’t comprehensive enough in the beginning. We encountered our first major problem when trying to implement the AR interactions with 3D models in A-Frame. We couldn’t track the objects on camera without using markers, and adding our most desired feature, interactions with users was simply out of the question. We tried to use MediaPipe to detect the hand’s movements to manipulate the positions of the objects, but after spending all of Friday night working on it we were unsuccessful and ended up changing the trajectory of our project.
Our next challenge materialized when we attempted to add a map to our function. We wanted the map to display nearby rooms, and allow users to join any open room within a certain radius. We had difficulties pulling the location of the rooms from other files, as we didn’t understand how Svelte deals with abstraction. We were unable to implement the search radius due to the time limit, but we managed to add our other desired features after an entire day and night of work.
We encountered various other difficulties as well, including updating the rooms when new users join, creating and populating icons on the map, and configuring the DNS for our domain.

## Accomplishments that we're proud of

Our team is extremely proud of our product, and the effort we’ve put into it. It was ¾ of our members’ first hackathon, and we worked extremely hard to build a complete web application. Although we ran into many challenges, we are extremely happy that we either overcame or found a way to work around every single one. Our product isn’t what we initially set out to create, but we are nonetheless delighted at its usefulness, and the benefit it could bring to society, especially to people whose mental health is suffering due to the pandemic. We are also very proud of our voice chat API, which we built from scratch.

## What we learned

Each member of our group has learned a fair bit over the last 36 hours. Using new frameworks, plugins, and other miscellaneous development tools allowed us to acquire heaps of technical knowledge, but we also learned plenty about more soft topics, like hackathons and collaboration. From having to change the direction of our project nearly 24 hours into the event, we learned that it’s important to clearly define objectives at the beginning of an event. We learned that communication and proper documentation is essential, as it can take hours to complete the simplest task when it involves integrating multiple files that several different people have worked on. Using Svelte, Leaflet, GitHub, and Node.js solidified many of our hard skills, but the most important lessons learned were of the other variety.

## What's next for ARoom

Now that we have a finished, complete, usable product, we would like to add several features that were forced to remain in the backlog this weekend. We plan on changing the map to show a much more general location for each room, for safety reasons. We will also prevent users from joining rooms more than an arbitrary distance away from their current location, to promote a more of a friendly neighborhood vibe on the platform. Adding a video and text chat, integrating Google’s Translation API, and creating a settings page are also on the horizon.
## Inspiration

BThere emerged from a genuine desire to strengthen friendships by addressing the subtle challenges in understanding friends on a deeper level. The team recognized that nuanced conversations often go unnoticed, hindering meaningful support and genuine interactions. In the context of the COVID-19 pandemic, the shift to virtual communication intensified these challenges, making it harder to connect on a profound level. Lockdowns and social distancing amplified feelings of isolation, and the absence of in-person cues made understanding friends even more complex. BThere aims to use advanced technologies to overcome these obstacles, fostering stronger and more authentic connections in a world where the value of meaningful interactions has become increasingly apparent.

## What it does

BThere is a friend-assisting application that utilizes cutting-edge technologies to analyze conversations and provide insightful suggestions for users to connect with their friends on a deeper level. By recording conversations through video, the application employs Google Cloud's facial recognition and speech-to-text APIs to understand the friend's mood, likes, and dislikes. The OpenAI API generates personalized suggestions based on this analysis, offering recommendations to uplift a friend in moments of sadness or providing conversation topics and activities for neutral or happy states. The backend, powered by Python Flask, handles data storage using Firebase for authentication and data persistence. The frontend is developed using React, JavaScript, Next.js, HTML, and CSS, creating a user-friendly interface for seamless interaction.

## How we built it

BThere involves a multi-faceted approach, incorporating various technologies and platforms to achieve its goals. The recording feature utilizes WebRTC for live video streaming to the backend through sockets, but also allows users to upload videos for analysis. Google Cloud's facial recognition API identifies facial expressions, while the speech-to-text API extracts spoken content. The combination of these outputs serves as input for the OpenAI API, generating personalized suggestions. The backend, implemented in Python Flask, manages data storage in Firebase, ensuring secure authentication and persistent data access. The frontend, developed using React, JavaScript, Next.js, HTML, and CSS, delivers an intuitive user interface.

## Accomplishments that we're proud of

* Successfully integrating multiple technologies into a cohesive and functional application
* Developing a user-friendly frontend for a seamless experience
* Implementing real-time video streaming using WebRTC and sockets
* Leveraging Google Cloud and OpenAI APIs for advanced facial recognition, speech-to-text, and suggestion generation

## What's next for BThere

* Continuously optimizing the speech to text and emotion analysis model for improved accuracy with different accents, speech mannerisms, and languages
* Exploring advanced natural language processing (NLP) techniques to enhance conversational analysis
* Enhancing user experience through further personalization and more privacy features
* Conducting user feedback sessions to refine and expand the application's capabilities
## Saving animals, one step at a time

## Inspiration

Over the course of our lifetimes, we've seen many species go from being at-risk to being endangered to becoming extinct because of poaching. Rangers simply cannot patrol the large reserves under their jurisdiction to prevent poachers from hunting these endangered animals.

## What it does

PoachNot uses microphones set up at multiple points in a wildlife reserve to monitor the area for signs of poaching activity. After hearing gunshot sounds, it uses the microphones to pinpoint the location of the poacher and alerts the authorities so that the poacher can be caught.

## How we built it

We built a dashboard on React which communicates with a Python backend using FastAPI. Our frontend polls the backend to detect whether there are gunshots nearby. We trained a model on PyTorch using data labelled on Scale's Rapid platform which we use to distinguish between gunshots and other sounds. We constructed our features from each audio file by constructing a spectrogram of the sound signal. We also trained a model on Mage which ascribes a probability of a poacher being present in a given location given historic data, in order to ascertain we have not reached a false-positive.

## Challenges we ran into

We had an authentication platform that could allow correct access to the right people to use the platform. We ended up using Docker and Redis as our database holders to keep track of all this information. We faced some difficulties setting up our Docker environment and some of our python modules would not be recognized despite having been included in the requirements.txt and installed in the environment manually.

## Accomplishments that we're proud of

We're proud of the accuracy we achieved with the limited amount of data we had in distinguishing between gunshots and other sounds. We achieved an accuracy of over 95%.

## What we learned

On the technical side, we learned about the plethora of available platforms that allow us to perform our application development much more efficiently, including ScaleAI and Mage's API integrations. In a broader sense, we learned how an idea can be used for a variety of applications, as described in the next section.

## What's next for PoachNot

Detecting gunshots and identifying the relevant authorities in real-time is not unique to poaching. It has many other applications in urban society. For example, Berkeley had a shooting scare on Friday which resulted in many people getting hurt. If the police had been notified immediately, they would have been able to proactively prevent people from panicking. Additionally, alarms could be installed in public spaces that respond to a shooting alert, this way people nearby the scene of incidence can be notified about the danger without the need for someone to manually report the incident once they reach a safe space. Moreover, the authenticity of a real gunshot vs a mock noise can be distinguished using our platform.