## Inspiration

Public speaking is an incredibly important skill that many seek but few master. This is in part due to the high level of individualized attention and feedback needed to improve when practicing. Therefore, we want to solve this with AI! We have created a VR application that allows you to get constructive feedback as you present, debate, or perform by analyzing your arguments and speaking patterns.
While this was our starting motivation for ArticuLab, we quickly noticed the expansive applications and social impact opportunities for it. ArticuLab could be used by people suffering from social anxiety to help improve their confidence in speaking in front of crowds and responding to contrasting opinions. It could also be used by people trying to become more fluent in a language, since it corrects pronunciation and word choice.

## What it does

ArticuLab uses AI in a VR environment to recommend changes to your pace, argument structure, clarity, and boy language when speaking. It holds the key to individualized public speaking practice. In ArticuLab you also have the opportunity to debate directly against AI, who'll point out all the flaws in your arguments and make counterarguments so you can make your defense rock-solid.

## How we built it

For our prototype, we used Meta's Wit.AI natural language processing software for speech recognition, built a VR environment on Unity, and used OpenAI's powerful ChatGPT to base our feedback system on argument construction and presenting ability. Embedding this into an integrated VR App results in a seamless, consumer-ready experience.

## Challenges we ran into

The biggest challenge we ran into is using the VR headset microphone as input for the speech recognition software, and then directly inputting that to our AI system. What made this so difficult was adapting the formatting from each API onto the next. Within the same thread, we ran into an issue where the microphone input would only last for a few seconds, limiting the dialogue between the user and the AI in a debate. These issues were also difficult to test because of the loud environment we were working in.
Additionally, we had to create a VR environment from scratch, since there were no free assets to fit our needs.

## Accomplishments that we're proud of

We're especially proud of accomplishing such an ambitious project with a team that is majority beginners! Treehacks is three of our integrants' first hackathon, so everyone had to step up and do more work or learn more new skills to implement in our project.

## What we learned

We learned a lot about speech to text software, designing an environment and programming in Unity, adapting the powerful ChatGPT to our needs, and integrating a full-stack VR application.

## What's next for ArticuLab

Naturally, there would be lots more polishing of the cosmetics and user interface of the program, which are currently restricted by financial resources and the time available. Among these, would be making the environment a higher definition with better quality assets, crowd responses, ChatGPT responses with ChatGPT plus, etc.
ArticuLab could be useful both academically and professionally in a variety of fields, education, project pitches like Treehacks, company meetings, event organizers… the list goes on! We would also seek to expand the project to alternate versions adapted for the comfort of the users, for example, a simplified iOS version could be used by public speakers to keep notes on their speech and let them know if they're speaking too fast, too slow, or articulating correctly live! Similarly, such a feature would be integrated into the VR version, so a presenter could have notes on their podium and media to present behind them (powerpoint, video, etc.), simulating an even more realistic presenting experience.
Another idea is adding a multiplayer version that would exponentially expand the uses for ArticuLab. Our program could allow debate teams to practice live in front of a mix of AI and real crowds, similarly, ArticuLab could host online live debates between public figures and politicians in the VR environment.
## Inspiration

Many of us have a hard time preparing for interviews, presentations, and any other social situation. We wanted to sit down and have a real talk... with ourselves.

## What it does

The app will analyse your speech, hand gestures, and facial expressions and give you both real-time feedback as well as a complete rundown of your results after you're done.

## How We built it

We used Flask for the backend and used OpenCV, TensorFlow, and Google Cloud speech to text API to perform all of the background analyses. In the frontend, we used ReactJS and Formidable's Victory library to display real-time data visualisations.

## Challenges we ran into

We had some difficulties on the backend integrating both video and voice together using multi-threading. We also ran into some issues with populating real-time data into our dashboard to display the results correctly in real-time.

## Accomplishments that we're proud of

We were able to build a complete package that we believe is purposeful and gives users real feedback that is applicable to real life. We also managed to finish the app slightly ahead of schedule, giving us time to regroup and add some finishing touches.

## What we learned

We learned that planning ahead is very effective because we had a very smooth experience for a majority of the hackathon since we knew exactly what we had to do from the start.

## What's next for RealTalk

We'd like to transform the app into an actual service where people could log in and save their presentations so they can look at past recordings and results, and track their progress over time. We'd also like to implement a feature in the future where users could post their presentations online for real feedback from other users. Finally, we'd also like to re-implement the communication endpoints with websockets so we can push data directly to the client rather than spamming requests to the server.

![Image](https://i.imgur.com/aehDk3L.gif)

Tracks movement of hands and face to provide real-time analysis on expressions and body-language.

![Image](https://i.imgur.com/tZAM0sI.gif)
## Inspiration

With our inability to acquire an oculus rift at YHack, we were looking forward to using one in any implementation. As we are entering the job market soon, in search of summer internships, we thought about how many people, students in particular, do not have sufficient interviewing experience. Thus, our idea was to use VR to provide these people with the ability to train and succeed with interviews.

## What it does

This hack simulates an interview scenario to aid in the practice of interviewing skills and the removal of improper speech patterns using Unity, Oculus and IBM Watson.

## How I built it

Using Unity, starting and interview scenes were created and animations were implemented. The backend of the speech processing system was built with IBM Watson Unity tools. These tools allowed for the integration for IBM speech to text api's and allow the user's speech to be converted into text and later processed for the output.

## Challenges I ran into

While implementing IBM Watson speech to text, Oculus Rift compatibility, and Unity Scenes, we came across a few errors. Firstly, we had been working in different versions of unity, so when the time came to compile our projects together there were compatibility issues.

## Accomplishments that I'm proud of

One things we are proud of how we overcame the many challenges that arose during this project. We were also proud of the overall implementation of the different facets of the hack and how we were able to mesh them all together.

## What I learned

All of us had to learn how to use and implement virtual reality as none of us had the opportunity to work with it before. We also learned a lot about Unity and implementing items such as VR and IBM Watson speech to text.

## What's next for InterVR

Implementing more thorough testing on the speech to text file would be the first large improvement that could be made for InterVR. Another improvement that could be made for InterVR is a more diverse cast of interviewers, additional scenes for the interviews, and more questions to allow the user to have a more specialized experience.