## Inspiration

Public speaking is an incredibly important skill that many seek but few master. This is in part due to the high level of individualized attention and feedback needed to improve when practicing. Therefore, we want to solve this with AI! We have created a VR application that allows you to get constructive feedback as you present, debate, or perform by analyzing your arguments and speaking patterns.
While this was our starting motivation for ArticuLab, we quickly noticed the expansive applications and social impact opportunities for it. ArticuLab could be used by people suffering from social anxiety to help improve their confidence in speaking in front of crowds and responding to contrasting opinions. It could also be used by people trying to become more fluent in a language, since it corrects pronunciation and word choice.

## What it does

ArticuLab uses AI in a VR environment to recommend changes to your pace, argument structure, clarity, and boy language when speaking. It holds the key to individualized public speaking practice. In ArticuLab you also have the opportunity to debate directly against AI, who'll point out all the flaws in your arguments and make counterarguments so you can make your defense rock-solid.

## How we built it

For our prototype, we used Meta's Wit.AI natural language processing software for speech recognition, built a VR environment on Unity, and used OpenAI's powerful ChatGPT to base our feedback system on argument construction and presenting ability. Embedding this into an integrated VR App results in a seamless, consumer-ready experience.

## Challenges we ran into

The biggest challenge we ran into is using the VR headset microphone as input for the speech recognition software, and then directly inputting that to our AI system. What made this so difficult was adapting the formatting from each API onto the next. Within the same thread, we ran into an issue where the microphone input would only last for a few seconds, limiting the dialogue between the user and the AI in a debate. These issues were also difficult to test because of the loud environment we were working in.
Additionally, we had to create a VR environment from scratch, since there were no free assets to fit our needs.

## Accomplishments that we're proud of

We're especially proud of accomplishing such an ambitious project with a team that is majority beginners! Treehacks is three of our integrants' first hackathon, so everyone had to step up and do more work or learn more new skills to implement in our project.

## What we learned

We learned a lot about speech to text software, designing an environment and programming in Unity, adapting the powerful ChatGPT to our needs, and integrating a full-stack VR application.

## What's next for ArticuLab

Naturally, there would be lots more polishing of the cosmetics and user interface of the program, which are currently restricted by financial resources and the time available. Among these, would be making the environment a higher definition with better quality assets, crowd responses, ChatGPT responses with ChatGPT plus, etc.
ArticuLab could be useful both academically and professionally in a variety of fields, education, project pitches like Treehacks, company meetings, event organizers… the list goes on! We would also seek to expand the project to alternate versions adapted for the comfort of the users, for example, a simplified iOS version could be used by public speakers to keep notes on their speech and let them know if they're speaking too fast, too slow, or articulating correctly live! Similarly, such a feature would be integrated into the VR version, so a presenter could have notes on their podium and media to present behind them (powerpoint, video, etc.), simulating an even more realistic presenting experience.
Another idea is adding a multiplayer version that would exponentially expand the uses for ArticuLab. Our program could allow debate teams to practice live in front of a mix of AI and real crowds, similarly, ArticuLab could host online live debates between public figures and politicians in the VR environment.
## Inspiration

Knowtworthy is a startup that all three of us founded together, with the mission to make meetings awesome. We have spent this past summer at the University of Toronto’s Entrepreneurship Hatchery’s incubator executing on our vision. We’ve built a sweet platform that solves many of the issues surrounding meetings but we wanted a glimpse of the future: entirely automated meetings. So we decided to challenge ourselves and create something that the world has never seen before: sentiment analysis for meetings while transcribing and attributing all speech.

## What it does

While we focused on meetings specifically, as we built the software we realized that the applications for real-time sentiment analysis are far more varied than initially anticipated. Voice transcription and diarisation are very powerful for keeping track of what happened during a meeting but sentiment can be used anywhere from the boardroom to the classroom to a psychologist’s office.

## How I built it

We felt a web app was best suited for software like this so that it can be accessible to anyone at any time. We built the frontend on React leveraging Material UI, React-Motion, Socket IO and ChartJS. The backed was built on Node (with Express) as well as python for some computational tasks. We used GRPC, Docker and Kubernetes to launch the software, making it scalable right out of the box.

For all relevant processing, we used Google Speech-to-text, Google Diarization, Stanford Empath, SKLearn and Glove (for word-to-vec).

## Challenges I ran into

Integrating so many moving parts into one cohesive platform was a challenge to keep organized but we used trello to stay on track throughout the 36 hours.
Audio encoding was also quite challenging as we ran up against some limitations of javascript while trying to stream audio in the correct and acceptable format.
Apart from that, we didn’t encounter any major roadblocks but we were each working for almost the entire 36-hour stretch as there were a lot of features to implement.

## Accomplishments that I'm proud of

We are super proud of the fact that we were able to pull it off as we knew this was a challenging task to start and we ran into some unexpected roadblocks. There is nothing else like this software currently on the market so being first is always awesome.

## What I learned

We learned a whole lot about integration both on the frontend and the backend. We prototyped before coding, introduced animations to improve user experience, too much about computer store numbers (:p) and doing a whole lot of stuff all in real time.

## What's next for Knowtworthy Sentiment

Knowtworthy Sentiment aligns well with our startup’s vision for the future of meetings so we will continue to develop it and make it more robust before integrating it directly into our existing software. If you want to check out our stuff you can do so here: <https://knowtworthy.com/>
## Inspiration

We were debating about the classic marvel vs dc and then it struck to us, why not let other fans join our debate, "virtually" to the same debate, and thus came the idea for DebateX.

## What it does

DebateX is a Google Cardboard compatible VR App which let's users join debate rooms of their choice and express their ideas with others, it even let's users witness other debates as audience.

The app also provides a space to practice one's speaking skills and also helps overcome the fear of public speaking by simulating a live audience.

The app consists of three modes right now

**1) Debate mode -**

In this mode the user connects to a debate room on a podium along with a fellow debator. This room can be joined by the spectators in real time to witness the debate.

**2) Spectator mode -**

The user can attend the ongoing debates or conversations and be a silent spectator for the same.

**3) Practice Mode -**

In this mode you will be put in front of a live class so that you can practice your speaking skills and develop that confidence and skill.

The overall objective of this project is to provide a virtual Reality environment which is easily accessible by anyone and it offers a way to effectively improve your speaking skills. We also integrated this app with a Google speech to text API which will inform you about your talking speed and will give you a feedback on how frequently you use emphatic words.(Proposed with proof of concept ready)

## How we built it

So we started building the project on Unity on a Light Weight Render Pipeline to involve better graphics for the app. We first made the main screen with 3 options to choose from. We built custom shaders for the options and added post processing to them. We added GoogleVR Unit package to the project and made the necessary changes. On testing we found out that there was a bug with LWRP and GVR when used together so we had to scrape that part and start over in a normal 3d environment.

After finalizing the UI part, we moved on to the next major milestone, that is, adding realtime voice communication from the Vr app.
To do so we used the Agora Voice SDK. After going through the documentation we added the SDK to Unity and tested it out on 2 mobile devices.
No bugs were found and we moved on to the multiplayer part of the app. We had 2 options, UNet framework which would host our app on Unity Servers,
or building our own server. Since we were keen to learn new stuff we decided to do the latter. The server used sockets as a major part of the communication.Sockets.io package was also added to Unity. We made custom events for the same and deployed the server on our device.
We faced some issue while syncing the server with the application and it did take a while to resolve them. Final changes to UI were made and the app was then tested on Google Cardboard. ( Super grateful to the organizers to arrange a pair for us )

## Challenges we ran into

Lack of feasible hardware.

Making it accessible for everyone

Lack of professional textures and 3d models readily available.

## Accomplishments that we are proud of

Implemented Agora Voice SDK in unity.

The entire app is built on google cardboard making it very accessible for the general public.

Making our own socket.io server on NodeJS which runs in real time.

## What's next for DebateX

We have laid the foundation for a new mode of communication and in the future it can be integrated with better VR hardware to involve immersive features like voice modulation , hand gestures , stage movement , etc. The future looks really promising.