## **Inspiration**

Speech language pathologists (SLPs) are medical professionals that offer speech therapy to individuals who experience speech and language disorders. A typical initial SLP assessment is broken down into several sessions due to the volume of tests needed to fully assess the person's possible speech and language disorder. What if there is a program to screen individuals before taking an initial SLP assessment?

## **What it does**

**slp.ai** helps streamline the assessment process by giving an initial screening test. This program focuses on testing elementary-aged (3-7 years old) children on their cognitive level and their ability to speak in a coherent manner. The test asks for a child to describe an image and returns a probability of how accurate the child's description is. Based on the result, a SLP can analyze whether or not further testing is necessary.

## **How we built it**

In the test, the child is asked to describe an image. The program analyzes the recording and turns it into text. Then, we use Co:Here's classify algorithm to check the accuracy and coherency of the recording sample. This algorithm returns the relevance of the child's response, which an SLP can use to determine if an assessment is required.

## **Challenges**

Some challenges that we faced include learning how to use python’s speech recognition module and Google Cloud’s speech-to-text services. We manually built test cases for Co:Here’s clarify algorithm so that it can intelligently learn how to determine the accuracy of a child’s description of the image. Our main difficulty was the high learning curve when building and designing front-end and back-end, figuring out how to merge both of them, and incorporating all the different programs into the web application.

## **Accomplishments**

We learned so much about how to create a fully functional web application. Coming from a team with little to no experience in web development and what even a hackathon was, we are so proud of what we built in the little time that we had.

## **What we learned**

We learned

* how to use front-end languages and tools, such as React.js, HTML, CSS, etc
* how to delegate segments of the project
* how to think outside of the box to find creative solutions to tough problems
* how to work together as a team

## **What's next for slp.ai?**

Our hope is to further develop slp.ai so that there will be more types of tests available, including sections of an actual SLP assessment, such as the OWLS test. Since our program only targets the 3-7 years old age range, we also hope to expand to include initial screenings for other age groups.
## Inspiration

As we all know the world has come to a halt in the last couple of years. Our motive behind this project was to help people come out of their shells and express themselves. Connecting various people around the world and making them feel that they are not the only ones fighting this battle was our main objective.

## What it does

People share their thought processes through speaking and our application identifies the problem the speaker is facing and connects him to a specialist in that particular domain so that his/her problem could be resolved. There is also a **Group Chat** option available where people facing similar issues can discuss their problems among themselves.
For example, if our application identifies that the topics spoken by the speaker are related to mental health, then it connects them to a specialist in the mental health field and also the user has an option to get into a group discussion which contains people who are also discussing mental health.

## How we built it

The front-end of the project was built by using HTML, CSS, Javascript, and Bootstrap. The back-end part was exclusively written in python and developed using the Django framework. We integrated the **Assembly AI** file created by using assembly ai functions to our back-end and were successful in creating a fully functional web application within 36 hours.

## Challenges we ran into

The first challenge was to understand the working of Assembly AI. None of us had used it before and it took us time to first understand it's working. Integrating the audio part into our application was also a major challenge. Apart from Assembly AI, we also faced issues while connecting our front-end to the back-end. Thanks to the internet and the mentors of **HackHarvard**, especially **Assembly AI Mentors** who were very supportive and helped us resolve our errors.

## Accomplishments that we're proud of

Firstly, we are proud of creating a fully functional application within 36 hours taking into consideration all the setbacks we had. We are also proud of building an application from which society can be benefitted. Finally and mainly we are proud of exploring and learning new things which is the very reason for hackathons.

## What we learned

We learned how working as a team can do wonders. Working under a time constraint can be a really challenging task, aspects such as time management, working under pressure, the never give up attitude and finally solving errors which we never came across are some of the few but very important things which we were successful in learning.
## Inspiration

Alex K's girlfriend Allie is a writer and loves to read, but has had trouble with reading for the last few years because of an eye tracking disorder. She now tends towards listening to audiobooks when possible, but misses the experience of reading a physical book.

Millions of other people also struggle with reading, whether for medical reasons or because of dyslexia (15-43 million Americans) or not knowing how to read. They face significant limitations in life, both for reading books and things like street signs, but existing phone apps that read text out loud are cumbersome to use, and existing "reading glasses" are thousands of dollars!

Thankfully, modern technology makes developing "reading glasses" much cheaper and easier, thanks to advances in AI for the software side and 3D printing for rapid prototyping. We set out to prove through this hackathon that glasses that open the world of written text to those who have trouble entering it themselves can be cheap and accessible.

## What it does

Our device attaches magnetically to a pair of glasses to allow users to wear it comfortably while reading, whether that's on a couch, at a desk or elsewhere. The software tracks what they are seeing and when written words appear in front of it, chooses the clearest frame and transcribes the text and then reads it out loud.

## How we built it

**Software (Alex K)** -
On the software side, we first needed to get image-to-text (OCR or optical character recognition) and text-to-speech (TTS) working. After trying a couple of libraries for each, we found Google's Cloud Vision API to have the best performance for OCR and their Google Cloud Text-to-Speech to also be the top pick for TTS.

The TTS performance was perfect for our purposes out of the box, but bizarrely, the OCR API seemed to predict characters with an excellent level of accuracy individually, but poor accuracy overall due to seemingly not including any knowledge of the English language in the process. (E.g. errors like "Intreduction" etc.) So the next step was implementing a simple unigram language model to filter down the Google library's predictions to the most likely words.

Stringing everything together was done in Python with a combination of Google API calls and various libraries including OpenCV for camera/image work, pydub for audio and PIL and matplotlib for image manipulation.

**Hardware (Alex G)**: We tore apart an unsuspecting Logitech webcam, and had to do some minor surgery to focus the lens at an arms-length reading distance. We CAD-ed a custom housing for the camera with mounts for magnets to easily attach to the legs of glasses. This was 3D printed on a Form 2 printer, and a set of magnets glued in to the slots, with a corresponding set on some NerdNation glasses.

## Challenges we ran into

The Google Cloud Vision API was very easy to use for individual images, but making synchronous batched calls proved to be challenging!
Finding the best video frame to use for the OCR software was also not easy and writing that code took up a good fraction of the total time.

Perhaps most annoyingly, the Logitech webcam did not focus well at any distance! When we cracked it open we were able to carefully remove bits of glue holding the lens to the seller’s configuration, and dial it to the right distance for holding a book at arm’s length.

We also couldn’t find magnets until the last minute and made a guess on the magnet mount hole sizes and had an *exciting* Dremel session to fit them which resulted in the part cracking and being beautifully epoxied back together.

## Acknowledgements

The Alexes would like to thank our girlfriends, Allie and Min Joo, for their patience and understanding while we went off to be each other's Valentine's at this hackathon.