## 💡 Inspiration

The objective of our application is to devise an effective and efficient written transmission optimization scheme, by converting esoteric text into an exoteric format.

If you read the above sentence more than once and the word ‘huh?’ came to mind, then you got my point. Jargon causes a problem when you are talking to someone who doesn't understand it. Yet, we face obscure, vague texts every day - from ['text speak'](https://www.goodnewsnetwork.org/dad-admits-hilarious-texting-blunder-on-the-moth/) to T&C agreements.

The most notoriously difficult to understand texts are legal documents, such as contracts or deeds. However, making legal language more straightforward would help people understand their rights better, be less susceptible to being punished or not being able to benefit from their entitled rights.

Introducing simpl.ai - A website application that uses NLP and Artificial Intelligence to recognize difficult to understand text and rephrase them with easy-to-understand language!

## 🔍 What it does

simpl.ai intelligently simplifies difficult text for faster comprehension. Users can send a PDF file of the document they are struggling to understand. They can select the exact sentences that are hard to read, and our NLP-model recognizes what elements make it tough. You'll love simpl.ai's clear, straightforward restatements - they change to match the original word or phrase's part of speech/verb tense/form, so they make sense!

## ⚙️ Our Tech Stack

[![Tech-Diagram-drawio.png](https://i.postimg.cc/1RprSfYf/Tech-Diagram-drawio.png)](https://postimg.cc/gr2ZqkpW)

**Frontend:** We created the client side of our web app using React.js and JSX based on a high-fidelity prototype we created using Figma. Our components are styled using MaterialUI Library, and Intelllex's react-pdf package for rendering PDF documents within the app.

**Backend:** Python! The magic behind the scenes is powered by a combination of fastAPI, TensorFlow (TF), Torch and Cohere. Although we are newbies to the world of AI (NLP), we used a BART model and TF to create a working model that detects difficult-to-understand text! We used the following [dataset](https://www.inf.uni-hamburg.de/en/inst/ab/lt/resources/data/complex-word-identification-dataset/cwishareddataset.zip) from Stanford University to train our [model](http://nlp.stanford.edu/data/glove.6B.zip)- It's based on several interviews conducted with non-native English speakers, where they were tasked to identify difficult words and simpler synonyms for them. Finally, we used Cohere to rephrase the sentence and ensure it makes sense!

## 🚧 Challenges we ran into

This hackathon was filled with many challenges - but here are some of the most notable ones:

* We purposely choose an AI area where we didn't know too much in (NLP, TensorFlow, CohereAPI), which was a challenging and humbling experience. We faced several compatibility issues with TensorFlow when trying to deploy the server. We decided to go with AWS Platform after a couple of hours of trying to figure out Kubernetes 😅
* Finding a dataset that suited our needs! If there were no time constraints, we would have loved to develop a dataset that is more focused on addressing tacky legal and technical language. Since that was not the case, we made do with a database that enabled us to produce a proof-of-concept.

## ✔️ Accomplishments that we're proud of

* Creating a fully-functioning app with bi-directional communication between the AI server and the client.
* Working with NLP, despite having no prior experience or knowledge. The learning curve was immense!
* Able to come together as a team and move forward, despite all the challenges we faced together!

## 📚 What we learned

We learned so much in terms of the technical areas; using machine learning and having to pivot from one software to the other, state management and PDF rendering in React.

## 🔭 What's next for simpl.ai!

**1. Support Multilingual Documents.** The ability to translate documents and provide a simplified version in their desired language. We would use [IBM Watson's Language Translator API](https://cloud.ibm.com/apidocs/language-translator?code=node)

**2. URL Parameter** Currently, we are able to simplify text from a PDF, but we would like to be able to do the same for websites.

* Simplify legal jargon in T&C agreements to better understand what permissions and rights they are giving an application!
* We hope to extend this service as a Chrome Extension for easier access to the users.

**3. Relevant Datasets** We would like to expand our current model's capabilities to better understand legal jargon, technical documentation etc. by feeding it keywords in these areas.
## Inspiration

The whiteboard or chalkboard is an essential tool in instructional settings - to learn better, students need a way to directly transport code from a non-text medium to a more workable environment.

## What it does

Enables someone to take a picture of handwritten or printed text converts it directly to code or text on your favorite text editor on your computer.

## How we built it

On the front end, we built an app using Ionic/Cordova so the user could take a picture of their code. Behind the scenes, using JavaScript, our software harnesses the power of the Google Cloud Vision API to perform intelligent character recognition (ICR) of handwritten words. Following that, we applied our own formatting algorithms to prettify the code. Finally, our server sends the formatted code to the desired computer, which opens it with the appropriate file extension in your favorite IDE. In addition, the client handles all scripting of minimization and fileOS.

## Challenges we ran into

The vision API is trained on text with correct grammar and punctuation. This makes recognition of code quite difficult, especially indentation and camel case. We were able to overcome this issue with some clever algorithms. Also, despite a general lack of JavaScript knowledge, we were able to make good use of documentation to solve our issues.

## Accomplishments that we're proud of

A beautiful spacing algorithm that recursively categorizes lines into indentation levels.
Getting the app to talk to the main server to talk to the target computer.
Scripting the client to display final result in a matter of seconds.

## What we learned

How to integrate and use the Google Cloud Vision API.
How to build and communicate across servers in JavaScript.
How to interact with native functions of a phone.

## What's next for Codify

It's feasibly to increase accuracy by using the Levenshtein distance between words. In addition, we can improve algorithms to work well with code. Finally, we can add image preprocessing (heighten image contrast, rotate accordingly) to make it more readable to the vision API.
**[DEMO VIDEO LINK — On Loom!](https://www.loom.com/share/2b15581b7fe34ec7974c26f24ad3829f?sid=87ea8fee-5efa-46c1-984f-bb6bab8d664a)** ✨

## Inspiration

We were inspired by how Wikipedia links to other Wikipedia pages so that you can find the information quickly and painlessly if you don't know what a word means. We felt this technology would save people a lot of time parsing jargon-heavy documents, and help you *understand* your readings for school!

## What it does

Source Scanner takes raw text, and intelligently selects words/phrases that can be reasonably assumed to need definition or extra information, and makes it so that we can hover over those words and find helpful definitions, as well as links off-site to places such as Wikipedia for further reading!✨

We know that when you're reading, it takes a while for your brain to really process and understand words, and on occasion, we "gloss" over words we can usually infer through context. While this helps our comprehension skills, we want to make information more accessible for students to *actually* learn new vocabulary and concepts that they may not have learned on their own!

## How we built it 👩🏻‍🔬

On the backend, when a user makes a request for our web app to parse input and return more information, our app sends the user-inputted information to OpenAI's API with an engineered prompt to parse the user input for words and concepts that may need to be defined and puts those, along with definitions, into an easily parse-able data structure. From there, each of those concepts is fed into [Metaphor's API](https://metaphor.systems/) to generate relevant links, and the best link by confidence score is displayed as a hyperlink to the user upon hovering over the previously selected word, along with a definition. We use MongoDB to store user information, Google Cloud for user authentication, and Vercel for website deployment.

* ReactTS
* TailwindCSS
* NextJS
* tRPC
* MongoDB
* Prisma
* Google Cloud
* Metaphor API
* OpenAI API

## Challenges we ran into

One of the main challenges our group ran into was integrating so many moving parts and getting them to work together seamlessly. Getting OAuth with Google Cloud to work with tRPC, Prisma, and MongoDB Atlas was quite the challenge with the different ways they interact with each other. Another hurdle was prompt engineering to request meaningful data from ChatGPT and Metaphor because there's a lot of specificity and accuracy that need to come together to get something that would benefit the user. Finally, having specific UI components, such as tooltips, actually helps, rather than hinders, the user experience was a definite learning curve.

## Accomplishments that we're proud of

We are very proud of the fact that we not only successfully achieved a design with a large number of moving parts, but we did so while building out several well-designed pages that are easily understandable for site users.

## What we learned 🫶🏼

We learned a lot about working with the OpenAI and Metaphor APIs, as well as the power of a good prompt and a better AI when it came time to parse through the input. We also appreciated the opportunity to refamiliarize ourselves with React and design elements of websites.

## What's next for SourceScanner ⚡️⚡️

The next step for SourceScanner is adding more of what we call "lenses," or areas of the internet to search for answers. We only have one (the "Wikipedia" lens), but we plan to expand to other prompts and websites for more niche information, such as C or C++ documentation.