## Inspiration

Have either of the following happened to you?

* Ever since elementary school you've been fascinated by 17th century Turkish ethnography. Luckily, you just discovered a preeminent historian's blog about the collapse of the Ottoman Empire. Overjoyed, you start to text your friends, but soon remember that they're into 19th century Victorian poetry. If only you could share your love of historical discourse with another intellectual.
* Because you're someone with good taste, you're browsing Buzzfeed. Somehow "27 Extremely Disturbing Wikipedia Pages That Will Haunt Your Dreams" is not cutting it for you. Dang. If only you could see what your best friend Alicia was browsing. She would definitely know how to help you procrastinate on your TreeHacks project.
* On a Friday night, all your close friends have gone to a party, and you are bored to death. You look through the list of your Facebook friends. There are hundreds of people online, but you feel awkward and don’t know how to start a conversation with any of them.

Great! Because we built PageChat for you. We all have unique interests, many of which are expressed through our internet browsing. We believe that building simple connections through those interests is a powerful way to improve well-being. We built a *convenient* and *efficient* tool to connect people through their internet browsing.

## What it does

PageChat is a Google Chrome extension designed to promote serendipitous connections by offering one-on-one text chats centered on internet browsing. When active, Pagechat

* displays what you are your friends are currently reading, allowing you to discover and share interesting articles
* centers the conversation around web pages by giving friends the opportunity to chat each other directly through Chrome
* intelligently connects users with similar interests by creating one-on-one chats for users all over the world visiting the same webpage

## How we built it

### Chatting

The chrome extension was built with Angular. The background script keeps track of the tab updates and activations, and this live information is sent to the backend. The Angular App retrieves the list of friends and other users online and displays it on to the chrome extension. For friends, the title of the page they are currently reading is displayed. For users that are not friends, only those who are on the same web page are displayed. For each user displayed on the Chrome extension, we can start a chat. Then, the list view is changed to a chat room where users can have a discussion.

Instead of maintaining our own server, we used Firebase extensively. The live connections are managed by Realtime Database. Yet, since Firestore is easier to work with, we use Cloud Function to reflect the changes of live usage to Firestore using Cloud Function. Thus, there is a ‘status’ collection that contains live information about the connection state and the url and page title each user is looking at. The friends relations are maintained with ‘friends’ collection. We use Firechat, which is an open-source realtime chatting app using Firebase. Thus, all the chatting activities and histories are saved in the chats collection.

One interesting collection is the ‘feature’ collection. It stores the feature vector, which is an array of 256 numbers, for each user. Whenever a user visits a new page (in the future we plan to change the feature vector update criterion), a Cloud Function is triggered, and using our model, the feature vector is updated. The feature vector is used to find better matches i.e. people that would have similar interests among friends and other users using PageChat. As more data is accumulated, the curated list of people users would want to talk to would improve.

### User Recommendations

If several people are browsing the same website and want to chat with each other, how do we pair them up? Intuitively, people who have similar browsing histories will have more in common to talk about, so we should group them together. We maintain a dynamic **feature vector** for each user, which is based off of their reading history. We want feature vectors with small cosine distance to be similar.

When is active on PageChat and visits a site on our whitelist (we don't want sites with generic titles, so we stick to mostly news sites), we obtain the title of the page. We make the assumption that the title is representative of the content our user is reading. For example, we would expect that "27 Extremely Disturbing Wikipedia Pages That Will Haunt Your Dreams" has different content from "Ethnography Museum of Ankara". To obtain a reasonable embedding of our title, we use [SBERT](https://arxiv.org/abs/1908.10084), a BERT-based language model trained to predict the representation of sentences. SBERT can attend to the salient keywords in each title and can obtain a global representation of the title.

Next, we need some way to update feature vectors whenever a user visits a new page. This is well-suited for a recurrent neural network. These models maintain a hidden state that is continually updated with each new query. We will use an [LSTM](http://colah.github.io/posts/2015-08-Understanding-LSTMs/) to update our feature vectors. It takes in the previous feature vector and new title and outputs the new feature vector.

Finally, we need to train our LSTM. Fortunately, UCI has released a [dataset](https://archive.ics.uci.edu/ml/datasets/News+Aggregator) of news headlines along with their respective category (business, science and technology, entertainment, health). We feed these headlines as training data for the model. Before training, we preprocess the headlines, embedding all of them with SBERT. This greatly reduces training times. We use the following three-step procedure to train the model:

1. *Train an [autoencoder](https://arxiv.org/abs/1502.04681) to learn a compressed representation of the feature title.* SBERT outputs features vectors with 768 elements, which is large and unwieldy. An autoencoder is an unsupervised learning model that is able to learn high-fidelity compressed representations of the input sequence. The encoder (an LSTM) encodes a sequence of headlines into a lower-dimensional (128) space and the decoder (another LSTM) decodes the sequence. The model's goal is to output a sequence as close as possible to the original input sequence. After training, we will end up with an encoder LSTM that is able to faithfully map 768 element input vector to a 128 element space and maintain the information of the representation. We use a [contractive autoencoder](https://icml.cc/2011/papers/455_icmlpaper.pdf), which adds an extra loss term to promote the encoder to be less sensitive to variance in the input.
2. *Train the encoder to condense feature vectors that share a category.* Suppose that Holo reads several business articles and Lawrence also reads several business articles. Ideally, they should have similar feature vectors. To train, we build sequences of 5 headlines, where each headline in a sequence is drawn from the same category. The encoder LSTM from the previous step encodes these sequences to feature vectors. We train it to obtain a higher cosine similarity for vectors that share a category than for vectors that don't.
3. *Train the encoder to condense feature vectors that share a story.* The dataset also has a feature corresponding to the specific news story an article covers. Similar to step 2, we build sequences of 5 headlines, where each headline in a sequence is drawn from the same story. By training the encoder to predict a high cosine similarity for vectors that share a story, we further improve the representation of the feature vectors.

We provide some evaluations for our model and document our training process in this [Google notebook](https://colab.research.google.com/drive/1B4eWINVyWntF0VESUUgRt2opA4QFoS-M?usp=sharing). Feel free to make a copy and tinker with it!

## What's next for Pagechat

* Implementation of more features (e.g. group chatting rooms, voice chats, reading pattern analysis per user) that make the app more fun to use.
## Inspiration

Has your browser ever looked like this?

![](https://i.imgur.com/waCM1K0.png)
... or this?
![](https://i.imgur.com/WX2dTfz.png)

Ours have, *all* the time.

Regardless of who you are, you'll often find yourself working in a browser on not just one task but a variety of tasks. Whether its classes, projects, financials, research, personal hobbies -- there are many different, yet predictable, ways in which we open an endless amount of tabs for fear of forgetting a chunk of information that may someday be relevant.

Origin aims to revolutionize your personal browsing experience -- one workspace at a time.

## What it does

In a nutshell, Origin uses state-of-the-art **natural language processing** to identify personalized, smart **workspaces**. Each workspace is centered around a topic comprised of related tabs from your browsing history, and Origin provides your most recently visited tabs pertaining to that workspace and related future ones, a generated **textual summary** of those websites from all their text, and a **fine-tuned ChatBot** trained on data about that topic and ready to answer specific user questions with citations and maintaining history of a conversation. The ChatBot not only answers general factual questions (given its a foundation model), but also answers/recalls specific facts found in the URLs/files that the user visits (e.g. linking to a course syllabus).

Origin also provides a **semantic search** on resources, as well as monitors what URLs other people in an organization visit and recommend pertinent ones to the user via a **recommendation system**.

For example, a college student taking a History class and performing ML research on the side would have sets of tabs that would be related to both topics individually. Through its clustering algorithms, Origin would identify the workspaces of "European History" and "Computer Vision", with a dynamic view of pertinent URLs and widgets like semantic search and a chatbot. Upon continuing to browse in either workspace, the workspace itself is dynamically updated to reflect the most recently visited sites and data.

**Target Audience**: Students to significantly improve the education experience and industry workers to improve productivity.

## How we built it

![](https://i.imgur.com/HYsZ3un.jpg)

**Languages**: Python ∙ JavaScript ∙ HTML ∙ CSS

**Frameworks and Tools**: Firebase ∙ React.js ∙ Flask ∙ LangChain ∙ OpenAI ∙ HuggingFace

There are a couple of different key engineering modules that this project can be broken down into.

### 1(a). Ingesting Browser Information and Computing Embeddings

We begin by developing a Chrome Extension that automatically scrapes browsing data in a periodic manner (every 3 days) using the Chrome Developer API. From the information we glean, we extract titles of webpages. Then, the webpage titles are passed into a pre-trained Large Language Model (LLM) from Huggingface, from which latent embeddings are generated and persisted through a Firebase database.

### 1(b). Topical Clustering Algorithms and Automatic Cluster Name Inference

Given the URL embeddings, we run K-Means Clustering to identify key topical/activity-related clusters in browsing data and the associated URLs.

We automatically find a description for each cluster by prompt engineering an OpenAI LLM, specifically by providing it the titles of all webpages in the cluster and requesting it to output a simple title describing that cluster (e.g. "Algorithms Course" or "Machine Learning Research").

### 2. Web/Knowledge Scraping

After pulling the user's URLs from the database, we asynchronously scrape through the text on each webpage via Beautiful Soup. This text provides richer context for each page beyond the title and is temporarily cached for use in later algorithms.

### 3. Text Summarization

We split the incoming text of all the web pages using a CharacterTextSplitter to create smaller documents, and then attempt a summarization in a map reduce fashion over these smaller documents using a LangChain summarization chain that increases the ability to maintain broader context while parallelizing workload.

### 4. Fine Tuning a GPT-3 Based ChatBot

The infrastructure for this was built on a recently-made popular open-source Python package called **LangChain** (see <https://github.com/hwchase17/langchain>), a package with the intention of making it easier to build more powerful Language Models by connecting them to external knowledge sources.

We first deal with data ingestion and chunking, before embedding the vectors using OpenAI Embeddings and storing them in a vector store.

To provide the best chat bot possible, we keep track of a history of a user's conversation and inject it into the chatbot during each user interaction while simultaneously looking up relevant information that can be quickly queries from the vector store. The generated prompt is then put into an OpenAI LLM to interact with the user in a knowledge-aware context.

### 5. Collaborative Filtering-Based Recommendation

Provided that a user does not turn privacy settings on, our collaborative filtering-based recommendation system recommends URLs that other users in the organization have seen that are related to the user's current workspace.

### 6. Flask REST API

We expose all of our LLM capabilities, recommendation system, and other data queries for the frontend through a REST API served by Flask. This provides an easy interface between the external vendors (like LangChain, OpenAI, and HuggingFace), our Firebase database, the browser extension, and our React web app.

### 7. A Fantastic Frontend

Our frontend is built using the React.js framework. We use axios to interact with our backend server and display the relevant information for each workspace.

## Challenges we ran into

1. We had to deal with our K-Means Clustering algorithm outputting changing cluster means over time as new data is ingested, since the URLs that a user visits changes over time. We had to anchor previous data to the new clusters in a smart way and come up with a clever updating algorithm.
2. We had to employ caching of responses from the external LLMs (like OpenAI/LangChain) to operate under the rate limit. This was challenging, as it required revamping our database infrastructure for caching.
3. Enabling the Chrome extension to speak with our backend server was a challenge, as we had to periodically poll the user's browser history and deal with CORS (Cross-Origin Resource Sharing) errors.
4. We worked modularly which was great for parallelization/efficiency, but it slowed us down when integrating things together for e2e testing.

## Accomplishments that we're proud of

The scope of ways in which we were able to utilize Large Language Models to redefine the antiquated browsing experience and provide knowledge centralization.

This idea was a byproduct of our own experiences in college and high school -- we found ourselves spending significant amounts of time attempting to organize tab clutter systematically.

## What we learned

This project was an incredible learning experience for our team as we took on multiple technically complex challenges to reach our ending solution -- something we all thought that we had a potential to use ourselves.

## What's next for Origin

We believe Origin will become even more powerful at scale, since many users/organizations using the product would improve the ChatBot's ability to answer commonly asked questions, and the recommender system would perform better in aiding user's education or productivity experiences.
## Inspiration

As our world becomes more digitalized and interactions become a more permanent, our team noticed a rise in online anxiety stemming from an innate fear of being judged or making a mistake. In Elizabeth Armstrong's book *Paying for the Party* , Armstrong mentions the negative impacts of being unique at a school where it pays to not stand out. This exact sentiment can now be seen online, except now, everything can be traced back to an identity indefinitely. Our thoughts, questions, and personal lives are constantly be ridiculed and monitored for mistakes. Even after a decade of growth, we will still be tainted with the person we were years before. Contrary to this social fear, many of us started off a childhood with a confidence and naivety of social norms allowing us to simply make friends based on interests. Everyday was made for show-and-tell and asking questions. Through this platform, we seek to develop a web app that allows us to reminisce about the days when making friends was as easy as turning to stranger on the playground and asking to play.

## What it does

Our web app is designed to make befriending strangers with shared interests easier and making mistakes less permanent. When opening the app, users will be given a pseudo name and will be able to choose their interests based on a word cloud. Afterwards, the user can then follow one of three paths. The first would be a friend matching path where the user will receive eight different people who share common interests with them. In these profiles, each person's face would be blurred and the only thing shared would be interests and age. The user can select up to two people to message per day. The second path allows for learning. Once a user selects a topic they'd like to learn more about, they will then be matched to someone who is volunteering to share information. The third consists of a random match in the system for anyone who is feeling spontaneous. This was inspired by Google's "I'm feeling lucky" button. Once messaging begins, the both people will have the ability to reveal their identity at any point, which would resolve the blurred image on their profile for the user they are unlocking it for.

The overall objective is to create a space for users to share without their identity being attached.

## How we built it

Our team built this by taking time to learn UI design in Figma and then began to implement the frontend through html and css. We then attempted to build the back-end using python through Flask. We then hosted the web app on Azure as our server.

## Challenges we ran into

Our team is made up of 100% beginners with extremely limited coding experience, so finding the starting point for web app development was the biggest challenge we ran into. In addition, we ran into a significant amount of software downloading issues which we worked with a mentor to resolve for several hours. Due to these issues, we never fully implement the program.

## Accomplishments that we're proud of

Our team is extremely proud of the progress we have made thus far on the project. Coming in, most of us had very limited skills so being able to have learned Figma and launch a website in 36 hours feels incredible. Through this process, all of us were able to learn something new whether that be a software, language, or simply the process of website design and execution. As a group coming from four different schools from different parts of the world, we are also proud of the general enthusiasm, friendship, and team skills we built through this journey.

## What we learned

Coming in as beginner programmers, our team learned a lot about the process of creating and designing a web app from start to finish. Through talking to mentors, we were able to learn more about the different softwares, frameworks, and languages many applications use as well as the flow of going from frontend to backend. In terms of technical skills, we picked up Figma and html and css through this project.

## What's next for Playground

In the future, we hope to continue designing the frontend of Playground and then implementing the backend through python since we never got to the point of completion. As a web app, we hope to be able to later implement better matching algorithms and expanding into communities for different "playground."