## Inspiration

>
> Email search sucks... A lot...
>
>
>

We've all had negative experiences with email. To be specific, it's quite hard to find specific emails. Email search is a cross-platform pain point, regardless of mobile or web: the core problem stems from the fact that most email search systems are reactive, simplistic, and keyword-based. They rely on older indexing and retrieval methods rather than understanding the true intent behind a query.

To figure out what we could do to make some progress, we talked to over 50 hackers this weekend before we got to building.

We learned users typically don’t remember exact email keywords but instead rely on contextual clues—like who the email was from, the approximate date, or a related attachment. Most email clients force users to manually filter by these parameters, which introduces friction.

We hypothesized that creating a retrieval system that had semantic understanding would likely provide a much better UX. Additionally, we could use a language model to provide responses to queries in natural language.

What if your inbox could understand everyday language? You could ask, "When’s my next meeting?" or "Show me the latest sales report," and get the right answer right away. Our idea turns email search into a happier, easier experience. With embeddings, vector databases, and language models, we’re not just making search better — we’re changing how people deal with emails, helping them save time and work more efficiently.

## What it does

Scope streamlines email search by making it fast and intuitive. After logging in, Scope retrieves, preprocesses, and cleans your emails. It then embeds these emails as high-dimensional vectors in ChromaDB, a vector database designed for efficient semantic search.

When you input a natural language query, Scope processes it using a language model to understand intent rather than just keywords. It semantically searches the vector database and returns the most relevant emails alongside a concise, natural language response using Groq's super-speed inference times. We even multi-modally embed images contained in emails for a heightened understanding. Whether you're looking for a specific message or general information, Scope provides quick, accurate results—without manual filtering.

Engage with your email like you would with a human. *It just makes more sense*.

## How we built it

We built Scope using Electron to support both cross-platform desktop apps and a website, ensuring accessibility across devices. The frontend is crafted with JavaScript, Electron, React, and designed in Figma for a clean, intuitive UI. For the backend, we chose Python, Flask, and Groq. Groq provides a ~40x speed improvement in inference over OpenAI, allowing for faster query processing. We used ChromaDB for its built-in, customizable embeddings and efficient vector search, enabling us to handle semantic queries seamlessly. Our v0 started with 100 synthetically generated emails, and the final product includes Google login integration, allowing users to sign in with their Gmail and search their inbox with natural language.

## Challenges we ran into

One of the key challenges we faced was figuring out the best way to preprocess emails. Emails are complex, often containing HTML, tags, images, and various types of data. We needed a way to extract relevant content without pulling in unnecessary noise. Additionally, since emails come in many different formats, we required a flexible, non-discriminative preprocessing approach that could handle any structure effectively.

Integrating with Google was another hurdle, especially since it was our first time using Google Cloud Platform. We had to dive deep into understanding access tokens and credentials to obtain READ privileges for emails. This also involved building communication routes between the frontend and backend to allow users to log in with their Gmail accounts and stay logged in seamlessly. Additionally, the GMail API rate-limits us, forcing us only to index 500 emails at a time.

Lastly, choosing the most effective embedding model was quite challenging. We experimented with several, including sentence transformers, word transformers, multimodal models, and models from OpenAI and Hugging Face. Through rigorous testing and quantitative evaluation, we were able to identify the most effective model for our needs, ensuring accurate and efficient email search results.

## Accomplishments that we're proud of

* **FAST email search and LLM inference**: We achieved rapid email retrieval by optimizing our embedding model and leveraging Groq's 40x faster inference, significantly improving response times over traditional methods.
* **Built-in integration with Gmail**: Implementing seamless Google login with full email READ access was a major win, allowing users to authenticate easily and use Scope with their Gmail accounts to search emails without friction.
* **Customizing ChromaDB setup**: We tailored ChromaDB to fit our specific use case, optimizing its embeddings and vector search for fast and accurate semantic query results, ensuring our users get relevant email matches.
* **Clean, intuitive, beautiful user interface**: Our frontend design is sleek and user-friendly, thanks to careful collaboration between Figma and React, making Scope feel simple yet powerful for all users.
* **Turning user research into a working product in under 40 hours**: Through rapid iteration and focusing on user pain points, we were able to transform our insights into a fully functional solution, bringing real impact in a short time frame.
* **Scoping hard and finishing v0 early**: By setting clear, realistic goals, we delivered our v0 ahead of schedule, which gave us valuable extra time to refine and improve the product before final submission.

## What we learned

* Working with new technologies like Groq, ChromaDB, and Google API taught us how to integrate cutting-edge tools and optimize them for our specific needs, significantly enhancing our product's performance.
* User research is crucial for identifying real problems. By speaking with users, we were able to hone in on their actual pain points and design a solution that directly addressed them.
* We gained a deep understanding of high-dimensional vector spaces and how they can be leveraged to retrieve semantic meaning, improving the accuracy of our email search.
* Fast-paced development is both rewarding and risky. By keeping our development goals top-of-mind, we were able to reduce bugs, iterate quickly, and still deliver a polished product under tight time constraints.

## What's next for Scope

* Integrate with email server protocols instead of relying solely on the Google API, allowing Scope to support a wider range of email services and offer greater flexibility.
* Build out mail extensions for a seamless experience, enabling users to access Scope through a global keyboard shortcut, similar to Spotlight search, for even faster email querying.
* We're excited to take Scope to the next level! Over the coming weeks, we'll be refining the product, setting up meetings with potential stakeholders, and working to identify a strong product-market fit.

**Boom.**
## Inspiration

With the flood of daily emails, it's easy to lose track of important deadlines and tasks buried within. We needed a solution that could help organize and prioritize emails efficiently, saving time and reducing the risk of missing crucial deadlines. The idea was to build an intuitive email tool that fetches emails, summarizes them, and highlights the most critical information, such as deadlines, while categorizing them based on priority.

## What it does

Ezemail is a Chrome extension that:

* Connects to your Gmail account using the Gmail API.
* Fetches emails over a selected time range.
* Uses the OpenAI API to summarize the content of the emails.
* Categorizes emails based on urgency, importance, and subject.
* Extracts and highlights deadlines or time-sensitive information.
* Sends an email to the user with a summary of all emails within the selected period, including deadlines and categorized insights.

## How we built it

* Frontend: We built the Chrome extension UI using HTML, CSS, and JavaScript. We focused on creating a clean and easy-to-navigate interface for reading email summaries and deadlines.
* Backend: The core logic integrates Gmail's API to retrieve emails and the OpenAI API to process and summarize the email content. After the summarization process is completed, the summary is emailed back to the user with organized insights.
* Email Categorization: By analyzing the email content, we used the OpenAI API to determine the importance and urgency, automatically grouping emails into categories like "High Priority," "Reminders," "Tasks," etc.
* Deadline Extraction: By scanning the email text, the tool pulls out dates and times and organizes them into a visual calendar.

## Challenges we ran into

* Authentication for Gmail and OpenAI APIs: The procedure involved many settings on the Google Cloud Platform, the Chrome extension, and our code. We visited some online resources but they did not work. This was quite challenging for us as we have never done that. It required deep understanding of the interaction between the user and the extension.
* API Rate Limits: We encountered challenges with the Gmail and OpenAI API rate limits, which required us to batch requests and handle throttling.
* Parsing complex emails: Some emails contain attachments, rich formatting, or embedded media, making it difficult to summarize the key points accurately.

## Accomplishments that we're proud of

* Successfully integrating two powerful APIs (Gmail and OpenAI) to create a seamless email management solution.
* Creating a user-friendly extension that simplifies email workflows and makes important information (like deadlines) easily accessible.
* Implementing deadline extraction and reminder features, helping users stay on top of important tasks and dates.
* Automating the process to send users email summaries, eliminating the need to check emails frequently.

## What we learned

* Deepened our understanding of API integrations and handling their limitations.
* Gained insights into how AI-driven summarization can vastly improve email management.
* Learned the importance of UX design in productivity tools, making sure the tool is intuitive for users.

## What's next for Ezemail

* Smart Notifications: Implement push notifications for approaching deadlines or high-priority emails.
* Advanced Email Filtering: Improve categorization by learning from the user's interaction with emails (e.g., marking something as high/low priority).
* Attachment Summarization: Extend the summarization feature to attachments like PDFs and documents.
* Multiple Email Accounts: Allow users to connect and manage multiple Gmail accounts from within the extension.
## Inspiration

Knowtworthy is a startup that all three of us founded together, with the mission to make meetings awesome. We have spent this past summer at the University of Toronto’s Entrepreneurship Hatchery’s incubator executing on our vision. We’ve built a sweet platform that solves many of the issues surrounding meetings but we wanted a glimpse of the future: entirely automated meetings. So we decided to challenge ourselves and create something that the world has never seen before: sentiment analysis for meetings while transcribing and attributing all speech.

## What it does

While we focused on meetings specifically, as we built the software we realized that the applications for real-time sentiment analysis are far more varied than initially anticipated. Voice transcription and diarisation are very powerful for keeping track of what happened during a meeting but sentiment can be used anywhere from the boardroom to the classroom to a psychologist’s office.

## How I built it

We felt a web app was best suited for software like this so that it can be accessible to anyone at any time. We built the frontend on React leveraging Material UI, React-Motion, Socket IO and ChartJS. The backed was built on Node (with Express) as well as python for some computational tasks. We used GRPC, Docker and Kubernetes to launch the software, making it scalable right out of the box.

For all relevant processing, we used Google Speech-to-text, Google Diarization, Stanford Empath, SKLearn and Glove (for word-to-vec).

## Challenges I ran into

Integrating so many moving parts into one cohesive platform was a challenge to keep organized but we used trello to stay on track throughout the 36 hours.
Audio encoding was also quite challenging as we ran up against some limitations of javascript while trying to stream audio in the correct and acceptable format.
Apart from that, we didn’t encounter any major roadblocks but we were each working for almost the entire 36-hour stretch as there were a lot of features to implement.

## Accomplishments that I'm proud of

We are super proud of the fact that we were able to pull it off as we knew this was a challenging task to start and we ran into some unexpected roadblocks. There is nothing else like this software currently on the market so being first is always awesome.

## What I learned

We learned a whole lot about integration both on the frontend and the backend. We prototyped before coding, introduced animations to improve user experience, too much about computer store numbers (:p) and doing a whole lot of stuff all in real time.

## What's next for Knowtworthy Sentiment

Knowtworthy Sentiment aligns well with our startup’s vision for the future of meetings so we will continue to develop it and make it more robust before integrating it directly into our existing software. If you want to check out our stuff you can do so here: <https://knowtworthy.com/>