## Inspiration

Everybody these days lead a very busy life with no time to spend on long newspapers and news articles. This site was designed to help these people catch up with the world without spending a ton of time reading the articles!

## What it does

It provides a collection of articles, simplified AI-generated summary, link to the original article and an audio version of the news that one can listen to as they start the day.

## How we built it

We built a python webserver that can crawl the news sites, generate summaries using AI and serve the articles to end user in a simplified and clean layout. The app is hosted in Google Cloud App Engine and uses Firebase cloud firestore as NoSQL data store. We also use Google Cloud Speech to Text API service to transcribe the generated summary to audio.

Summarizer developed is an Extractive text summarizer. Extractive text summarization involves the selection of phrases and sentences from the source document to make up the new summary. Techniques involve ranking the relevance of phrases to choose only those most relevant to the meaning of the source. The algorithm uses several features to score the relevance of a sentence:

* TF IDF: is a numerical statistic that reflects how important a word is to a document in a collection or corpus. The tf–idf value increases proportionally to the number of times a word appears in the document and is offset by the number of documents in the corpus that contain the word.
* Similarity to headline
* Position of the sentence in the article: This assumes that sentences in the starting and in the ending are more important
* Length of the sentence

All of these are weighted to produce a single score for each sentence in the article and the most important sentences are picked to form the summary.

## Challenges we ran into

We developed a crawler to crawl the web for latest updates but data quality/text cleaning became an issue. The text that we crawled had many external links, noise and adds that we couldn't get rid of in the timelines. This degraded the quality of the summarizer. This led us to crawl a specific news site and generate the summaries for it.

## Accomplishments that we’re proud of

* We were successfully able to get the application to the Google Cloud app engine and use Firebase effectively to host the application. This should help us scale the application easily!
* Summarizer efficiently summarizes the articles to present the most important parts of the article.
* Summary as speech gives an excellent option for the busy user to listen in and helps multi-tasking!
## What we learned
We learn a lot of concepts on NLP, hosting an application in Google Cloud, using firestore as a datastore and many more! Also learning to collaborate virtually is worth a mention :D

## What’s next for tech-insights - simplified news site for busy technologists

* Customize user-specific news feed based on categories of interest
* Crawler to dynamically crawl for new tech news
* Try Abstractive text summarization - involves generating entirely new phrases and sentences to capture the meaning of the source document.
## Inspiration

With the flurry of new tech developments and media tools, people have access to an incredibly large amount of news and information about the world. To assist in keeping up with current events, we built Squeeze, a web-app that takes top stories from major news outlets and summarizes + highlights keywords from the article, allowing the user to explore more headlines and find what interests them.

## What it does

Squeeze summarizes and highlights stories from top media outlets, yielding a highlighted summary but maintaining complete links to the original source. Squeeze takes in RSS feeds from major news websites, and scrapes their article contents. This data is fed into machine learning algorithms to clean and summarize their contents. Keywords are chosen using the Google Cloud natural language API. The user, upon choosing a headline of interest, are presented with a highlighted text summary, and easy-access links to the original source if they are further interested.

## How we built it

We used the Python package Feedparser to parse RSS feeds of new websites, and the package BeautifulSoup to scrape data from these feeds. Machine Learning algorithms were used to clean and summarize the scraped HTML data. Keywords in the article were chosen using the Google Cloud natural language API, and built the custom search engine using the Google Custom Search API. Front-end work was performed using Javascript + HTML.

## Challenges we ran into

There were couple of challenges that we ran into. 1) How to deploy the summarized text on the Google Cloud. 2) How to connect and visualize the scraped data through the search engine.

## Accomplishments that we're proud of

We are proud that we were able to finish the ML algorithm for text summation and how to find keywords in the text.

## What we learned

We learnt how to ML in order to clean the HTML data as well as how to find keywords in the text.

## What's next for Squeeze

To be continued
## Inspiration

The biggest irony today is despite the advent of the internet, students and adults are more oblivious than ever to world events, and one can easily understand why. Of course, Facebook, YouTube, and League will be more interesting than reading Huffington Post; coupled with the empirical decrease in the attention span of younger generations, humanity is headed towards disaster.

## What it does

Our project seeks to address this crisis by informing people in a novel and exciting way. We create a fully automated news extraction, summarization, and presentation pipeline that involves an AI-anime character news anchor. The primary goal of our project is to engage and educate an audience, especially that of younger students, with an original, entertaining venue for encountering reliable news that will not only foster intellectual curiosity but also motivate them to take into deeper consideration of relevant issues today, from political events to global warming.

The animation is basically a news anchor talking about several recent news, where related news is discussed in a short blurb.

## Demo Video Explanation

The demo video generally performs well, except for the first few seconds and the Putin/Taliban part. This is because the clusters are too small so many clusters get merged together as our kmeans has fixed number of clusters. A quick fix is to simply calculate the internal coherence of the cluster and filter based on that. more advanced methods can be based on those described in the Scatter Gather paper by Karger et al.

## How we built it

### News Summarization

For extraction and summarization, our first web scrapes news articles from trusted sources (CNN, New York Times, Huffington Post, Washington Post, etc…) to obtain the texts of recent news articles. Then it generates a compact summary of these texts using an in-house developed two-tier text summarization algorithm based on state-of-the-art natural language processing techniques. The algorithm first does an extractive summarization of individual articles. Next, it computes an overall 'topic feature' embedding. This embedding is used to cluster related news, and the final script is generated using these clusters and DL-based abstractive summarization.

### News Anchor Animation

Furthermore, using the google cloud text-to-speech API, we generate speech with our custom pitch and preferences and we then have code that generates a video using an image of any interesting, popular anime character. In order for the video to feel natural to the audience, we accounted for accurate lip and facial movement; there are calculations made using specific speech traits of the .wav file that produces realistic and not only educational but also humorous videos that will entertain the younger audience.

### Audience Engagement

Moreover, we wrote code using the Twitter API to automate the process of uploading videos to our Twitter account MinervaNews which is integrated within the project’s server that uploads a video initially when the server starts and automatically generates every 24 hours after a new video using the new articles from the sources.

## What's next for Minerva Daily News Reporter

Our project will have a lasting impact on the education of an audience ranging in all age groups. Anime is one great example of a venue that can broadcast news, and we selected anime characters as a humorous and eye-catching means to educate the younger audience. Our project and its customization allow for the possibility of new venues and greater exploration of making education more fun and accessible to a vast audience. We hope to take our project further and add more animations as well as more features.

## Challenges

Our compute platform, Satori has a unique architecture called IBM ppe64le that makes package and dependency management a nightmare.

## What we learned

8 hours in planning = 24 hours in real time.

## Github

<https://github.com/gtangg12/liszt>