## Inspiration

Among many things, news cycles are jam-packed with polls. Turn on the nightly broadcasts and you'll be hit by a wave of numbers, from the latest statistics about the percentage of millenials who believe in god to the number of adults between the age of 65 and 80 that use Spotify. During election years, in particular, political polls seem to be all the rage- every other headline seems to be some variation of "[politician]'s popularity dropped by [x] points following [insert significant world event here].

So famous media outlets like CNN or Fox deem that political polls make great headlines, but how accurate are they? How much faith should we put into polls, particularly political polls that aim to predict the outcome of major elections? Who conducts these polls, and how do they decide who to ask? And how up to date are the numbers?

## What it does

Given how politically charged this particular election year has been, from the COVID-19 outbreak to the Trump impeachment trials to the George Floyd movement, our team thought it'd be interesting to present our take on an alternative to political polls.

**PolitiSense** scrapes data on hundreds of thousands of headlines, posts and comments across the internet, gathering the opinions of the masses from tweets on Twitter or subreddits on Reddit while also sifting through volumes of news headlines from major media outlets such as Fox or CNN. We then see what people are saying about a particular political figure at any given time, analyzing sentiment with word content and level of engagement (number of likes, retweets, upvotes, etc) and aggregating into a single sentiment score.

Using the search feature, find out how the internet feels about a political figure in real-time!

## How we built it

### Front End

* HTML/CSS
* Vanilla javascript with a hint of charts.js

### Back End

* Flask for serving templates and running scraping tasks
* MongoDB to store scraped data
* Python and various APIs (beautiful soup, pushshift, tweepy, etc) for scraping data
* NLTK for sentiment analysis
* Google cloud hosting backend

## Challenges we ran into

* various frustrating front end tasks (e.g. the search bar that wouldn't center)
* Getting backend functions up and running on google cloud
* Scraping data from a variety of sources on the internet
* Formatting data for sentiment analysis
* Handoff between front end and back end
* Trying to get a twitter developer account, panicking because you googled and it said that the average developer account gets approved in 19 days, but then realizing you still have a valid set of authentication tokens from a twitter bot tutorial from the high school coding club five years ago

## Accomplishments that we're proud of

First time our team has built a coherent end to end system from front end to back end, so we're pretty excited about that. Also proud of the animation of the search bar when the webpage first loads because god knows how long that took

## What we learned

Exposure to various APIs, but mainly learned how to smoothly connect the backend to the frontend to deliver a product.

## What's next for PolitiSense

Our team thinks there's a lot more that can be done-- our minimum viable product implements only the barest of features, which is the ability to search one politician at a time, with data only from one day prior.
In the future, we'd like to add:

* filters adjusting time range of data scraped
* analysis of sentiment on not just politicians, but also political issues or current events in general
* general dashboard that shows data breakdowns of how the aggregate sentiment score was achieved (for example, the sentiment scores from Reddit vs Twitter vs CNN)
* more data being scraped (add in facebook, more subreddits, instagram, etc)
* Live sentiment tracker for events such as political debates
* improve sentiment score calculation algorithms
* tidy up code because there were some questionable segments in there
## Inspiration

An informed electorate is as vital as the ballot itself in facilitating a true democracy. In this day and age, it is not a lack of information but rather an excess that threatens to take power away from the people. Finding the time to research all 19 Democratic nominee hopefuls to make a truly informed decision is a challenge for most, and out of convenience, many voters tend to rely on just a handful of major media outlets as the source of truth. This monopoly on information gives mass media considerable ability to project its biases onto the public opinion. The solution to this problem presents an opportunity to utilize technology for social good.

## What it does

InforME returns power to the people by leveraging Google Cloud’s Natural Language API to detect systematic biases across a large volume of articles pertinent to the 2020 Presidential Election from 8 major media sources, including ABC, CNN, Fox, Washington Post, and Associated Press. We accomplish this by scraping relevant and recent articles from a variety of online sources and using the Google Cloud NLP API to perform sentiment analysis on them. We then aggregate individual entity sentiments and statistical measures of linguistic salience in order to synthesize our data in a meaningful and convenient format for understanding and comparing the individual biases major media outlets hold towards or against each candidate.

## How we built it and Challenges we ran into

One of the many challenges we faced is learning the new technology. We dedicated ourselves to learning multiple GCP technologies throughout HackMIT from calling GCP API to serverless deployment. We employed Google NLP API to make sense of our huge data set scraped from major news outlets, Firebase real-time database to log data, and finally GCP App Engine for deployments of our web apps. Coming into the hackathon with little experience with GCP, we found the learning curve to be steep yet rewarding. This immersion in GCP technology renders us a deeper understanding of how different components of GCP work together, and how much potential GCP has for contributing to social good.

Another challenge we faced is how to represent the data in a visually meaningful way. Though we were able to generate a lot of insightful technical data, we chose to represent the data in a straightforward, easy-to-understand way without losing information or precision. It’s undoubtedly challenging to find the perfect balance between technicality and aesthetics, and our front-end design tackles this task of using technology for social good in an accessible way without compromising the complexity of current politics. Just as there’s no simple solution to current social problems, there’s no perfect way to contribute to social good. Despite this, InforME is an attempt to return power to the people, providing for a more just distribution of information and better informed electorate, a gateway to a society where information is open and accessible.

## What's next for InforME

Despite our progress, there is room for improvement. First, we can allow users to filter results by dates to better represent data in a more specific time range. We can also identify pressing issues or hot topics associated with each candidate via entity sentiment analysis. Moreover, with enough data, we can also build a graph of relationships between each candidates to better serve our audience.
## Inspiration

Prolonged covid restrictions have caused immense damage to the economy and local markets alike. Shifts in this economic landscape have led to many individuals seeking alternate sources of income to account for the losses imparted by lack of work or general opportunity. One major sector that has seen a boom, despite local market downturns, is investment in the stock market. While stock market trends at first glance, seem to be logical, and fluid, they're in fact the opposite. Beat earning expectation? New products on the market? *It doesn't matter!*, because at the end of the day, a stock's value is inflated by speculation and **hype**. Many see the allure of rapidly increasing ticker charts, booming social media trends, and hear talk of town saying how someone made millions in a matter of a day *cough* **GameStop** *cough* , but more often then not, individual investors lose money when market trends spiral. It is *nearly* impossible to time the market. Our team sees the challenges and wanted to create a platform which can account for social media trends which may be indicative of early market changes so that small time investors can make smart decisions ahead of the curve.

## What it does

McTavish St. Bets is a platform that aims to help small time investors gain insight on when to buy, sell, or hold a particular stock on the DOW 30 index. The platform uses the recent history of stock data along with tweets in the same time period in order to estimate the future value of the stock. We assume there is a correlation between tweet sentiment towards a company, and it's future evaluation.

## How we built it

The platform was build using a client-server architcture and is hosted on a remote computer made available to the team. The front-end was developed using react.js and bootstrap for quick and efficient styling, while the backend was written in python with flask. The dataset was constructed by the team using a mix of tweets and article headers. The public Twitter API was used to scrape tweets according to popularity and were ranked against one another using an engagement scoring function. Tweets were processed using a natural language processing module with BERT embeddings which was trained for sentiment analysis. Time series prediction was accomplished through the use of a neural stochastic differential equation which incorporated text information as well. In order to incorporate this text data, the latent representations were combined based on the aforementioned scoring function. This representation is then fed directly to the network for each timepoint in the series estimation in an attempt to guide model predictions.

## Challenges we ran into

Obtaining data to train the neural SDE proved difficult. The free Twitter API only provides high engagement tweets for the last seven days. Obtaining older tweets requires an enterprise account costing thousands of dollars per month. Unfortunately, we didn’t feel that we had the data to train an end-to-end model to learn a single representation for each day’s tweets. Instead, we use a weighted average tweet representation, weighing each tweet by its importance computed as a function of its retweets and likes. This lack of data extends to the validation side too, with us only able to validate our model’s buy/sell/hold prediction on this Friday's stock price.
Finally, without more historical data, we can only model the characteristics of the market this week, which has been fairly uncharacteristic of normal market conditions. Adding additional data for the trajectory modeling would have been invaluable.

## Accomplishments that we're proud of

* We used several API to put together a dataset, trained a model, and deployed it within a web application.
* We put together several animations introduced in the latest CSS revision.
* We commissioned McGill-themed banner in keeping with the /r/wallstreetbets culture. Credit to Jillian Cardinell for the help!
* Some jank nlp

## What we learned

Learned to use several new APIs, including Twitter and Web Scrapers.

## What's next for McTavish St. Bets

Obtaining much more historical data by building up a dataset over several months (using Twitters 7-day API). We would have also liked to scale the framework to be reinforcement based which is data hungry.