# [eagleEye](https://aliu139.github.io/eagleEye/index.html)

Think Google Analytics... but for the real world

![Eagle Eye](https://raw.githubusercontent.com/aliu139/eagleEye/master/res/ee.png "EagleEye")

## Team

* [Austin Liu](https://github.com/aliu139)
* Benjamin Jiang
* Mahima Shah
* Namit Juneja

## Al

EagleEye is watching out for you. Our program links into already existing CCTV feeds + webcam streams but provides a new wealth of data, simply by pairing the power of machine learning algorithms with the wealth of data already being captured. All of this is then output to a simple-to-use front-end website, where you can not only view updates in real-time but also filter your existing data to derive actionable insights about your customer base. The outputs are incredibly intuitive and no technical knowledge is required to create beautiful dashboards that replicate the effects of complicated SQL queries. All data automatically syncs through Firebase and the site is mobile compatible.

## Technologies Used

* AngularJS
* Charts.js
* OpenCV
* Python
* SciKit
* Pandas
* FireBase

## Technical Writeup

Image processing is done using OpenCV with Python bindings. This data is posted to Firebase which continuously syncs with the local server. The dashboard pulls this information and displays it using Angular. Every several minutes, the front-end pulls the data that has not already been clustered and lumps it together in a Pandas dataframe for quicker calculations. The data is clustered using a modified SciKit library and reinserted into a separate Firebase. The chart with filters pulls from this database because it is not as essential for these queries to operate in real time.

All of the front-end is dynamically generated using AngularJS. All of the data is fetched using API calls to the Firebase. By watching $scope variables, we can have the charts update in real time. In addition, by using charts.js, we also get smoother transitions and a more aesthetic UI. All of the processing that occurs on the filters page is also calculated by services and linked into the controller. All calculations are done on the fly with minimal processing time.
## Inspiration'

With the rise of IoT devices and the backbone support of the emerging 5G technology, BVLOS drone flights are becoming more readily available. According to CBInsights, Gartner, IBISworld, this US$3.34B market has the potential for growth and innovation.

## What it does

**Reconnaissance drone software that utilizes custom object recognition and machine learning to track wanted targets.** It performs close to real-time speed with nearly 100% accuracy and allows a single operator to operate many drones at once. Bundled with a light sleek-designed web interface, it is highly inexpensive to maintain and easy to operate.

**There is a Snapdragon Dragonboard that runs physically on the drones capturing real-time data and processing the video feed to identify targets. Identified targets are tagged and sent to an operator that is operating several drones at a time. This information can then be relayed to the appropriate parties.**

## How I built it

There is a Snapdragon Dragonboard that runs physically on the drones capturing real-time data and processing the video feed to identify targets. This runs on a Python script that then sends the information to a backend server built using NodeJS (coincidentally also running on the Dragonboard for the demo) to do processing and to use Microsoft Azure to identify the potential targets. Operators use a frontend to access this information.

## Challenges I ran into

Determining a way to reliably demonstrate this project became a challenge considering the drone is not moving and the GPS is not moving as well during the demonstration. The solution was to feed the program a video feed with simulated moving GPS coordinates so that the system believes it is moving in the air.

The training model also required us to devote multiple engineers to spending most of their time training the model over the hackathon.

## Accomplishments that I'm proud of

The code flow is adaptable to virtually an infinite number of scenarios with virtually **no hardcoding for the demo** except feeding it the video and GPS coordinates rather than the camera feed and actual GPS coordinates

## What I learned

We learned a great amount on computer vision and building/training custom classification models. We used Node.js which is a highly versatile environment and can be configured to relay information very efficiently. Also, we learned a few javascript tricks and some pitfalls to avoid.

## What's next for Recognaissance

Improving the classification model using more expansive datasets. Enhancing the software to be able to distinguish several objects at once allowing for more versatility.
## Inspiration

We are both the type to see the world through numbers. Throughout the pandemic, articles would consistently be released touting the rising case numbers and with catchy headlines would off little in the way of data. We were both afraid but didn't know how worried we should be. Personally, I (Ben) looked for answers in the statistics released by the government. I was pleasantly surprised to see the data available was pretty detailed and specifically interested to see the data available for regions. Living in Toronto I based some of my choices around this data and knew places to avoid. If I went for a walk 4 km east I was in one of the most dangerous regions but sticking to the west was significantly safer. Together we decided this data should be made accessible so everyone can use it, so we created a project to do just that. Even if you aren't as neurotic as I am, I am sure anyone can find value from knowing the data.

## What it does

Our application uses publically available data that gets updated weekly or bi-weekly that each municipal government releases (pretty hard to find for the average Joe) and makes it accessible to everyone. Currently with a focus in Ottawa, our program lets, any user view a heat map that visually shows the most dangerous administrative regions that you should avoid (I now know more about the administrative regions of Ottawa than any student should). The heat map is weighted based on cases in the last 14 days per 100000 people. The user can select any marker that signifies each region and click it to see the number of cases reported in the last 14 days. Lastly, any user can add a marker on the map to see the exposure (risk) associated with going to that location. If the exposure is high where you plan to go then maybe you should rethink that trip. The exposure is based on the number of active cases in nearby regions.

## How we built it

We like to joke that we started the AWS session and never stopped it. In fact, the entire frontend is built in the react project set up from that session. On the frontend side we used a react based frontend, hosted by AWS. We import the Map component and associated resources from google-maps-react. Then we built a custom map component. Some data for the frontend is provided by the backend specifically the exposure score. The front react.js framework initiates a HTTP GET request to the backend server deployed on Heroku. The deployed server consists of a simple python webserver API built with Flask. The GET request provides the backend API methods with longitude and latitude coordinates so the backend processing endpoint can return an associated risk of exposure score related to the current longitude and latitudes. The risk score algorithm takes a weighted average of the N closest locations' exposure rates and returns a JSON response to the react frontend.

## Challenges we ran into

We both had no experience with the tools we were using. On the frontend side react was completely new, we had minimal experience with javascript and we had never used AWS. On the backend side...

Another challenge was getting the project to a place we were happy with. Especially with the frontend, projects like this can just grow so quickly, there are tons of features you always want to add. At some point, you just need to say there are 5 hours left, lets take what we have, clean it and present it nicely.

Lastly getting past CORs policy to make calls from the frontend to the backend. We are a two-person group so there was an obvious backend, frontend split. Sadly we never thought that getting data from one to another would take more than 30 minutes and 2 lines of code. We learned the hard way.

## Accomplishments that we're proud of

We are super proud we exclusively used tools we weren't familiar with. These past days have been equal parts learning and programming, we really left our comfort zone with this one. Yes, we are at a competition but we agreed it was more important we learn. Could we have made something impressive significantly faster and get much more sleep than we did? Maybe, but we wouldn't have learned nearly as much.

We are also really happy to have made a full-stack application. At the end of the day, we could have kept all the data used in the frontend files and referenced it without a backend. If we did that though this program would have no future. Eventually, most programs need a backend especially ones working with data. The python based backend can serve to fetch the new coronavirus numbers for us when it is updated. If we expand the system for new municipalities, it will eventually be too much to just hold in the front end.

## What we learned

We learned a bit about AWS, though I feel like we just touched the surface on that one. I was pleasantly surprised at the continuous integration offered by the amplifying console, I didn't know that was offered and it helped a lot. We definitely learned a ton about react, its structure, and specific components like the ones offered by Google. Where there is react there is Javascript and we learned a lot about javascript as well. From the backend, we learned a lot about how to create a python webserver API with Flask from scratch. In addition, we learned how to deploy our backend API to a remote server such as Heroku and learned a bit about configuring server resources such as adding on a PostgreSQL database. Overall, we definitely gained a lot of valuable experiences in debugging many different build technologies.

## What's next for safetrek

* Expand to new municipalities
* Add feature offering risk associated with traveling from one waypoint to a second (risk across path)
* Add feature offering alternate safer paths for travel
* Migrate more data from frontend files to requests to the backend
* Automate data updating to backend
* Integrate reverse-geocode, base code is there but never got it to fully work properly