## Inspiration

Driven by the harrowing reality that civilians comprised a staggering 95% of all cluster munition casualties in 2022, we were impassioned to devise a solution that could confront this pressing humanitarian crisis head-on. With up to 40% of these munitions failing to explode upon impact, the threat to civilian lives persists long after conflict ceases. In regions like Laos, where between 1964 and 1973, 260 million cluster bomblets were dropped, and a chilling 80 million failed to detonate, the urgency of our mission became abundantly clear. Inspired by the imperative to safeguard innocent lives in post-war zones, we embarked on the development of Project Horus.

## What it does

Project Horus utilizes cutting-edge technology to detect and localize unexploded cluster munitions in conflict-affected areas. Using a Parrot drone equipped with a custom-trained Convolutional Neural Network (CNN), our system autonomously scans vast territories, identifying potential threats. The drone's onboard Inertial Measurement Unit (IMU) aids in precisely localizing the detected munitions, even in electronic warfare or GPS-denied environments. Results are visualized in real-time as a heatmap, providing actionable insights to demining teams and enabling targeted removal of these deadly remnants of war.

## How we built it

We started by creating a robust training dataset by printing out images of cluster munitions, placing them on the ground, and capturing a video to partition into frames. Our CNN, built using TensorFlow, achieved an impressive 98% validation accuracy after rigorous training. Integration with the Parrot drone involved deploying grid search algorithms to autonomously search for munitions, leveraging the drone's capabilities for real-time bomb detection. Overcoming challenges such as uploading custom assets into the simulator and optimizing the model to run efficiently on non-hardware accelerated systems, we created a streamlined solution ready for deployment in the field.

## Challenges we ran into

Throughout the development process, we encountered several challenges, including difficulties uploading custom assets into the simulator and optimizing the CNN to run efficiently on hardware-constrained systems. Despite these hurdles, we adapted our approach, moving the computer vision processing off the drone and onto a separate system, ensuring our solution remained viable and effective.

## Accomplishments that we're proud of

We're proud to have successfully implemented grid search and bomb-detection capabilities on the Parrot drone, as well as leveraging the IMU for precise localization of munitions. Additionally, visualizing the detection results in a heatmap provides actionable intelligence for demining teams, marking a significant achievement in our mission to save civilian lives.

## What's next for Project Horus

Moving forward, we aim to enhance the robustness of our model by incorporating synthetic and real-world data for fine-tuning. Furthermore, we envision implementing multi-agent reinforcement learning techniques to enable collaborative scanning by fleets of drones, enhancing efficiency in covering large areas. Ultimately, we aspire to deploy Project Horus alongside demining teams in regions such as Ukraine, Israel, Afghanistan, and Southeast Asia, contributing to the safe removal of cluster munitions and the protection of civilian populations.
## Inspiration

We were inspired by the recent interest of many companies in drone delivery and drone search. In particular, we wanted to bring drone abilities to the consumer – and we ended up doing even more.

There are many applications that can stem from our work, from search and rescue missions, drone delivery, or just finding your keys. In addition, we’ve brought the ability for a consumer to, with just their voice, train a classifier for object recognition.

[Voice Controlled Delivery Drone](https://youtu.be/8HKiQQVDcKQ)

[Real Time Search Drone](https://www.youtube.com/watch?v=CjqaV1Kw308)

## What it does

We build a pipeline that allows anyone to visually search for objects using a drone and simplified computer vision and machine learning.

It consists of mainly 3 parts:
1) A search drone (controlled normally with your phone) that performs image classification in real time for a given object
2) Being able to train an image classifier on any object by just using your voice.
3) A voice-controlled drone that can perform targeted delivery

## How we built it

We used an Amazon Echo to handle voice input, and the transcribed input was sent to a AWS Lambda server. Depending on the text, it was classified into one of several categories (such as commands). This server updated a Firebase database with the appropriate commands/information. Next, our local computers were notified whenever the database changed, and executed appropriate commands -- whether that be train an image classifier or fly a drone.

To get the non-programmable drone to become a search drone, we had it live stream its video feed to an Android phone, and we had a script that constantly took screenshots of the Android phone and stored them on our computer. Then we could use this images either for training data or to classify them in real time, using image segmentation and IBM Watson.

To train a classifier with only your voice, we would take the search term and use the Bing Search API to get images associated with that term. This served as the training data. We would then feed this training data into IBM Watson to build a classifier. This classifier could later be used for the search drone. All the consumer had to do was use their voice to do computer vision -- we took care of getting the data and applying the machine learning applications.

## Challenges we ran into

We are working with sandboxed technologies meant for the average consumer – but that are not developer friendly. It wasn't possible to take pictures or move the drone programmatically. We had to hack creative ways to enable this new capabilities for technologies, such as the screenshot pulling described above.

Additionally, the stack of coordinating communication with Alexa’s server, databases, and sending commands to the drone was quite a relay.

## Accomplishments that we're proud of

-Being able to create a super consumer-friendly way of training image classifiers.
-Taking a non-programmable drone and being able to hack with it still
-Being able to do voice control in general!

## What we learned

Hardware is finnicky.

## What's next for Recon

Even more precise control of the drone as well as potentially control over multiple drones.
## Inspiration

Hurricane Milton, the most devastating disaster in over 30 years, left more than **3 million people** without power and overwhelmed emergency services scrambling to respond. **While drone technology now floods us with vast amounts of data, the real challenge lies in making sense of it in rapidly evolving environments**—operators are still stuck manually sifting through critical information when time is running out. In moments of chaos, the ability to scale autonomous search-and-rescue missions and intelligently uncover patterns from data becomes essential.

SkySearch enables operators to uncover hidden insights on the environment in vast seas of data by integrating real-time data on the environment, telemetry, and previous missions into a single pane of glass (software mission control system). Operators can deploy fleets of drones to investigate regions and collect video feed used to reconstruct the scene, enabling them to drill-down on areas of interest through a semantic search engine.

## What it does

Our goal is to enable operators to **interact with data** and uncover hidden patterns effortlessly.

SkySearch is built around the end-to-end search-and-rescue workflow in the following use cases:

1. Search: Drones are deployed through the software by operators and autonomously navigate through terrain to identify objects of interest in real-time.
2. Rescue: Operators can interact with live data to isolate hazards and locate people through a unified search interface. Based on this data, the system then recommends risk-aware, optimized rescue routes for first responders.

Core features

* Environment reconstruction of damaged regions and infrastructure with Gaussian splatting
* Risk-aware pathfinding for rescue operations and pathfinding
* Semantic Search through disparate data sources to uncover patterns and recommend actions

## How we built it

We designed an embedded architecture that enables software and hardware interfaces to bidirectionally communicate information and commands.

* Drone SDK used for live video streaming
* TP-Link Antennas for a local wifi system to create a more robust data pipeline between the drone and the software interface - rather than relying on Satellites and Wifi
* OpenCV and Apple Depth Pro used to process footage and classify data
* SingleStore for real-time database management

## Challenges we ran into

* Accounting for low battery drones
* Integration between hardware and software interfaces
* Balancing human judgement with autonomy

## Accomplishments that we're proud of

* Implemented autonomous swarming framework to detect
* Integrated gaussian splatting
* Risk-aware map traversal and recommended "safe routes" for emergency responders
* Dynamic Data Generation to generate and query data dynamically allows for efficient testing and analysis, improving the app's responsiveness and visibility into critical information during rescue missions.