## Inspiration

Our team was inspired to work on this project mainly due to two reasons:

1. As a Google Home owner, I regularly use voice-commands to control my smart home devices such as my Philips Hue lights and Wemo Smart Plugs. However, one Saturday morning, after a night of alcohol and karaoke, I opened my mouth in an attempt to turn on the lights, but the only sounds that came out of my mouth were “hey ggg...gg…, t…the...lights”. I’ve lost my voice, and along with it, my ability to turn on the lights! And then another thought hit me: how can the 18 million US adults (8% of the US population) who had suffered from voice problems in 2015, or the other billions of people who don't speak the language or have speech impairments, turn on and off their lights? Bam! An idea was born.
2. After experimenting with its capabilities and data output, our team immediately decided to create a hardware-integrated hack involving the Myo Armband. As a team of 4 mechanical engineers and 1 programmer, it was clear from the beginning that our best chances of success would be a hardware hack with either a web or mobile app GUI.

## What it does

Using the **Myo Armband**, users can control their smart home devices such as lights and fans at the flick of their wrist and arm. Based on how many devices there are, users can use the **MyoHome web app** to configure the gestures for each device, and save those customized gestures to a user profile. Although there are only 5 basic gestures built in to the **Myo Connect** app, there are essentially an infinite number of gestures that could be configured based on the raw data output from the **accelerometer**, **EMG sensors**, and **gyroscope**.

## How we built it

At first, we performed diagnostic tests to determine the accuracy, range, and capabilities of the Myo Armband through Myo Connect to get a feel of the range of motions and the capabilities of this armband. We then split into sub-teams to tackle the different challenges, including: **website and interface design, electronics control, mock house model design, and demo/presentation strategy**. Finally it all came back together with our completed prototype that shows how the Myo armband can truly be integrated into our lives and enable everyone - whether you’re mute or having speech difficulties - to fully enjoy smart home technologies.

## Challenges we ran into

* Designing a C++ application that could smoothly and efficiently bind to Myo’s C++ bindings

## Accomplishments that I'm proud of

* Prepared a powerful presentation of our proof-of-concept demonstration with our beautifully-built home
* Developed gesture combinations that are not native to the Myo SDK based on sensors data

## What we learned

* How to control hardware using a Myo Armband and Arduino

## What's next for MyoHome

Due to time constraint, the MyoHome app currently only works with hacked arduino-enabled LEDs and motors. The natural progression of this project would be towards enabling integration with smart home products and apps currently on the market from tech giants like Google and Samsung, as well as from other companies like Philips, TP-Link, and Wemo.
## Inspiration

MISSION: Our mission is to create an intuitive and precisely controlled arm for situations that are tough or dangerous for humans to be in.

VISION: This robotic arm application can be used in the medical industry, disaster relief, and toxic environments.

## What it does

The arm imitates the user in a remote destination. The 6DOF range of motion allows the hardware to behave close to a human arm. This would be ideal in environments where human life would be in danger if physically present.

The HelpingHand can be used with a variety of applications, with our simple design the arm can be easily mounted on a wall or a rover. With the simple controls any user will find using the HelpingHand easy and intuitive. Our high speed video camera will allow the user to see the arm and its environment so users can remotely control our hand.

## How I built it

The arm is controlled using a PWM Servo arduino library. The arduino code receives control instructions via serial from the python script. The python script is using opencv to track the user's actions. An additional feature use Intel Realsense and Tensorflow to detect and track user's hand. It uses the depth camera to locate the hand, and use CNN to identity the gesture of the hand out of the 10 types trained. It gave the robotic arm an additional dimension and gave it a more realistic feeling to it.

## Challenges I ran into

The main challenge was working with all 6 degrees of freedom on the arm without tangling it. This being a POC, we simplified the problem to 3DOF, allowing for yaw, pitch and gripper control only. Also, learning the realsense SDK and also processing depth image was an unique experiences, thanks to the hardware provided by Dr. Putz at the nwhacks.

## Accomplishments that I'm proud of

This POC project has scope in a majority of applications. Finishing a working project within the given time frame, that involves software and hardware debugging is a major accomplishment.

## What I learned

We learned about doing hardware hacks at a hackathon. We learned how to control servo motors and serial communication. We learned how to use camera vision efficiently. We learned how to write modular functions for easy integration.

## What's next for The Helping Hand

Improve control on the arm to imitate smooth human arm movements, incorporate the remaining 3 dof and custom build for specific applications, for example, high torque motors would be necessary for heavy lifting applications.
## Inspiration

Members of our team know multiple people who suffer from permanent or partial paralysis. We wanted to build something that could be fun to develop and use, but at the same time make a real impact in people's everyday lives. We also wanted to make an affordable solution, as most solutions to paralysis cost thousands and are inaccessible. We wanted something that was modular, that we could 3rd print and also make open source for others to use.

## What it does and how we built it

The main component is a bionic hand assistant called The PulseGrip. We used an ECG sensor in order to detect electrical signals. When it detects your muscles are trying to close your hand it uses a servo motor in order to close your hand around an object (a foam baseball for example). If it stops detecting a signal (you're no longer trying to close) it will loosen your hand back to a natural resting position. Along with this at all times it sends a signal through websockets to our Amazon EC2 server and game. This is stored on a MongoDB database, using API requests we can communicate between our games, server and PostGrip. We can track live motor speed, angles, and if it's open or closed. Our website is a full-stack application (react styled with tailwind on the front end, node js on the backend). Our website also has games that communicate with the device to test the project and provide entertainment. We have one to test for continuous holding and another for rapid inputs, this could be used in recovery as well.

## Challenges we ran into

This project forced us to consider different avenues and work through difficulties. Our main problem was when we fried our EMG sensor, twice! This was a major setback since an EMG sensor was going to be the main detector for the project. We tried calling around the whole city but could not find a new one. We decided to switch paths and use an ECG sensor instead, this is designed for heartbeats but we managed to make it work. This involved wiring our project completely differently and using a very different algorithm. When we thought we were free, our websocket didn't work. We troubleshooted for an hour looking at the Wifi, the device itself and more. Without this, we couldn't send data from the PulseGrip to our server and games. We decided to ask for some mentor's help and reset the device completely, after using different libraries we managed to make it work. These experiences taught us to keep pushing even when we thought we were done, and taught us different ways to think about the same problem.

## Accomplishments that we're proud of

Firstly, just getting the device working was a huge achievement, as we had so many setbacks and times we thought the event was over for us. But we managed to keep going and got to the end, even if it wasn't exactly what we planned or expected. We are also proud of the breadth and depth of our project, we have a physical side with 3rd printed materials, sensors and complicated algorithms. But we also have a game side, with 2 (questionable original) games that can be used. But they are not just random games, but ones that test the user in 2 different ways that are critical to using the device. Short burst and hold term holding of objects. Lastly, we have a full-stack application that users can use to access the games and see live stats on the device.

## What's next for PulseGrip

* working to improve sensors, adding more games, seeing how we can help people

We think this project had a ton of potential and we can't wait to see what we can do with the ideas learned here.

## Check it out

<https://hacks.pulsegrip.design>

<https://github.com/PulseGrip>