## Inspiration

\_ play piano (p) \_

Playing a piano can be challenging for many, especially because of the instrument's difficulty to transport. To make piano playing more accessible, we sought to create a more portable alternative that would allow young musicians to practice and engage in this wonderful hobby, regardless of their environment or access to traditional pianos.

## What it does

Inspired from the elegance and art of learning how to play the piano, we have created an one octave piano visualizer for beginners to learn creatively using a display screen and a LED matrix. Our piano visualizer is amazing in terms of portability and provides a safe space for beginners to practice with no noise complaints :).

The piano visualizer displays the respective key note on the LCD display when the piano key is pressed, and the LED matrix displays different colours for each key. When a major or minor is played, a combination of colours is displayed on the matrix.

## How we built it

We used the Arduino kit components to produce the hardware assembly which includes dupont wires, a LCD display, a LED matrix, and push buttons. The piano keys are 3D printed using the Prusa printer.

## Challenges we ran into

* Connectivity issues with wires were a major challenge during the hardware setup phase. Some wires were not making proper contact with their respective connectors, causing intermittent disconnections. While a custom PCB or soldering could have been an ideal solution, the limited time during the hackathon made it challenging to implement such solutions.
* Another major challenge was cable management. The LCD screen and LED matrix required a significant number of digital pins, which led to a cluttered and tangled mess of wires. This made it difficult to trace wires and isolate issues during debugging. In hindsight, better planning and organization of the wires could have helped mitigate this issue.
* Figuring out how LCD monitor and adapting to limited resources (we were not able to get everything we wanted from the hackathon hardware team, so we had to improvise and make do with what we had - e.g. photoresistors/buttons)

## Accomplishments that we're proud of

* Adapting and ploughing through to the finish line. It was great to see the project come together and create this unique product

## What we learned

* 3D Printing takes time! There may be issues with our initial printed prototypes.
* More about hardware IO and design

## What's next for Pixel Piano

* Extending the prototype by adding more black keys
♫♫ ##Inspiration ♫♫

Wouldn't everyone like someone to sit in their house and serenade them on the piano? With any song they want? Wouldn't it be easier for everyone to learn songs on the piano if there was a machine that simply listened to songs and taught it to them? Why should you listen to music through a speaker when you could have it played live in your house? Introducing **Happy Keys**.

🎹🎹 ##What it does 🎹🎹

Happy Keys will...

1. Listen to any song that is either played/sang at the machine or an audio file uploaded to our software.
2. Simplify the song to a series of notes that can be played on the piano and their respective durations
3. Play this song for you on the piano

⚙️⚙️ ##How it works ⚙️⚙️

Software (identifying how to play the music): Fourier analysis is a method for expressing a function as a sum of its periodic components. Using a python library called Librosa, we used this analysis to simplify a complicated song into a series of notes composing the melody. We then measured how long these notes are played in the song to measure their respective durations. This information gets exported to an Arduino which in turn controls the hardware.

Hardware (playing the notes):
The hardware consists entirely of 2 micro controllers, 4 stepper motors and their drivers.
We were fortunate to have had 2 stepper motors beforehand which made the access to resources a little easier, but still challenging. The power must come from multiple sources since all the piano and different motors have different consumptions and requirements. The Piano must be plugged into the wall, the NEMA motors use batteries in series to get the recommended voltage. The 28BYJ stepper motors take advantage of the voltage regulator on the Arduinos and obtain their 5 volts from the batteries as well. An external battery pack is also used incase of deficient current.

💪💪 ##Challenges we ran into 💪💪

1. **Integrating python and C++ code in the Arduino IDE**. Our software was coded in Python but we coded our hardware in C++. Unfortunately, Arduino IDE best works with one file in one language, so this integration was generally difficult.
2. **Power source resources and wire management**. We had trouble finding enough power, with the right voltage and current to power our hardware.

🎩🎩 ## Accomplishments that we're proud of 🎩🎩

1. Some of us learned to code in C++ or Python.
2. Powering the stepper motors
3. Using Python to control the Arduinos
4. Using LT93D Arduino Nano and Nima Motor together

## What's next for Happy Keys

Currently, our project can only play 4 keys. This is obviously due to lack of materials and sufficient power. However, going forward, we plan to simply replicate the same machinery across more keys, possibly over the whole keyboard. We also coded the project in Python so the code is not as efficient as it could have been. Coding the project in C++ or a more efficient language would reduce any delay times we encountered and be able to play very advanced songs. Hopefully, at the same speed as a human. Finally, we would like to attach our machinery to the hardware of the piano (the inner machinery) in order to make our product work better.
## Inspiration

3D Printing offers quick and easy access to a physical design from a digitized mesh file. Transferring a physical model back into a digitized mesh is much less successful or accessible in a desktop platform. We sought to create our own desktop 3D scanner that could generate high fidelity, colored and textured meshes for 3D printing or including models in computer graphics. The build is named after our good friend Greg who let us borrow his stereocamera for the weekend, enabling this project.

## How we built it

The rig uses a ZED stereocamera driven by a ROS wrapper to take stereo images at various known poses in a spiral which is executed with precision by two stepper motors driving a leadscrew elevator and a turn table for the model to be scanned. We designed the entire build in a high detail CAD using Autodesk Fusion 360, 3D printed L-brackets and mounting hardware to secure the stepper motors to the T-slot aluminum frame we cut at the metal shop at Jacobs Hall. There are also 1/8th wood pieces that were laser cut at Jacobs, including the turn table itself. We designed the power system around an Arduino microcontroller and and an Adafruit motor shield to drive the steppers. The Arduino and the ZED camera are controlled by python over a serial port and a ROS wrapper respectively to automate the process of capturing the images used as an input to OpenMVG/MVS to compute dense point clouds and eventually refined meshes.

## Challenges we ran into

We ran into a few minor mechanical design issues that were unforeseen in the CAD, luckily we had access to a 3D printer throughout the entire weekend and were able to iterate quickly on the tolerancing of some problematic parts. Issues with the AccelStepper library for Arduino used to simultaneously control the velocity and acceleration of 2 stepper motors slowed us down early Sunday evening and we had to extensively read the online documentation to accomplish the control tasks we needed to. Lastly, the complex 3D geometry of our rig (specifically rotation and transformation matrices of the cameras in our defined world coordinate frame) slowed us down and we believe is still problematic as the hackathon comes to a close.

## Accomplishments that we're proud of

We're proud of the mechanical design and fabrication, actuator precision, and data collection automation we achieved in just 36 hours. The outputted point clouds and meshes are still be improved.