## Bringing your music to life, not just to your ears but to your eyes 🎶

## Inspiration 🍐

Composing music through scribbling notes or drag-and-dropping from MuseScore couldn't be more tedious. As pianists ourselves, we know the struggle of trying to bring our impromptu improvisation sessions to life without forgetting what we just played or having to record ourselves and write out the notes one by one.

## What it does 🎹

Introducing PearPiano, a cute little pear that helps you pair the notes to your thoughts. As a musician's best friend, Pear guides pianists through an augmented simulation of a piano where played notes are directly translated into a recording and stored for future use. Pear can read both single notes and chords played on the virtual piano, allowing playback of your music with cascading tiles for full immersion. Seek musical guidance from Pear by asking, "What is the key signature of C-major?" or "Tell me the notes of the E-major diminished 7th chord." To fine tune your compositions, use "Edit mode," where musicians can rewind the clip and drag-and-drop notes for instant changes.

## How we built it 🔧

Using Unity Game Engine and the Oculus Quest, musicians can airplay their music on an augmented piano for real-time music composition. We used OpenAI's Whisper for voice dictation and C# for all game-development scripts. The AR environment is entirely designed and generated using the Unity UI Toolkit, allowing our engineers to realize an immersive yet functional musical corner.

## Challenges we ran into 🏁

* Calibrating and configuring hand tracking on the Oculus Quest
* Reducing positional offset when making contact with the virtual piano keys
* Building the piano in Unity: setting the pitch of the notes and being able to play multiple at once

## Accomplishments that we're proud of 🌟

* Bringing a scaled **AR piano** to life with close-to-perfect functionalities
* Working with OpenAI to synthesize text from speech to provide guidance for users
* Designing an interactive and aesthetic UI/UX with cascading tiles upon recording playback

## What we learned 📖

* Designing and implementing our character/piano/interface in 3D
* Emily had 5 cups of coffee in half a day and is somehow alive

## What's next for PearPiano 📈

* VR overlay feature to attach the augmented piano to a real one, enriching each practice or composition session
* A rhythm checker to support an aspiring pianist to stay on-beat and in-tune
* A smart chord suggester to streamline harmonization and enhance the composition process
* Depth detection for each note-press to provide feedback on the pianist's musical dynamics
* With the up-coming release of Apple Vision Pro and Meta Quest 3, full colour AR pass-through will be more accessible than ever — Pear piano will "pair" great with all those headsets!
## Inspiration

Pianos are usually heavy and expensive, so a keyboard made from cheap and portable materials like paper can be very useful. That's why we came up with the paper piano idea. With a paper piano, it's natural to connect it to a computer and achieve multiple purposes like composing. While most composers rely on midi file export to view and make adjustment to their compositions, we wish to realize real-time visual representation of the music being played so people can view their music notes in real time.

## What it does

Pianeer is a highly innovated and convenient music composing software+hardware system. Multiple modes are available for different musical functions. It includes a paper piano which provides real experience of playing keyboard but can be easily rolled up and carried. The composing mode of the software provides real time visual representation of the music that's being played and exports a midi file. The play mode provides sheet music and keyboard highlights for practicing purpose. The practice mode allows users to practice their compositions casually without being recorded. Pianeer is extremely multi-functional and portable, and can be used for both beginners and mature composers.

## How we built it

We put electric paint on a hard paper and connect it to arduinos via wires. Using capacitive sensing library we are able to give input signals by simply putting fingers on the paper. We exported the input signal to our program to generate sound and realize other functions.

## Challenges we ran into

* It's very hard to accurately draw so many keys on the paper using electric paint since each key has to be clearly separated.
* Adjusting delay and accuracy of the paper piano is very challenging.
* Convert music input signals to visual representation.
* User Interface design.

## Accomplishments that we're proud of

* Lovely user interface that corresponds to the superhero theme.
* Pretty accurate sensing and sound generation of the paper piano.
* Real time visual representation of the music
* Various functions and modes we've accomplished.

## What we learned

Pygame, arduino, multi-threading

## What's next for Pianeer

Make a more delicate prototype of the paper piano and realize the player mode we have no time to accomplish this time.
## Inspiration

Social networks have become increasingly prolific, and the number of widely used social media platforms has been growing every year. Because of the lack of integration between social networking platforms, adding a person on multiple social media outlets remains a time consuming task and annoying task. Making friends and meeting new people should not be this difficult.

## What it does

Social Self is a social media aggregator. Simply sign up on socialself.net with your facebook account, and add your ID for your social media accounts. The website will generate a QR code, which you can port to the iOS application. When the QR code is scanned within the app,it will prompt the recipient with a scrolling page view with all of the user's available social media accounts.

## How we built it

Swift and Objective-C for iOS development. HTML, CSS, JavaScript, and Bootstrap for front end. PHP and SQL for back end.

## Challenges we ran into

We needed a database to store all of our user account info, but none of use knew any backend. As a result, creating and trying to use a SQL server ended up being a huge time sink that in the end, wasn't feasible to implement in the scope of this hackathon.

We first set up a Microsoft Azure MSSQL database, however, connecting to it through php took quite a while, and a lot of troubleshooting. Unfortunately, even after we managed to create a connection, we still couldnt get mssql to work. We then switched to MySQLi & php on a localhost server. Setting up the localhost server was also very time consuming. We were able to successfully make a backend module with php and MySQLi that allowed developers to add users, and update/retrieve their info. However, we were unable to find a server to host our database, as our Azure plan did not allow mySQL hosting, and we never heard back from AWS after we signed up. We were also worried about our database security (hint: it wasn't), and integrating with our front end. With a deadline looming over our heads, and other areas to focus on, we decided to scrap it in and save it for a later date.

We also ran into trouble developing an android platform, integrating facebook login onto the website, redirecting our github.io website to our domain, and using git.

## Accomplishments that we're proud of

Incredibly fast execution: it takes less than 30 seconds to scan the QR code and add the person on all social media accounts. Now that's fast.

Plus we are a group of first years who met for the first time at this hackathon. For 3 of us, it was our first time using git for team production. We created something worth presenting, and we're all tremendously proud of that.

## What we learned

Understand the capabilities of every technology that we consider using. We can't assume that we can accomplish something with a given technology just because it seems like it could work, especially not when we're under such a strict schedule.

## What's next for Social Self

This is a service that makes sense. We're planning to really polish our execution and utilize the technologies that we didn't have time to explore while here at QHacks. Our current product has a lot of potential to go much further.

Obviously, we'll have to set up a database for ourselves.

One thing that we're looking into immediately is adding more social platforms. Currently, we support facebook, instagram, twitter, and linkedin profiles. We're working on adding options to add email, phone numbers, snapchat profiles, personal websites, and possibly more social platforms in the future.

We're also working on easier integration and sign up - instead of users adding their usernames for each account manually, we're working on options for users to connect to their accounts with the click of a button through the various social media API. We've managed to add logins with facebook, and there's many more to follow.

Another idea that has potential is creating multiple configurations for users - for example, a user could have one QR code for business (shares your email, phone number, linkedin, and personal website), and one for pleasure (facebook, snapchat, twitter). Using SocialSelf in the future could be as normal as business cards today.

SocialSelf also has the potential as to be a social media content manager - managing all your different platforms, posting, viewing content, etc - through one site and one app. This would not only be a great convenience to regular users, but could also be useful for PR for companies, marketing campaigns, and celebrities, who need to be able to maintain their image on all their social platforms.