## Inspiration

Osu! players often use drawing tablets instead of a normal mouse and keyboard setup because a tablet gives more precision than a mouse can provide. These tablets also provide a better way to input to devices. Overuse of conventional keyboards and mice can lead to carpal tunnel syndrome, and can be difficult to use for those that have specific disabilities. Tablet pens can provide an alternate form of HID, and have better ergonomics reducing the risk of carpal tunnel. Digital artists usually draw on these digital input tablets, as mice do not provide the control over the input as needed by artists. However, tablets can often come at a high cost of entry, and are not easy to bring around.

## What it does

Limestone is an alternate form of tablet input, allowing you to input using a normal pen and using computer vision for the rest. That way, you can use any flat surface as your tablet

## How we built it

Limestone is built on top of the neural network library mediapipe from google. mediapipe hands provide a pretrained network that returns the 3D position of 21 joints in any hands detected in a photo. This provides a lot of useful data, which we could probably use to find the direction each finger points in and derive the endpoint of the pen in the photo. To safe myself some work, I created a second neural network that takes in the joint data from mediapipe and derive the 2D endpoint of the pen. This second network is extremely simple, since all the complex image processing has already been done. I used 2 1D convolutional layers and 4 hidden dense layers for this second network. I was only able to create about 40 entries in a dataset after some experimentation with the formatting, but I found a way to generate fairly accurate datasets with some work.

## Dataset Creation

I created a small python script that marks small dots on your screen for accurate spacing. I could the place my pen on the dot, take a photo, and enter in the coordinate of the point as the label.

## Challenges we ran into

It took a while to tune the hyperparameters of the network. Fortunately, due to the small size it was not too hard to get it into a configuration that could train and improve. However, it doesn't perform as well as I would like it to but due to time constraints I couldn't experiment further. The mean average error loss of the final model trained for 1000 epochs was around 0.0015
Unfortunately, the model was very overtrained. The dataset was o where near large enough. Adding noise probably could have helped to reduce overtraining, but I doubt by much. There just wasn't anywhere enough data, but the framework is there.

## Whats Next

If this project is to be continued, the model architecture would have to be tuned much more, and the dataset expanded to at least a few hundred entries. Adding noise would also definitely help with the variance of the dataset. There is still a lot of work to be done on limestone, but the current code at least provides some structure and a proof of concept.
# SmartKart

A IoT shopping cart that follows you around combined with a cloud base Point of Sale and Store Management system. Provides a comprehensive solution to eliminate lineups in retail stores, engage with customers without being intrusive and a platform to implement detailed customer analytics.

Featured by nwHacks: <https://twitter.com/nwHacks/status/843275304332283905>

## Inspiration

We questioned the current self-checkout model. Why wait in line in order to do all the payment work yourself!? We are trying to make a system that alleviates much of the hardships of shopping; paying and carrying your items.

## Features

* A robot shopping cart that uses computer vision to follows you!
* Easy-to-use barcode scanning (with an awesome booping sound)
* Tactile scanning feedback
* Intuitive user-interface
* Live product management system, view how your customers shop in real time
* Scalable product database for large and small stores
* Live cart geo-location, with theft prevention
## Inspiration

By attending the Health Panel held at TreeHacks, we had the opportunity to listen and better understand the needs of the health industry. Afterwards, we consulted with a surgeon, who mentioned that while scrubbed in and sterile in the operating room, it is very important to reference the scans containing important details relating to the surgical operation to avoid any mishaps.

## What it does

Ammalia is a Leap Motion assisted app that allows medical professionals to easily view and manipulate scans while scrubbed in, sterile and ready to perform a surgery.

The Leap Motion controller tracks the professionals hands using infrared cameras, motions such as swipe, grab and pinch were some of the movements integrated in our application. Authentication and database storage are made possible by using the Dropbox API

## Primary use cases

The areas we believe Ammalia could be the most useful is with radiology image manipulation / scrolling while scrubbed in & sterile in the OR.

## How we built it

**[Leap Motion API]**(<https://developer.leapmotion.com/>)

* With Leap Motion, you can interact with digital content in virtual and augmented reality, Mac and PC using your hands just as you would in the real world.
* The unique combination of software and hardware tracks the movement of hands and fingers with very low latency, converting in into 3D input.
* We used the Leap Motion API to program the gestures in order to manipulate the medical imaages just using our hands and the external Leap Motion hardware

**[Dropbox API]**(<https://www.dropbox.com/developers>)

* Dropbox is a personal cloud storage service that is frequently used for file sharing and collaboration
* Use Dropbox API for uploading/storing images of patient’s
* Use Dropbox API for user authentication as patient security is an issue in the current health care system

**[Flask]** (<http://flask.pocoo.org/>)

* Flask is a "microframework" primarily aimed at small applications with simpler requirements

## Challenges we ran into

* Working with the Leap Motion API was difficult since a limited number of our team members has experience with JS and front-end development in general.
* The gestures are very mathematically dependent so programming and calibrating the sensor was difficult to perfect.
* We had to utilize the help of the mentors available to us to get the Dropbox API to work correctly but we were able to get it working eventually.

## Accomplishments that we're proud of

* We were very excited when we successfully got the Leap Motion API to work with the actual hardware as we didn't think we'd be able to finish the hack in time.
* Getting the Dropbox to link up correctly with the medical images that had to be manipulated was challenging.

## What we learned

* Effie learned how to edit a webpage.
* Frances and Diana learned the basics of the Leap Motion API.
* Stephanie learned how to create a web app using Flask and linking it to the Dropbox API.

## What's next for Ammalia

Ammalia has endless possibilities as it can be applied to various industries including medical, engineering, mining, etc..

* We can see Ammalia potentially pivoting to include 3D manipulation of MRI and CT scans mid-procedure, which is important when, for example, determining the extent of a tumor in real-time.
* Another area we could look into is radiology and cardiology intervention which would actually involve catheter manipulation.
* Also, depending on the patient chosen, Ammalia could also showcase different health record data points that pertain to the patient being showcased (i.e. name, age, background, etc).