## Inspiration

In many developed countries across the world, the population is rapidly aging. This poses a variety of issues to senior citizens, including social isolation, an overburdened healthcare system unable to meet their needs, and the widespread effects of neurodegenerative conditions. We aimed to build a solution which would address all three of these issues in a way which is easily accessible and empowering to senior citizens.

## What it does

MemoryLane allows senior citizens to relive and share their cherished memories. The web application combines three main functionalities, which include a journaling and recall feature for important memories, an AI-powered match and chat system for users to discuss their experiences which are shared with other users, and an analytics dashboard which can be used by healthcare professionals to track key indicators of neurodegenerative conditions. Overall, MemoryLane allows users to not only keep their memories fresh but also weave a tapestry of connections with others with similar life experiences.

## How we built it

In order to develop a clean and responsive front-end and versatile back-end, we used Reflex.dev to develop entirely in Python. We also used the InterSystems IRIS database to easily perform vector search as well as other database operations to support the backend functionalities required by MemoryLane. Additionally, we made use of the Together.AI inference API to generate embeddings to match users based on shared experiences, perform sentiment analysis to find trends within memory recall data, and to create sample data to test our web app with. Finally, we used Google Cloud to implement speech-to-text functionality to increase ease of access to our platform for senior citizens. The majority of our app was built with Python, with a little JavaScript.

## Challenges we ran into

As 2 of our team members had never done full-stack dev before and one was attending his first hackathon, learning the nuances of new frameworks was initially a challenge, especially getting our environments set up. We’re incredibly grateful to the supportive mentors and sponsors for helping us get unstuck when we ran into issues, which indubitably helped us build our final product.

## Accomplishments that we're proud of

We’re very proud of our clean, intuitive UI which aims to make the product as accessible as possible to our target audience, senior citizens. Additionally, we believe that MemoryLane is a truly unique product which fills a niche which hasn’t been focused on before social media for the elderly, especially in combination with its potential benefits of improving the healthcare industry by aggregating data about the elderly.
Also, half of our team was able to go from near-zero web dev knowledge to familiarity with important tools and techniques, which we thought was very representative of the spirit of hackathons – coming together to meet new people and learn new things in a fast-paced creative environment.

## What we learned

Our journey with MemoryLane has been an enlightening dive into several new technologies. We harnessed the power of Reflex.dev for frontend and full stack development, explored the nuances in our data with InterSystems IRIS’s vector search on text embeddings from TogetherAI, and learned how to bring text to life with Google Cloud. Together AI has also become our ally in understanding our users' needs and narratives with natural language processing.

## What's next for MemoryLane

Looking to the horizon, we are definitely looking into expanding MemoryLane’s reach. Our roadmap includes scaling our solution and refining our data model to improve performance, and looking into business models which are sustainable and align with our mission. We envision forming partnerships with healthcare providers, memory care centers, and senior living communities. Integrating IoT could also redefine ease of use for seniors. Keeping innovation in mind, we'll dive deeper into Reflex's capabilities and explore bespoke AI models with Together AI. We aim to improve the technical aspects of our platform as well, including venturing into voice tone analysis to add another layer of emotional intelligence to our app. **We believe that MemoryLane is not just a walk in the past – it's a stride into the future of senior healthcare.**
## Inspiration

Our inspiration came from the importance of connecting with family and cherishing proud personal stories. We recognized that many elderly people in nursing homes feel isolated from their families, despite the wealth of memories they carry. These memories hold so much value about family history, wisdom, and identity. By creating a platform that enables them to reflect on and share these moments, we aimed to bridge generational gaps and strengthen family bonds. Through storytelling, we want to foster a tight family bond, ensuring that cherished memories are passed down and that the elderly feel heard, valued, and connected.

We wanted to emphasize the story aspect of these memories. When people want to share their memories with their family, especially virtually, they aren't able to fully relive or cherish that memory- a simple text message can't fully do justice to a fond memory. Thus, we wanted to bring life into these memories that shared within families online and especially provide elderly people who might not meet their families often to have an immersive experience with their family's memories.

## What it does

Memento allows families to document fond memories that they have, and share them to the user. Families can upload memories that contain a date, description, and image. We target this product to the elderly in nursing homes who are usually alone and can benefit from having someone like family to talk to. The elderly user can then speak to the application and can have a conversation about the details of any memory. The application will also display the most relevant image to the conversation to help improve the experience. This enables the elderly user to feel like they are talking to a family member or someone they know well. It allows them to stay connected with their loved ones without the continuous presence of them.

## How we built it

We designed Memento to be simple and accessible for both elderly users and their families. For this reason, we used **Reflex** to implement an elegant UI, and implemented a **Chroma** database to store the memories and their embeddings for search. We also integrated Whisper, a speech-to-text model through **Groq’s** fast inference API to decode what the elderly person is saying. Using this input, we query our database, and feed this information through **Gemini**, an LLM developed by Google, to give a coherent response that incorporates information from the families’ inputs. Finally, we used **Deepgram’s** text-to-speech model to convert the LLM’s outputs back to an audio format that we could speak back to the elderly user.

## Challenges we ran into

* **Integration**: It was difficult to integrate all of the sponsor’s softwares into the final application; we had to pore over documentation while becoming familiar with each API, which led to many hours of debugging.
* **Non-determinism**: Our models were non-deterministic; errors caused by specific outputs from the LLM were hard to replicate. Due to the background noise, we also could not efficiently test our speech-to-text model’s accuracy.
* **Inference speed**: Throughout this application, we make many API calls to large models, such as Whisper, Gemini, and the Aura TTS model. Because of this, we had to find clever optimizations to speed up the inference time to quickly speak back to the elderly user, especially since the WiFi was unusable most of the time.

## Accomplishments that we're proud of

* **Design and User Experience**: We are proud of our design since it encompasses the mood we were aiming for – a warm, welcoming environment, focusing on the good things that happen in life.
* **Large Language Model and Vector Search**: We are especially proud of how the LLM turned out and how well the RAG model worked. We spent lots of time prompting the different components to create the warm, empathetic, and welcoming environment the LLM provides.
* **TTS and STT**: Although we struggled a bit with this part, we are really proud about how it turned out. We feel we did a great job encompassing the ideals of the product by allowing users to reflect on past memories and connect closer with family.

## What we learned

Working with STT and TTS models: many members of our group had never worked with speech-to-text or text-to-speech models, so this was a learning experience for all of us. We learned about the impressive accuracy that the state-of-the-art models are able to achieve but also encountered some of the drawbacks of these models, since many of them don’t work as well with moderate levels of background noise.
How to make a great UI:

## What's next for Memento

Because of time constraints, there were many features and improvements we wanted to implement but could not.

* **Continuous LLM Conversation**: We wanted to be able to talk to the LLM continuously without having to press a microphone button. Due to time constraints, we were not able to implement this feature
* **User Personalization and Customization**: We aimed to personalize the website to users by adding custom themes, colors, and fonts, but we ran out of time to do so.
Link to Website: <http://www.ewharton.ml/>

## Inspiration

We noticed there was a severe lack of good virtual tour options, especially ones that were fun and exciting. So we decided to combine a pretty serious matter with something a little more light-hearted, Roblox. By building a virtual campus, you can get a much better experience by running around and exploring than by simply clicking through images.

## What it does

We constructed almost the entire Wharton building in Roblox. We also built a fully responsive website with images from the virtual campus and a virtual tour feature. Within the campus we have "tour guides" that will give players useful information as they tour the campus and our virtual tour has an easy to use interface to navigate throughout the campus. We also integrated a social experience into the project, adding a music festival on Koo Family Plaza. With our virtual campus, you can hang out with friends at the music festival or explore the Wharton Complex. The campus is fully interactive and allows the player to feel like they really are at UPenn.

## How I built it

We built the website from scratch using html, javascript, and css.
We constructed the campus in Roblox studio using lua. Some of the assets were created in Blender and imported into Roblox studio. We mainly chose Roblox because of its intuitive game framework and asset modeling program. The program, called Roblox studio, allowed us to additively and subtractively create a variety of buildings. The backend on the Roblox side was mainly initializing the player, assigning each one a dorm number based on the number of vacant dorms in the game.

## Challenges I ran into

One the biggest challenges we ran into was building a framework for the virtual tour. It took us several hours to construct a framework that we could use to add components as we built more. Another challenge was just simply getting so many of the components of the buildings and campus done in time. We decided to just do the Wharton building because of our time constraints, but both the interior and exterior are fully decked out. It was also difficult to construct the Wharton complex because none of the team attends UPenn, so we had to research the campus from online images and Google map services.

## Accomplishments that I'm proud of

We're proud of how the final website looked and definitely how complete and realistic the Wharton Complex feels, with a 1:1 ratio.

## What I learned

We learned a lot about 3D design when constructing the campus and it took a lot of research to make many of the components of the website work.

## What's next for eWharton

The next steps would be to build the entire uPenn campus!