Virtual Memory Palace

THE IDEA

Remembering is a form of creating. Given that memory is subjective, imperfect, and elusive – remembering can be thought of more as re-creating than simply as straightforward information retrieval.

Memory Architect is a project about Space and Memory. We created a Virtual Reality framework that allows a user to compose past experiences, organize their memories, and create new connections between them in a spatially immersive virtual environment. We envision utilizing space as an interface. This project utilizes VR spaces as a way to augment this inherent mechanism of re-creating through remembering. Through being able to physically walk in a virtual space, elements such as motion, pace, volume, material, mass, and flow become tools of engaging the digital world.

WHAT IS A MEMORY PALACE?

Cicero, 55 BCE

Emma Willard, 1851

Ancient Greek & Roman thinkers such as Cicero commonly used the method of a Memory Palace. It is a mnemonic technique of mentally walking through a familiar environment, such as your home, in order to evoke particular memories.

When creating this Memory Palace, you first visualize something (objects, faces, etc.) that you associate with the memory. You mentally place this artifact in a specific location of the environment. As this process continues, you eventually have a series of associated images along a path within your memory palace. When it is time to recall information, while giving a speech for example, you visualize walking through the path along your palace. Each location has a defined association to a specific memory.

Physically Walking Through The Virtual Memory Palace

PROCESS

System Ideation

Iterations of UX Sequence

UX Sketches

Developing Interface

Sketching Memory Spaces

Designing Memory Space

Embedding Portals

Unity 3D Implementation

User Uploaded Imagery

User Testing VR Memory Palace

Human Computer Interaction & Symbiosis.

This project was part of a class at MIT's Media Lab called Human Computer Interaction & Symbiosis. The prompt was to conceive, research, design, and prototype a technology that augments, not replaces, a natural human property.

After some initial research & brainstorming sessions, our team was very interested in the relationship between memory & physical space. Moreover, we were very intrigued in the fact that human memory is not perfect - and we liked that! It is subjective, at times illusive, and continually changing. In fact, we loved that remembering could be thought of more as re-creating than simply as straightforward information retrieval. This is where we wanted to focus:

"How can we leverage spatial & immersive interfaces to augment the creative aspect of human memory?"

This is when the VR Memory Palace was conceived. Moreover, we imagined the users to be creative individuals undergoing a learning experience. We felt very connected to this type of user.

Designing a Interface for Creativity & Discovery.

How could this system help users to learn or create new things?

Now our team had to put together a project proposal for the class with thorough research, a project road map with specific goals, and a clear research question.

One of our first collective decisions was to create a system that is not only accessible through VR, but that can be built through a mobile app. We wanted to utilize the unique benefits of each mediums (mobile & VR) to strengthen the overall interaction.

My role here was to, based on the teams decisions, propose a series of interactions that showed a user's flow while using the system from throughout the process. The primary structure of the interface was to "Create, Log, and Explore."

Overall, there were three components to the project that we had to think through & design simultaneously:

How the user creates a memory palace (through the App)
How to utilize 3D space as the interface for the user's memories (designing space for memory).
How the system functions to meld these two forms of interaction seamlessly.

Walkable Digital Environment

The project proposal yielded a lot of valuable feed back. Now it was time to fine-tune, implement, and prototype. The most valuable decision we made at this point, was to develop the system through Google Tango. We wanted the user to be able to physically walk through a virtual environment. Overall there were three main aspects to develop for the Alpha phase of the prototype:

VR Environment Design
System Integration through Unity & Tango
User Interactions

My role at this point was to design a series of VR Environments (the Memory Palaces) to test how a user would move through them to explore their memories. Similarly, I was to develop the interactive aspects of both the memory palace and the mobile app. My partner, a computer scientist, was in charge of system integration: how do Unity, Tango, and the mobile app talk to each other.

User Interactions in Virtual Space

We tested two different iterations of a memory palace through Tango. Here we thought it would be very interesting to be able to have portal between different palaces - this method could essentially "remix" memories and potentially create new associations between otherwise distant experiences.

One of the more interesting and engaging interactions we developed was the Memory Sphere. We populated the virtual palaces with a series of reflective objects. Each memory sphere was embedded with a unique memory. For example, a sound file of a favorite class lecture, or an image of a dear quote. The content of the Memory Sphere would only be accessed (viewed or heard) when the person came in close proximity to the memory sphere.

Similarly, through the mobile app, the user was able to take pictures, videos, sound clips, or files and upload them to their specified VR Memory Palace. Once uploaded, the person could get on Tango, walk around the palace, and view the uploaded files of their memory instances.

VR APP INTERFACE

Our system consists of three stages: Create, Log, and Explore. Through a mobile app, users can build memory cells –basic units of memory containers– and attach their memories into spaces. The mobile app allows users to create, edit and merge different memory cells. Users may upload any type of media – from texts and images to three-dimensional models to their virtual memory palace. Once they are uploaded, the system is ready to be explored.

MA App

Home Screen

Creating & Composing Memory Cells

3D Capture of Real Space

SPACE AS INTERFACE & ORGANIZATIONAL SCHEMAS

As the person develops their memory palace, they could begin to shape and model it after the real world by capturing 3D spatial scans. This build up over time is accelerated and enhanced through the sharing feature. Eventually the user could have a virtual duplicate of their home, school, town, etc., in which they compose and montage events and moments they want to remember of that particular place.

Empty Space Cell

Portals Connecting Spaces

Through the use of Google’s Project Tango and Unity, these models are turned into a VR space in which the user is able to both physically, and virtually walk round in. The user, by moving, kneeling, walking, etc., is able to navigate their virtual architecture holding their composed memories. In this project we explore the possibilities of using the immediate interaction between memory and space, and allow users to creatively interact with their own memories.

Through the use of our MA App, the person is able embed images, videos, document files, sound recordings, and even real 3D spaces into the digital models they create. Then, they are able to organize these spaces as a way to alter the memory’s context. In other words, they can place two very different memories together as way to gain a different perspective on the scenarios; they can increase or decrease the size of certain memories; they can combine two separate events into one memory, or break one into its separate aspects.