Annotated Bibliography
Cosley, Dan, et al. "Experiences with designing tools for everyday reminiscing." Human–Computer Interaction 27.1-2 (2012): 175-198. Accessed 23 September 2019
This article provides a holistic view of designing experiences for memory products and introduces some key design principles from a human-computer interaction perspective. The author mentions that there is a tendency to designing technologies for experiences and human-centered content. To achieve that, research has been conducted in two areas, one is to understand people’s reminiscing behavior, another one is to explore how technology could support that. My work also follows this research framework: in the user research process, I asked people questions in two topics: their reminiscing behavior in everyday life, another one is about their use of technology and memory products. (Facebook memory, Instagram story, etc.) in order to understand how technology has affected people’s reminiscence practices.
Before starting design, authors first conduct research to prove the assumption that social media content might support reminiscing, which is also the foundation of my project: utilizing content on social media to help people retrieve the memory. In the article, through the process of designing Pensive, a system that provides snippets of content one has previously posted on social media and remind people to revisit those content via email reminder, the authors aim to establish design principles and explore product designs that can support people’s reminiscence practices. At the beginning of design, they set up four goals to achieve the maximum effect: integrate the product to everyday life, leveraging existing content, enable lightweight content capture and respect privacy. I referred to these four goals when I was designing my own application, as these are the important rules to enable the best user experience and effectiveness of technology-supported reminiscence. My user research and interviews also uncovered insights that are similar to these rules.
In the end, the author mentioned that the development of the Pensive project could contribute to the design and development of reminiscing product in the future which is also the goal I have for my project: a pilot project, a proof-of-concept purpose design principles of how emerging technology like augmented reality could be utilized in people’s everyday life.
Cutler, Ben, et al. “Holoportation.” Microsoft Research, 2019, www.microsoft.com/enus/research/project/holoportation-3/ . Accessed 3 October 2019.
This is a Mixed reality prototype created by Microsoft. In the video, it visualizes how MR helps people to communicate and interact with the digital avatar of people who stay at another place, which is called virtual 3d teleportation. This technology allows people in different place to present in the same environment and purposes a new way for people to interact with each other. As demonstrated in the video, the researcher in MR office can interact with her daughter who is in another place. The rendering can be a bit glitch due to bandwidth issue, but it is quite realistic as you can see the girl walking around in the space.
Later on, both the researcher and his daughter is captured in a video and the researcher can re-enter this scene to watch how he and his daughter interact, which was described by the researcher himself as “it’s almost like walking into a living memory that I can see for another pair of eyes from any perspectives”. This project shows the potential of applying XR in experience rebuilding, and the notion of time travel, as well as demonstrate the ultimate goal of my project: creating a 3d experience that allows people to re-enter their past memory with a MR headset. Although due to the technical limitation we have nowadays, this project is still at a very early exploration phase, it provides me great inspiration in terms of the concept of time travel utilizing MR technology, and also shows the promising potential of my work as company like Microsoft is also exploring the similar idea and investigating on the technological challenge.
The research also explains technological challenges and how Microsoft is solving those problems in order to improve the experience in MR, such as reducing bandwidth using innovative compression algorithms as well as challenges in rendering, lighting and vibration. These are the technical barriers XR tech is facing before achieve mainstream adoption point, and also the barriers for my project to become a mature product.
Exposito, Marc, and Oscar Rosello. “Project Overview ' NeverMind: Using AR for Memorization.” MIT Media Lab, 2016, www.media.mit.edu/projects/nevermind/overview/.
Nevermind is an AR project aiming to explore how to help people memorize more effectively. It combines location-based AR technology and memory palace, a memorize method first developed in ancient Greece and used for public speaking preparation. It includes 3 steps, first convert the object or the content you want to memorize to a mental image, then place that image in an imaginary scene. To recall the object, you will only need to imagine you are in that scene, and the object will automatically appear in your mind. The project nevermind takes the same approach and builds a tool that allows people to drop the image of the object at a specific location, and every time when people pass by the place they are able to see the object and then enhance the memorization of the object. According to the testing result, it suggests that this type of memory retrieval makes the accuracy even tripled to conventional paper-based memorizing techniques, and therefore augmented reality technology can support learning.
I was originally inspired by this project and fascinated by the idea of how contextual information such as geo-location could help people improve memorize skills. This project shows the power and the effectiveness of utilizing location as a memory trigger which is the fundamental theory of my work. This project consists of two-part, an AR environment with AR glasses, and a mobile application that allows users to customize the setting and drop content at each location. These ideas show me how to connect the AR world and a more user-centered application to deliver end-to-end experience, and inspired me to create my work using a similar approach. I believe that the ultimate version of my project should also be transformed into a head-mounted device to allow people to move freely and be completely immersed in the experience.
Goode, Lauren. “Are Holograms the Future of How We Capture Memories?” The Verge, The Verge, 7 Nov. 2017, www.theverge.com/2017/11/7/16613234/next-level-ar-vr-memories-holograms-8i-actress-shoah-foundation.
8i is a volumetric video production studio building its repository for hologram projects. In 2017, they taped a volumetric video of a mom and her child and later on create a hologram project that allows people to watch, walk around and even put that digital content in their hands with a VR headset. The project is at a very early stage, and the company is experimenting with the emotional connection between real people and the digital person captured in the volumetric video. 8i also started to create volumetric content of survivors from the holocaust. Their presence is recorded in the three-dimensional space and act as a new way of testimony. It allows people to interact with the survivor and make them feel emotionally connected. This project makes more impact on the way people experience memory and historical events. The whole production process is very fascinating and attracts a lot of interest around the world. On the other hand, it also poses a question on the way people interacting with each other: the ethics question around the content itself, trauma, negative experiences… There are also security issues that need to be discussed: the governing of these digital content, and the misusage for inappropriate activities.
This project shows me the great potential and underlying impact of my project once volumetric capturing and hologram become commercially available and accessible. Although my project focuses more on the individual experience, the project created by 8i illustrates the broader use case of my work. It also provides the in-depth walk-through of how the content was created in the first place and gives me a deeper understanding of the technological challenges, such as video capture, rendering, and potential disruption in storytelling, as it is breaking the limitation of 2d screen, but moving toward a fully immersive environment.
Hoven, Elise, and Berry Eggen. "Informing augmented memory system design through autobiographical memory theory." Personal and Ubiquitous Computing 12.6 (2008): 433-443. Accessed 23 September 2019
This article provides an in-depth explanation of autobiographical memory theory and gives an overview of design augmented memory systems. The author first argues the six functions of memory and why it matters, which provides the context of my work: why we need an application to help people obtain their memory. The three major functions of memory are the construction of self-concept, maintain social relationships, and predict the future.
The author argues that the construction of self-concept allows people to understand and remember who they think they are by letting them recall past experiences and connect those experienced with the current situation, which ultimately helps them develop self-identity. Maintain social relationships means sharing experiences with other people: when we meet friends or new people, the most of time we tend to share our experiences or stories to prove our argument or build empathy in order to enhance the social bond. My work helps to facilitate the memory retrieval process by providing context-related trigger which is the location where those experiences happened. Meanwhile, the article also mentions that memory can help people predict the future as we constantly reflect on past experiences and try to recognize similar patterns in current and future scenarios. All the arguments here helps me to identify the significance of my work and how my work could bring benefits to other people.
In the article, the authors provide a lot of helpful design recommendations and analysis of memory products. The theory mentioned in this article also illustrates the fundamental mechanism of how memory is retrieved and gave me inspiration on what cues I should include in my design process, such as geolocation. The article also introduces languages and vocabularies in augmented memory design which helps me write my own thesis and research paper.
Isaacs, Ellen etal. "Technology-mediated memory: Is technology altering our memories and interfering with well-being?." ACM Transactions on Computer-Human Interaction (TOCHI) 23.4 (2016): 23. Accessed 25 September 2019
The authors argue that nowadays technology provides a rich record of our memory and makes memory retrieval easier and more accessible. My work takes advantage of the rich technology-recorded memory material on social media, photo album, voice memo and extract the location data attached to put those experiences back to the place where happened and create a memory footprint map. My application acts as an agent for memory sorting and reflection. The authors also purpose some design guidelines for future research and projects. My project follows a lot of design principles from this article, and I get a significant amount of insights as to the theoretical background.
This article also compares the differences between organic human memory and technology-mediated memory and evaluates whether the latter can have a negative impact on well-being. Authors believe that organic memory can be seen as the raw material but when people recall the memory, they unconsciously develop biases for well-being. In contract, technology-mediated memory shows people what they actually did which might bring negative feelings because we all have memories or experiences that we might not want to revisit. therefore, the author acknowledges that TMM might pose a threat to the well-being and use adaptive theory and autobiographical theory to prove that it is wrong. To explore this, the authors created a prototype called Echo measure the influence of TMM, and find that when people revisit and reflect on past experiences, they tend to embrace a more positive mindset, a rosy retrospection, fading affect bias, which all suggest that TMM does not pose threat on mental health, but help to maintain well-being in the long run. This paper helps to answer some of my concerns when I first started my project: how people feel when they are reminded of negative experiences? how would they react? and how they might change their well-being status.
Konrad, Artie, et al. "Technology and reflection: Mood and memory mechanisms for well-being." Psychology of Well-being 6.1 (2016): 5. Accessed 1 October 2019
In this article, the author argues that mood is an important aspect of memory, and it can trig events that are emotionally congruent and incongruent to our current feeling. The idea of how emotion could affect memory retrieval is intriguing and it inspired me to explore reminiscing from the perspective of emotion and helps me to broaden my research framework.
The author also mentions that people selectively recall and reflect on memory to regulate their current mood, which is proved in my research where some people in the interviews admit that sometimes they intentionally recall positive memory for well-being and in hope of changing their current negative feeling. On top of that, I was inspired by the emotion theory and created a filter based off the idea of eight major emotions purposed by Plutchik (joy, acceptance, surprise, anticipation, fear, ease, and anger) which would allow people to selectively revisit their experience and even curate the memory content.
In the article, the authors identify and summarize some common patterns in memory retrieval and self-reflection: understanding, redemption, and distancing. These patterns also play a significant role when I am making design decisions in my project. Understanding means through emotionally writing, people can better understand the past experience and then improve well-being, thus I designed a writing feature in my work to encourage this type of emotionally writing of past experiences. Redemption means the contradictory feeling when people revisit negative experiences that happened in the past, and often people find themselves have a more positive feeling of it and appreciate how they overcome those difficulties, which is also the purpose of my project. Last but not least, distancing which means the shift from a first-person perspective to a third-person perspective when people revisit their past experiences. In my work, the idea of discovering past memory footprint and interacting with the floating orb transform people’s perspective to an adventure-like experience: although people are revisiting their memory, the mechanism makes it more like a new adventure.
Konrad, Artie. “Facebook Memories: the Research behind the Products That Connect You with Your Past.” Facebook Research, 21 Sept. 2017, research.fb.com/blog/2017/09/facebook-memories-the-research-behind-the-products-that-connect-you-with-your-past/.
Since 2016, Facebook starts to decline in organic content sharing: people no longer share content on Facebook. Facebook has been struggling to reverse a 21% feline in original sharing or personal updates form its 1.6n billion monthly active users. To improve this situation, Mark Zuckerberg develops the “time well-spent” strategy in order to focus on the quality engagement on Facebook even if the time on site suffers as a result. Facebook decides to change the focus to social interaction a meaningful community building. Their algorithm will prioritize posts that promote natural interaction between people over public content such as businesses or publishers.
In 2014, Facebook started to help users automatically generate lookback video and say thanks videos for the users, and allow people to re-experience their posts as a way of facilitating the content creation and encourage interaction between users. These videos are automatically generated using pictures, posts from Facebook, and are combined with creative elements, narrative, music, and some simple design to enhance the nostalgic impact. In 2018, these functions become a standalone feature under more tab and reflect on the moment people have shared before. An expanded version of “on this day” and friendship anniversary also published to help users engage with their Facebook friends and encourage community interaction.
Facebook memory feature shows me the foundation of utilizing social media content to generate new content, and allow people revisit past experiences, but according to my user research, some people think this type of content, especially ones about their friendship with people who they might not befriend with anymore, can be artificial and not authentic, which is not something people want to revisit. These kinds of comments let me consider the importance of content sorting and generating methods. Meanwhile, when people are reminded of negative experiences, they need to have control and be able to remove certain content. All of these findings help me to formulate my design decision of features such as remove content, and my idea of utilizing natural language processing and information analysis to better understand the original content and deliver more relevant and appropriate memories to the user.
Other works
Donalek, Julie G. “Phenomenology as a Qualitative Research Method.” Urologic Nursing, vol. 24, no. 6, Dec. 2005, doi:10.5539/ass.v10n21p298.
Donalek provides an overview of phenomenology and introduces its origins and development process. It describes the basic principle of phenomenology and its impact in understanding people’s behavior as a qualitative research method. After reading this article, I decided to use phenomenology as my research methodology and I also learned about how to use methods such as interviews to gather user insights.
Holm, Tine, and Dorthe Kirkegaard Thomsen. “Self-Event Connections in Life Stories, Self-Concept Clarity, and Dissociation: Examining Their Relations with Symptoms of Psychopathology.” Imagination, Cognition and Personality, vol. 37, no. 3, 2017, pp. 293–317., doi:10.1177/0276236617733839.
This article argues that the ability to connect life events and formulate stories is important for mental health, as these narratives help to connect the past self and the current self, and therefore clarify the self-identity and maintain self-unity. It also introduces theories and practices such as self-report, in narrative psychology which is one of the theoretical framework of my project, and inspired me to design a feature in the application that encourage people to generate a self-narrative.
Neubauer, Brian E., Catherine T. Witkop, and Lara Varpio. "How phenomenology can help us learn from the experiences of others." Perspectives on Medical Education 8.2 (2019): 90-97. Accessed 2 October 2019.
This article provides in-depth descriptions on phenomenology and research methods such as user interviews, focus group. It argues that phenomenology compared to other research methodologies such as participatory, focuses on lived experience and can help researchers understand complex phenomena by learning from individual behaviors, which aligns with research goals for my project as I also wanted to focus on quality not quantity in order to understand people’s mindsets in reminiscing.
Papagiannis, Helen. Augmented human: How technology is shaping the new reality. " O'Reilly Media, Inc.", 2017.
In this book, Papagiannis envisions how AR technology could change people’s life in all aspects, from transportation to film production. It analyze the unique aspect of AR technology, such as interaction model, haptic, relation with 3D space, and argues that XR could be seen as a mixed media format which can provide new opportunities and challenges for storytelling. I think the future scenario she illustrated in this book also aligns with what I imagined the my work will look like in its ultimate version: AR time capsules that host your experience, not limited to visual content, but can also include audio, olfactory and haptic elements.
Van Den Hoven, Elise, Corina Sas, and Steve Whittaker. "Introduction to this special issue on designing for personal memories: past, present, and future." Human–Computer Interaction 27.1-2 (2012): 1-12. Accessed 23 September 2019
This article argues that digital technology give people access to numerous memory material and people can effortlessly access those content. It also allows people to organize and curate the memory content based on their preference. The author also introduces the taxonomy of memory and provides a list of the most desirable memory content. This article helps to ground my project as it gives me insight on what content I should display in the app and shows me the bright prospect of memory application.
VR Intelligence. “ XR Industry Survey Report.” XR Industry Survey Report | VR Intelligence, 2019, www.vr-intelligence.com/xr-industry-survey-report. Accessed 14 September 2019.
This report provides an overview of the development status of XR technology in 2019, and combined both qualitative and quantitative approach to discover the current roadblock and articulate the problem XR technology is facing in order to the point of mainstream adoption. From this report, I learned that the increasing investment in XR shows that people are optimistic of its future, but the lack of content and usability issue still need to be solved. My work as a proof-of-concept project can contribute to the content of XR technology, and purpose some design principles for future XR application design.
West, David, Aaron Quigley, and Judy Kay. "MEMENTO: a digital-physical scrapbook for memory sharing." Personal and Ubiquitous Computing 11.4 (2007): 313-328. Accessed 30 September 2019
This article evaluates a memory system called MEMENTO, a multimodal digital scrapbook that allows people to collaboratively collect and share memory materials, and examine its effectiveness in reminiscence for both young and senior people. This article summarizes some common practices in reminiscence, and indicates that digital technology has positive impact on reminiscence. This article gives me an overview on how digital technology could support reminiscing and the prototype MEMENTO could be seen as basic version of my project.
Assets Credits
Image:
Holoportation by Microsoft Research
Footage credits
2046 by Wong Kar-wai
Portrait of a Lady on Fire by Céline Sciamma
Vertigo by Alfred Hitchcock
Room in Rome by Julio Medem
Paprika by Tinto Brass
Blade Runner 2049 by Denis Villeneuve
Free videos provided by creators on Pexels:
Taryn Elliott, Life On Super 8, Athena, Pixabay, cottonbro, Stef, Pressmaster, Mapo Dessin, Desmond Etsebeth, Carlos Arribas, Fauxels, Free Videos, Distill, Pixly Videos, Charlie Mounsey, Kelly Lacy, Joseph Redfield, Ricardo Esquivel