LumiNote

Spatial Interface for organization and recall

Read-Time: 7 Minutes

Role

UX Researcher

Timeline

Aug 2023 - May 2024

Team

Individual

Context

An Augmented Reality (AR) prototype that allows young adults to place custom reminders tied to objects and locations in their physical surroundings. It all begins by asking a simple question: 

What are the benefits of digital spatial interfaces in task management for young adults?

SUMMARY

Task Management for young adults is fragmented.

Each tool shines in certain use cases, but young adults benefit from using them together for optimal task management.

Digital Notes

Digital Calendars

Sticky Notes

Spatial Notes

Can only offer contextual reminders
Can only offer time sensitive reminders
Can only offer contextual reminders
Offers both contextual and time sensitive reminders
Allows priority indication through color, freeform writing
Only indicates most urgent not what's most important
Allows priority indication through color, freeform writing, and spatial arrangement
Allows priority indication through color, freeform writing, and spatial arrangement
Can access your notes from anywhere
Can access your calendar from anywhere
Need to take your mess with you
Can access your notes and calendar from anynwhere
Need to search with the exact keyword
Information is sorted through time
Physical context helps in finding information
Physical context helps in finding information
User can dynamically arrange information when writing or typing a digital note
User is limited in Information organization as it is only sorted through time
Physical arrangement allows three degree of freedom to organize information
Spatial Notes along with calendar reminders allow for both time and spatial organization

Core Features

Eliminate redundant reminders

Seamless integration between Calendars and Spatial Notes

Find any information easily via spatial sound

Spatial Sound alerts make it easy to find the exact information

Know which notes are important at a glance

Spatial proximity and color choice can be used as priority indicators

Access your notes from anywhere

Bring your notes with you wherever you go

SUMMARY

Task Management for young adults is fragmented.

Each tool shines in certain use cases, but young adults benefit from using them together for optimal task management.

Benefits Comparison

Digital Notes

Calendars

Sticky Notes

Spatial Notes

Can only offer contextual reminders
Can only offer time sensitive reminders
Can only offer contextual reminders
Offers both contextual and time sensitive reminders
Allows priority indication through color, freeform writing
Only indicates most urgent not what's most important
Allows priority indication through color, freeform writing, and spatial arrangement
Allows priority indication through color, freeform writing, and spatial arrangement
Can access your notes from anywhere
Can access your calendar from anywhere
Need to take your mess with you
Can access your notes and calendar from anynwhere
Only leverages the remembrance system -

Need to search with the exact keyword
Uses both reminder and remembrance system -

Calendars are sorted through time
Uses both reminder and remembrance system -

Physical context helps in both reminding and remembrance
Uses both reminder and remembrance system -

Physical context along with spatial audio helps in both reminding and remembrance
User can dynamically arrange information when writing or typing a digital note
User is limited in Information organization as it is only sorted through time
Physical arrangement allows three degree of freedom to organize information
Spatial Notes along with calendar reminders allow for both time and spatial organization

Core Features

The system emphasizes four core features that embody the fundamental pillars of effective task management
Learn more
  • Seamless integration between Calendars and Spatial Notes eliminates redundant reminders

  • Spatial Sound alerts make it easy to find the exact information

  • Know which notes are important at a glance - spatial context and color choice

  • Access your notes from anywhere

Research Question Evolution

How might we employ Augmented Reality to associate location-based reminders for specific daily tasks to optimize recall and workflow?

How can anchoring digital information onto locations in the physical environment create more intuitive organizational systems that align with innate cognitive structures?

How can we utilize Augmented Reality in synergy with established organizational tools to harness the context of physical locations and the accessibility of digital solutions, thus improving recall, streamlining workflow, and bolstering accountability in daily task management?

What benefits do digital spatial interfaces offer in physical environments that enhance task management for young adults?

THE HUMAN ASPECT

How do young adults organize information in physical space? 

The 1:1 exploratory semi-structured interviews provided a more profound understanding of the various methods people utilize to process information, particularly in critical areas such as ideation, information organization, and physical context.

In 2 rounds of semi-structured interviews, I learnt

Ideation

How do young adults prefer to ideate?

Information Organization

What are some of the preferred methods of information organization for task management?

Physical context

What role does physical context plays when organizing information?
indicated that effective task management systems need a blend of organizational methods with sensory cues.
interview participants use more than a single tool to help them effectively manage their daily tasks but rely on one significantly more than the rest.
interview participants prefer hierarchical organization while the rest prefer a more fluid organizational system of remembrance

Time Structured Tina

Age: 24
Occupation: HCI Masters Student
Preferred Organization Tool: Calendar (Organize using Time)

Affinity for calendars

Relies on a digital calendar to organize research deadlines, group meetings, and user testing sessions.

Prefers personalization

Uses calendar integrations with email and project management tools for seamless scheduling.

Visual timelines are beneficial

Prefers visualizing his schedule to allocate time for coursework and independent study.

Reminder creation redundancy

Tina often finds herself jotting down reminders and important points on physical notes during meetings or lectures due to their immediate accessibility. However, this practice leads to redundancy, as she later has to spend additional time transferring these reminders into her digital calendar, increasing the chances of missing critical details or duplicating entries.

Visual Valerie

Age: 26
Occupation: PhD Student in Industrial Design
Preferred Organization Tool: Sticky Notes (Organize using space and proximity)

Sticky notes fiend

Uses a combination of physical sticky notes and digital tools for brainstorming and organizing research ideas.

Visual thinker through and through

Creates visual mind maps to connect concepts in her research.

Reminders belong in her space

Values the tangibility of physical notes for initial idea generation.

Out of sight, out of mind

As she cannot carry her array of sticky notes everywhere, especially when she needs to work in different locations, she often misses out on accessing her reminders when away from her primary workspace.

List driven Leo

Age: 22
Occupation: Computer Science Student
Preferred Organization Tool: Lists or Notes (Organize using hierarchy)

Finding Relevant Partnerships

Uses digital note apps for creating detailed task lists and tracking coding assignments.

Negotiating Fair Compensation

Prefers breaking down complex programming tasks into step-by-step lists.

Managing Multiple Collaborations

Relies on list updates to gauge progress and stay organized

Difficulty Measuring Marketing Impact

He often forgets or overlooks important reminders since they are not prominently visible or immediately apparent amidst his digital lists, leading to missed deadlines or overlooked tasks.
1Gedeon, C. (2020, October 7). Moving beyond documents, slides, and spreadsheets.
2Bellotti, V., & Smith, I. (2000). Informing the design of an information management system with iterative fieldwork.

3Robin, J., Wynn, J., & Moscovitch, M. (2016). The spatial scaffold: The effects of spatial context on memory for events. Journal of Experimental Psychology. Learning, Memory, and Cognition, 42(2), 308–315.

LITERATURE REVIEW

Do you want to find the information yourself or the information to find you?

After parsing through 30+ written resources, I realized that...

  • People already link tasks to environments naturally with notes, piles, props in purposeful spots.1
  • The problem with task management is not failure to prioritize well. The real challenge is rather making sure that the important tasks get done, even if the unexpected occurs.2
  • Spatial context plays a leading role in remembering events as it serves as a scaffold for encoding and retrieving episodic memories.3

THE HUMAN ASPECT

How does your Work, Relax, and Play look like?

Complementing the surveys, the exploratory interviews provided a more profound understanding of the various methods people utilize to process information, particularly in critical areas such as ideation, information organization, and memory retrieval.

Time Structured Tina

Age: 24
Occupation: HCI Masters Student
Preferred Organization Tool: Calendar (Organize using Time)

Affinity for calendars

Relies on a digital calendar to organize research deadlines, group meetings, and user testing sessions.

Prefers personalization

Uses calendar integrations with email and project management tools for seamless scheduling.

Visual timelines are beneficial

Prefers visualizing his schedule to allocate time for coursework and independent study.

Reminder creation redundancy

Tina often finds herself jotting down reminders and important points on physical notes during meetings or lectures due to their immediate accessibility. However, this practice leads to redundancy, as she later has to spend additional time transferring these reminders into her digital calendar, increasing the chances of missing critical details or duplicating entries.

Visual Valerie

Age: 26
Occupation: PhD Student in Industrial Design
Preferred Organization Tool: Sticky Notes (Organize using space and proximity)

Sticky notes fiend

Uses a combination of physical sticky notes and digital tools for brainstorming and organizing research ideas.

Visual thinker through and through

Creates visual mind maps to connect concepts in her research.

Reminders belong in her space

Values the tangibility of physical notes for initial idea generation.

Out of sight, out of mind

As she cannot carry her array of sticky notes everywhere, especially when she needs to work in different locations, she often misses out on accessing her reminders when away from her primary workspace.

List driven Leo

Age: 22
Occupation: Computer Science Student
Preferred Organization Tool: Lists or Notes (Organize using hierarchy)

Finding Relevant Partnerships

Uses digital note apps for creating detailed task lists and tracking coding assignments.

Negotiating Fair Compensation

Prefers breaking down complex programming tasks into step-by-step lists.

Managing Multiple Collaborations

Relies on list updates to gauge progress and stay organized

Difficulty Measuring Marketing Impact

He often forgets or overlooks important reminders since they are not prominently visible or immediately apparent amidst his digital lists, leading to missed deadlines or overlooked tasks.
If I'm placing it somewhere that I know I'm not going to look much then I would forget about it.
Visual Valerie
PhD Student Industrial Design
Especially when I look at an empty block on my calendar I go Oh! Thank God I could take a break.
Time Structured Tina
Masters in Human Computer Interaction
Rather than having to go to different places and suddenly be thrown a nugget of information, I prefer a centralized repository of things
List driven Leo
Bachelors in Computer Science

DESIGN IMPLICATIONS AND REQUIREMENTS

Unveiling the Insights: Discoveries and Findings

Redundancy in Reminder Creation


Requires
Adaptive Organizational Structure
Physical tools are preferred because of convenience but digital tools are more manageable over time.
Design MUST help young adults minimize redundant reminder creation

Centralization in Note-Taking


Allows
Ubiquitous Access
The preference for a centralized repository for notes and reminders emerged as a strong theme among users.
Design MUST make it easy for young adults to recall when the reminder is due and access it easily

Challenges in Note Retention and Recall


Is addressed if
Information is easy to find
Design MUST streamline the physical and digital reminders and notes

Proximity and Urgency in Physical Spaces

Uses Physical Space as a Priority Indicator
Design SHOULD use proximity as a metric for reminder notification and/or sorting

Mitigating information overload


Is possible with a minimal
Notification System
Design SHOULD allow young adults to customize how many reminders are being shown to them

DEVELOPMENT TOOLKITS

Translating insights into action

Central to this development was the integration of the Microsoft Mixed Reality Toolkit 2 (MRTK2) and Meta's Presence Platform, which together provided a robust framework for building the augmented reality interface.

Core Technologies

PROTOTYPING

Building and testing a spatial task management system

Through these 1:1 semi-structured interview rounds conducted with the headset, the users preferred a symbiotic spatial note and calendar interface due to familiarity and ease of use
13 Participants
with diverse technical expertise
5 rounds
of interviews conducted with Meta Quest 3
3 designs
tested for their ease of use


Spatial Notes with sound reminders

Spatial Notes mimic sticky notes for familiarity

Spatial Calendar

Combined spatial note and calendar for better flexibility

Physical Space as a Reminder

Eliminated - too obtrusive for daily use
AR Note Mockup #7
During initial testing a user revealed
AR Note Mockup #24
Another user mentioned
"Note Colors should be interactable buttons."
AR Note Mockup #54
During initial testing a user revealed

"Dynamically change hitbox sizes to respond to user intention."
Design Idea #3

Spatial Calendar

Spatial Calendar Mockup #65
In the next rounds of testing, users said:

"Combine spatial note and calendar for better flexibility"
Spatial Calendar Mockup #70
In the next rounds of testing, users said:

"This adds too much clutter in my space. which adds to my anxiety"
Spatial Calendar Mockup #70
In the next rounds of testing, users said:

"Sound, Color, and Animations were the most effective attributes at grabbing attention"

Design Requirements

Design Ideas

Augmented Reality Note
Physical Space as a Reminder
Spatial Sound Alarms
Spatial Calendar
Minimizes redundant reminder creation
Offers Centralization in Digital Note-Taking
Tackles the Challenges in Digital Note Retention and Recall
Uses proximity as a metric to indicate urgency
Mitigates information overload

ATTRIBUTE EVALUATION

What if the note changes with distance?

Throughout this study, the distance between participant and spatial note was standardized, allowing us to isolate and understand the impact of each attribute variation on the control group.
6 Participants
with diverse technical expertise
5
Rounds
  • Sound Loudness & Pitch
  • Color Saturation & Hue
  • Animation Speed
3 Attributes
Sound, Color, and Animation

FINAL PRODUCT

Contextual notes help in both remembrance and reminding

Team

Self

Timeline

August 2023 - May 2024

Roles

UX Prototyper
3D Designer
By linking tasks to specific locations and objects, it provides a more intuitive and less intrusive way to manage daily responsibilities. This approach not only aids in better remembrance but also sets the stage for more seamless integration of phygital interfaces into our everyday lives.

Curious for more?

FLAT
UX DESIGN
FIGMA
4 MONTHS

Connecting small businesses with local influencers

A streamlined web platform that connects small businesses with local influencers, simplifying marketing partnerships and campaign management.

SPATIAL UX DESIGN
SWIFT UI
3 MONTHS

MobileAR promoting purchase of sustainably packaged grocery items

An Augmented reality mobile application that highlights sustainably packaged groceries, encouraging eco-friendly purchases through engaging, informative interactions.

UX Timeline

Vague Problem Statement

Goals and Motivations for Redesign

App Analysis

Sketch

App Redesign

Final Prototype

John Deere: Farm Management App

A class project with industry partners John Deere to improve farm management for small and novice farm owners.

Macy's Holiday Filter: Tiptoe

Macy’s interactive AR experience showcasing their holiday mascot!

Overview

Summary

Final Product

Research

Human Aspect

Design Ideation

System Design

Conclusion