Immersive smart home system for elderlies
1. Introduction and motivations:
According to the United Nations Population Division, in 2050, the total number of people over the age of 65 will be greater than the number of youth for the first time in human history. This rapid increase of the aging population comes with a higher demand on personal healthcare, particularly for seniors who require additional support. Furthermore, aging is the number one factor for developing disabling conditions such as major or minor neurocognitive disorders (NCDs); dementia, Alzheimer’s, and mild cognitive impairment.
Assistive smart home systems (SSH) can support aging in place for seniors with neurocognitive disorders (SwNCDs). Internet of Things (IoT) devices are effective in creating customized and affordable SSH environment. However, this approach requires a certain level of technology literacy that many SwNCDs don’t have. Head-mounted Mixed Reality (MR) devices blend virtual interactive objects with the physical world. Therefore, there is great potential for using MR applications to provide senior users with a tailored and effortless immersive SSH solutions. Because MR is an emerging field of study, there is a clear lack of design guidelines for supportive MR applications. Furthermore, evaluating MR applications in-person with senior participants became challenging during the Covid-19 pandemic which effected studying this field negatively.
2. Objectives and methods:
We use video prototypes to conduct a qualitative online Design Critique study with 24 participants: seniors, formal and informal caregivers, assistive technology experts and MR developers. These prototypes address two user case scenarios: medication reminder and cooking assistant. A course of Thematic Analysis took place on the Design Critique qualitative data to extract recommendations, potential design guidelines and suggestions for implementing the system. Furthermore, the relationships between these recommendations and participant categories are explored thoroughly. We used this design recommendations and guidelines to reiterate our system prototype design. At a final stage, we developed a high-fidelity system prototype using Unity game engine and the Microsoft MRTK. We deployed our system on a Hololens2 device and a set of IoT devices.
3. System prototype
The current version of the application supports two user modes: senior and caregiver user which we describe in the following subsections.
3.1 Caregiver mode
Th purpose of the caregiver mode is to setup the system or customize preferences. When the application runs for the first time, caregivers will be asked to introduce the real-world IoT devices to the application. Each IoT device is represented by one virtual object (game object). To assist the user, the system displays holographic animated hand gestures illustrating how to interact with these virtual objects (MRTK Hand Coach prefab). Thereafter, caregivers are asked to grab a virtual object that represents the pill dispenser and align it with the real-world pill dispenser. The same step is required to locate the stove. This approach allows us to identify the position of both devices in space and measure the distance between them and the HoloLens2 using Unity ’Vetcotr3’ struct. Upon introducing the IoT devices to the application, caregivers can customize the system preferences by exploring various pre-set memory prompts that we designed specifically for SwNCDS homecare based on our Design Critique study
3.2 Senior User Mode
The senior user mode is a simple and minimal approach that delivers memory prompts tailored to SwNCDs. Initially, the system sends a vibration signal to the Apple Watch to attract user attention. Then, when the user looks at the watch, a holographic message is displayed above the wrist. The reason behind this design decision is to engage more than one sense which increases the chances for user responsiveness. In addition, a short break between the vibration signal and the memory prompts gives the senior time to process information, as SwNCDs often suffer from neuro-delay. Failure to complete any task after receiving the initial memory prompt will trigger the system to proceed to the next memory prompt. The sequence of events after the initial prompt is completely configured by the caregiver based on senior’s needs.
Our system design unlocks possibilities for using IoT devices to create supportive smart home systems and facilitate seniors homecare. Although the UX and UI of our system are designed based on our previous Design Critique study, the final system prototype is yet to be tested in-person with potential end-users to further investigate its usability. With the ease of Covid-19 restrictions, we hope to conduct in-person testing in the future and add more use cases. Finally, it is important to acknowledge that current MR headsets are not suitable for day-long use. However, the near future promises lighter and more user-friendly devices. In the meantime, we continue to use the HoloLens for prototyping purposes.
Disclaimer: parts of this this web article is copied from L. Alabood and F. Maurer, An IoT-based Immersive Smart Home System for Seniors with Neurocognitive Disorders. EMPATHY: 3rd International Workshop on Empowering People in Dealing with Internet of Things Ecosystems. Workshop co-located with AVI 2022, June 06, 2022, Frascati, Rome, Italy
