SCOBES-AR

A study involving over 100 test subjects delivered meaningful results. The ever-present user-centered design process ensured that all activities and developments were viewed from the perspective of the target groups and tailored to their needs. A target group-oriented user experience was designed and evaluated using usability tests.

Healthy aging – utopia or vision of the future?

Thanks to improved living conditions and healthcare, people in Austria are living longer. However, data from the OECD shows that aging is associated with a lower quality of life compared to other EU member states. One influencing factor is individual functionality on a physical and cognitive level. Physical functionality is obvious, but cognitive impairment is preceded by a long latency phase. Early detection of dementia through screening and early intervention can contribute to a more favorable course of treatment. However, current strategies are primarily aimed at stabilizing neurocognitive deficits. Measures for early detection and prevention are therefore becoming the focus of attention.

Early detection for a better quality of life

There are many risk factors for the development of functional or neurocognitive impairments. For example, cardiovascular, behavioral, lifestyle, and psychosocial factors play a role in the onset of the disease. Deficits that may indicate the development of functional or neurocognitive impairments are often already apparent in everyday activities. The decline in sensory perception, especially olfactory function, personal eating habits, changes in gait, reduced cardiorespiratory fitness and responsiveness, and changes in language ability are considered predictive factors in the current literature. A summary and subsequent assessment of these factors in asymptomatic individuals could offer added value in early detection.

The main objective of the five-year project was therefore to develop a multimodal screening tool for the early detection of functional limitations in older adults (60+) for outpatient care. The potential of augmented reality and virtual reality technologies is being exploited for this purpose. These technologies represent an innovative basis for the development of assessment procedures. Real places and objects are augmented with virtual objects and tasks. Immersion in virtual worlds makes everyday activities available for assessments and training regardless of time and place. Cognitive functionalities can be recorded and evaluated with the assessment based on this. In addition, measurements of various parameters, automatic action recording, and semi-automated evaluations provide detailed results. The tool can thus provide a comprehensive functional picture of the test subjects.

From the idea to final product

The screening tool consists of a tablet PC, a smartphone with an AR app and head mount, and a VR headset, and is designed as a mobile set. Healthcare professionals and the people being screened were actively involved in all stages of development. The user-centered design process focused on future users and ensured good usability.

Technical expertise & public health strategies – a promising combination

SCOBES-AR is an excellent example of interdisciplinary collaboration between medical-technical professions and nursing, made possible by user-centered design in collaboration with experts in user experience and software development. The Institute of Dietetics at FH JOANNEUM is leading the project and conducting research in collaboration with the Institutes of Speech Therapy, Occupational Therapy, Physiotherapy, Health and Nursing, Health and Tourism Management, Business Informatics, and the Department of Research Organization and Services. The project is supported by the Austrian Institute of Technology and experts from JOANNEUM RESEARCH. The project was funded by the Austrian Research Promotion Agency (FFG) as part of the COIN program.

Test procedures within SCOBES-AR

SCOBES-AR combines several test procedures into one screening instrument. For each test procedure, the appropriate modality – augmented reality, virtual reality, or conventional smartphone applications – was used. The monitoring tool was implemented as a smartphone application on a tablet computer. The individual test procedures are described in detail below.

Erlangen Test of Activities of Daily Living in Persons with Mild Dementia or Mild Cognitive Impairment (ETAM)

Conventional test procedure

The so-called ETAM (Erlangen Test of Activities of Daily Living in Persons with Mild Dementia or Mild Cognitive Impairment) measures the correct performance of complex everyday practical skills that are of particular relevance to older people living alone.

The ETAM test procedures selected for the screening tool come from the areas of mobility and financial management.

In the course of the original, analogue test, the test person has to answer questions about six traffic situations depicted in photos and, in a fictitious shopping situation, select food from photos and count out the required coins.

Adapted test procedure

In the digitalised version of the test procedure, the test person finds themselves in a photorealistic traffic situation and has to decide and justify when, where and why they are going to the pharmacy. This was realised as a virtual reality (VR) application. This screening requires the Oculus Quest 2 VR glasses.

Also in a purely virtual, three-dimensional world, the test person has to carry out a number of activities in a supermarket that are important when shopping, such as selecting certain foods from a shopping list, recognising reasonable priced food products and counting money.

The “ETAM” test procedure is carried out using the Oculus Quest 2 VR headset and can be followed in real time using a tablet computer. Abnormalities can also be annotated. The answers to control questions are also documented on the tablet computer.

Dual Task Assessment

Conventional test procedure

Dual-task describes the ability to perform two tasks simultaneously. This ability usually declines with age. An above-average loss of this ability can indicate cognitive changes such as dementia.

The dual-task assessment carried out in the SCOBES-AR project measures the influence of a cognitive task on the gait pattern, whereby the cognitive task consists of counting backwards in three steps. The gait analysis is carried out once without and once with the cognitive task in order to be able to recognise differences. The data collected includes walking speed, stride length and the duration of the gait phases. Impaired people show a greater change in gait parameters if calculations are performed at the same time as walking.

The results are measured with the naked eye and a stopwatch as well as with sensors that are attached to the back of the foot using a strap.

Adapted test procedure

The digitalised version of the Dual Task Assessment takes place on the same walking route as the conventional procedure. For the adapted version of this test procedure, a smartphone with the specially developed app is mounted in a head mount. Augmented reality (AR) technologies are used to automatically measure the walking distance and provide localised instructions. The test supervisor can start and monitor the test procedure via a tablet. Documentation is also possible via the tablet. The test subject performs the test procedure with the head support and all parameters are automatically recorded via the smartphone.

The data from the dual-task assessment is recorded and stored automatically. Additional information can be added during the test procedure using a tablet computer.

Trail Making Test

Conventional test procedure

The Trail Making Test consists of two parts. In the first part, the numbers from 1 to 25, which appear to be randomly distributed, must be connected as quickly as possible in ascending order. In the second part, numbers and letters must be connected alternately in ascending or alphabetical order. This test is usually carried out with pen and paper or on a computer.

In order to successfully complete this test, several cognitive processes are required, such as visual attention, processing speed, cognitive flexibility and general cognitive ability. The test procedure can be used to investigate various neurological disorders, including dementia.

Adapted test procedure

In the digitalised Trail Making Test, balls labelled with numbers and letters have to be selected in the correct order. The application runs as an augmented reality (AR) application on a smartphone. The smartphone is pushed into a head mount so that the test subjects can perceive the balls three-dimensionally in real space. To select balls, the test subjects move a crosshair by turning and tilting their head over the ball to be selected. If the two overlap, a button must be pressed with the hand.

The trail making test in the augmented environment requires a smartphone with the corresponding app, a well-fitting head mount and a flic button.

Reaction test

Conventional test procedure

The Talent Diagnostic System Reaction Test determines the eye-hand and eye-leg reaction as well as the complex reaction ability. The time required to complete the test procedure is recorded.

The test system is set up on a table. On the table are contact plates for the right and left hand and under the table is the floor contact plate for the right and left foot. There is also a laptop on the table, which displays visual signals on the screen. More than one signal can be displayed at the same time. The test person stands in front of the table and must now touch the corresponding contact plates with their hand and/or foot. A new optical movement signal is then displayed on the computer screen. The contact of the hands and feet with the floor contact plate must be released after each signal.

Adapted test procedure

In the adapted version, the reaction test takes place on the smartphone, on whose screen white and black areas light up. Depending on the question and position, one or more flic buttons must be pressed. This setting has the advantage of being much more manageable and low-threshold.

The adapted reaction test requires a smartphone in a holder and the corresponding app as well as four Flic buttons with caps.

c, monitoring, and annotation

A customised mobile application for tablet computers was developed to control, monitor, and annotate the individual test methods. The test subjects’ activities are visualised in real time. Test supervisors are thus able to assess progress from the perspective of the test subjects. This provides valuable insights, especially for VR and AR test procedures. The virtual scenes can be viewed through the eyes of the test subjects. In addition, their actions with virtual objects can be retraced.