Assessment of a Novel Virtual Environment for Examining Human Cognitive-Motor Performance during Execution of Action Sequences

Published in HCII, 2022

Recommended Citation: Shaver A, Peri N, Mezebish R, Matthew G, Berson A, Gaskins C, Davis GP, Katz GE, Samuel I, Reinhard MJ, Costanzo ME, Reggia JA, Purtilo J, Gentili RJ. Assessment of a Novel Virtual Environment for Examining Human Cognitive-Motor Performance during Execution of Action Sequences. In: International Conference on Human Computer Interaction, HCII 2022

The examination of neural resource allocation during complex action sequence execution is critical to understanding human behavior. While physical systems are usually used for such assessment, virtual/remote systems offer other approaches with potential benefits such as remote training/evaluation. Here we describe a virtual environment (VLEARN) operated via the internet that has been developed to study the cognitive-motor mechanisms underlying the execution of goal-oriented action sequences in remote and laboratory settings. This study aimed to i) examine the feasibility of evaluating human cognitive-motor behavior when individuals operate VLEARN to complete various tasks; and ii) assess VLEARN by comparing its usability and the resulting performance , mental workload, and mental/physical fatigue during virtual and physical task execution. Results revealed that our approach allowed human cognitive-motor behavior assessment as the tasks completed physically and virtually via VLEARN had similar success rates. Also, there was a relationship between the complexity of the virtual control systems and the dependency on those to complete tasks. Namely, relative to controls with more functionalities, when VLEARN enabled simpler controls, above average usability and similar levels of cognitive-motor perfor- mance for both physical and virtual task execution were observed. Thus, a simplification of some aspects of the VLEARN control interface should enhance its usability. Our approach is promising for examining human cognitive-motor behavior and informing multiple applications (e.g., telehealth, remote training).

[Download Paper] [Code]