ABM Applications

The measurement and interpretation of the brain’s electrical activity to assess cognitive state, until recently, has been limited to laboratory investigations by academic researchers. Over the past eight years, Advanced Brain Monitoring and its collaborators pioneered the use of cognitive state analysis for real-world settings. These application categories include: Accelerate Learning, Improve Team Dynamics & Threat Identification, Interface Humans & Computers, and Manage Fatigue. Looking to the future, the company’s goal is to expand the scope of our collaborations to further integrate and operationalize our technologies in defense, education and healthcare in the U.S. and worldwide.

 

Team NeuroDynamics

Built on the foundation of the B-Alert® Cognitive State Metrics for individual Engagement and Workload levels, the Team NeuroDynamics platform provides valuable insight as to why some teams succeed while others do not. Team leaders, coaches, and instructors can access meta-cognitive state metrics that characterize team interactions to predict team efficiency, identify weaknesses in either the team or scenario, and track team improvements over time.

    How it works:
  1. Individuals assigned to team roles
  2. EEG-based Cognitive State Metrics sequenced for each individual
  3. EEG Metrics synchronized across individuals to derive Team Cognitive States
  4. Patterns of Team Cognitive States linked to key training events by role and team interaction
  5. Comparison of patterns across teams, events, and training scenarios detect skill acquisition and team dynamics
  6. Patterns assist the trainer in improving team and scenario performance
 
Applications in Marksmanship, Archery & Golf

The ability to accelerate proficiency in a novice is difficult when learning the acquired skill is dependent on consistently applying a well-defined set of sensory, motor and cognitive skills.

Marksmanship, golf and archery are examples in which optimal performance is reliant upon motor preparation, focus and concentration. In this study the pycho-physiological profile of marksmanship was compared to archery and golf using 13 USMC active-duty marksman coaches, three archers on the U.S. Olympic team, and four PGA-certified golf pros. Results showed that patterns of pre-shot alpha activity were similar across all three tasks, although pre-shot theta and heart-rate deceleration were not as pronounced during putting and archery. These findings suggest that marksmanship peak-performance training may be applicable to other sports which rely on motionless preparation and in which performance anxiety may be high (e.g., the “yips” in golf).

Berka, C., Behneman, A., Kintz, N., Johnson, R., and Raphael, G. (2010). Accelerated Training Using Interactive Neuro-Educational Technologies: Applications to Archery, Golf and Rifle Marksmanship. International Journal of Sport and Society, Vol. 1(4): pp. 87-104.

Combat Deadly-Force Judgment & Decision Making Performance

The decision of when to use deadly-force is one of the more challenging skills to train. Simulations must be sufficiently realistic to fully engage the trainees. Split-second life-or-death decisions require the decision maker to be optimally prepared mentally.

The psycho-physiology of deadly-force judgment and decision-making (DFJDM) was evaluated in 12 experts (military and police) and 12 novices. Over the course of a day, each participant completed eight training cycles. Each cycle included three x two-minute life-size scenarios containing real world demographics and locations of domestic disputes, vehicle interventions, etc. Deadly force was justified in 17 of 24 (70%) of the scenarios.

Across subjects and scenarios, fast-alpha EEG activity (10 – 12 Hz) was significantly suppressed in experts vs. novices. The magnitude of alpha activity over the right parietal site explained almost 80% of the variance in correct use of deadly force (i.e., hit rate) across subjects. The high alpha activity exhibited by Novices at the beginning of the study decreased as the study progressed (from cycle 1 to 8).

Experts showed significantly lower levels of heart rate and cognitive engagement during the scenarios as well as at rest. These findings suggest lower alpha power in the right parietal region is associated with DFJDM skill. The next phase of this research is to assess the benefit of training novices to control EEG and heart rate during DFJDM.

 
Improving the Pace and Efficiency of Rifle Marksmanship

Rifle marksmanship is a core skill for the Army and Marine Corps, acquired through classroom learning and field practice. The speed and efficiency by which novices transition to expert is dependent in part on the acquisition of motor skills which involve physiological control (respiration, heart rate, etc.).

For this study, the capability to accelerate marksmanship skill acquisition was assessed through the use of the Interactive Neuro-Educational Technology (I-NET). The I-NET is a suite of neuroscience-based evaluation and feedback systems that enhances the assimilation of well-defined sets of sensory, motor and cognitive skills.

To profile expert physiology, 13 off-duty qualified marksmanship trainers underwent a 45-min baseline physiological profiling session (physio-baseline) followed by 3 simulator trials of five shots. Marksmanship was studied in a kneeling position at simulated distance of 200m using a demilitarized “airsoft” replica of the M4 with an infrared laser-based training system for target projection and shot detection. Shot precision was defined as the mean distance of each shot from the center of the shot group, where lower values reflect better precision. Results showed that experts displayed a heart rate deceleration and a marked increase in EEG alpha and theta power three to four seconds preceding each trigger pull. This pattern suggested automated task execution performed with minimal conscious mental effort. The experts also exhibited significantly greater cardio- respiratory control when sitting quietly with their eyes opened or closed.

Novices (n-54) underwent the same physio-baseline session as the experts followed by and a 20-minute marksmanship instructional video. The group of 17 controls underwent up to eight marksmanship baseline trials of five shots. The second group of novices (APPT; n=37) underwent neuro-feedback training by completing four trials of five shots, five-minutes of alpha training, two trials of five shots, a second alpha training session, and the two final trials with neuro-feedback. The Accelerated Peak Performance Trainer (APPT) provided the neuro-feedback by delivering a vibrational pulse from a haptic motor taped to the neck triggered by each heart beat to assist the novice learn how to control their cardio-respiratory function. Feedback was programmed to terminate when patterns of increasing EEG match those of the expert.

The upper-left graph above shows that APPT training improved the shot precision learning trajectory by 230% and contributed to a significant improvement in the percent change in shot dispersion. Over the course of training, the levels of pre-shot fast theta and slow alpha increased significantly for the APPT group (right), while the controls decreased in pre-shot fast theta and slow alpha power. Although one training session with APPT significantly improved marksmanship skill, it was not sufficient to bring the majority of novices to the expert level. Additional studies are planned to determine how to optimize the use of the APPT.

Raphael, G., Berka, C. et al. (2009). Peak Performance Trainer (PPT®): Interactive Neuro-Educational Technology to Increase the Pace and Efficiency of Rifle Marksmanship Training. 13th Int’l Conference on Human Computer Interaction. San Diego, CA.

Raphael, G., Berka, C. et al. (2009). I-NET: Interactive Neuro-Educational Technology to Accelerate Skill Learning. 31st Annual International IEEE EMBS Conference. Minneapolis, MN.

 
Neuro Profiles that Predict Skill Competence

This investigation sought to define neuro-signatures useful in predicting marksmanship proficiency.

Profiling was performed using baseline recordings from a 20-min 3-Choice-Vigilance-Test (3C-VT), and 5-min of eyes-open and eyes-closed obtained prior to marksmanship training.

Results showed the amplitude of the “P300” component of the event related potential (ERP of the best performers in a marksmanship training session were significantly greater than the poor performers during the 3C-VT. This finding suggests that EEG-based P300 amplitude recorded during a visual discrimination may be a predictor of marksmanship aptitude. High cardio-sympathetic activity as measured by power spectral analysis of heart rate variability during eyes-open and eyes-closed was a negative predictor for marksmanship performance. A training intervention was developed for high anxiety individuals using a guided 18-min Relaxation Training video followed by a reassessment of heart rate/anxiety levels. Participants continued to marksmanship training once anxiety levels were within one standard deviation of the population mean or after two relaxation training sessions were completed. Ten of the 11 participants that completed the relaxation training exhibited decreased anxiety across training sessions.

 
Enhancement of Intelligence Analysts’ Extraction of Text-Based Content

A closed-loop analysis system was developed to use intelligence analysts’ neurophysiology to auto-extract text snippets relevant to the current analysis goal during processing textual data and associated decision making.

The combination of EEG from a wireless headset and eye gaze from an eye tracker were used to identify and track sub-conscious elements of interest to ensure all relevant information was taken into consideration, while still allowing the analyst to also manually extract items they perceive as relevant. This approach was intended to reduce the effects of analyst bias and inattention and provides a faster, more accurate extraction of evidence.

The system was validated in study of 27 healthy subjects. Participants were first shown a short background story that provided the analysis scenario and a related one-sentence proposition (analysis question), and were then asked to view a series of 30 sentences to determine their relevance to the provided proposition. Ten of the sentences were relevant to the analysis question, 10 were totally irrelevant and 10 were irrelevant but contained key words found in the study topic. The figures below marked as “significant” identify the areas of the brain where the sub-conscious elements of interest can be recognized with fast theta and slow alpha EEG frequency bands.

 

The amount of EEG theta activity, adjusted to account for individual differences in theta generation, was linearly related to the assimilation of information within the reader’s mental construct. When a reader began to read a sentence theta levels were high. As the reading of the sentence continued, if the information was consistent with the reader’s belief system, theta levels gradually dropped. If the information was inconsistent or conflicted with the reader’s mental model, theta values increased sharply. The results suggest that only EEG from the right hemisphere of the brain is needed for this text-based application. This study demonstrates how the number of sensors needed for a particular application can be reduced to improve the trade-off between increased speed and accuracy vs. user inconvenience when wide deployment is considered. Application of the automated linear theta scale has the potential to allow researchers to tailor content for desired outcomes.

Behneman, A., Kintz, N. et al. (2009). Enhancing Text-Based Analysis Using Neurophysiological Measures. 13th International Conference on Human-Computer Interaction, San Diego, CA.

 
EEG Correlates of Decision Making based on Mental Modeling

The role of an analyst includes scanning of imagery rapidly and accurately in order to identify operational threats. Indices which detect cognitive processing as the user matches their mental model of a threat/non-threat to the actual image could be used to accelerate learning and/or monitor performance.

A study was conducted to assess the feasibility of integrating real-time analysis of EEG indices of workload, task engagement and attention into the human-computer loop for training new luggage screeners (n=23). X-ray imagery of non-treats and threats (sharp objects and hand guns) with varying degree of detection difficulty were used in the study design.

The results showed a distinct cognitive processing pattern was apparent in 90% of the subjects beginning with the presentation of the image and ending with the treat/`no-threat decision. The magnitude of initial theta synchrony and the speed in which the theta activity desynchronized related to the difficulty of the task, whether the image was a threat or non-threat and the region of the brain most activated by the encoding. The desynchronization of theta activity was followed by the synchronization of alpha power which
was also influenced by and across stimulus difficulty and
response type.

Given these patterns were strongest with easy or target stimuli, weakest with more difficult, non-target stimuli, these findings suggest that the EEG theta and theta synchronization is likely correlated with subject skill acquisition and indicative of the progressive development of the individual’s cognitive mapping of their mental model. The relationship between EEG Alpha and correct vs. incorrect responses suggests a measure of efficient use of attention processes. Theta and alpha synchronization may also help to differentiate between hits, false alarms and correct rejections. The next phase of the research will be to reduce the number of scalp sites and provide a real-time assessment as to when the subject is guessing or actually believes to accurately recognize a target. Additional efforts will focus on neuro-cognitive signatures which profile those more likely to succeed in the role of a fast and accurate luggage screener.

 

Berka, C., Levendowski, D. et al. (2010) Enhancing Training Effectiveness Through Cognitive State Assessment Contract No. N10PC20026

 
Threat Identification in the Virtual Battlespace 2 (VBS2) Training Simulation

VBS2 is a complex and realistic training simulation platform that is widely used to train observation skills for military applications. In collaboration with NRL and UCF, ABM is developing an adaptive closed-loop interface with the VBS2 to allow EEG-derived cognitive and affective state metrics to drive the simulation scenarios. The goal is to accelerate learning with the simulation interactively adapted to individual’s skill level by combining measures of performance (e.g., accuracy, speed, and efficiency), cognition (engagement, confusion, workload) and stress with intervention strategies including slowing or speeding the pace of the scenario; introducing alarms or prompts to direct attention; or changing the level or complexity of the task could be applied to accelerate learning.

An initial study was conducted to characterize neural signatures of threat detection using 10 threatening and 10 non-threatening virtual scenes depicting people, vehicles and IEDs were extracted from VBS-2. Fourteen participants made threat assessment on 60 images which were presented at 2-sec intervals. Result showed detection of IEDs was significantly more difficult than people or vehicles based on lower accuracy and longer reaction times. The differentiation between threatening and non-threatening people was easiest based on reaction time and accuracy.


Significant differences in the mean amplitude of the event-related potentials (EPRs) were observed by category and region of the brain. The plots below show that identification of an IED threat resulted in the largest and most consistent late positive target detection component in the frontal and central regions.. A large amplitude late positive component in the left frontal and central hemisphere was associated with threatening people, while the detection of non-threatening people resulted in a substantially greater target detection component along the midline and in the right hemisphere. These results show that unique neural signatures of threat detection can be identified in association with VBS2 imagery. These neural signatures form the foundation for mapping a highly detailed description of the brain’s capability to identify and respond to both threatening and non-threatening imagery.

Berka, C., Pojman, N., et al. (2010) Merging Cognitive Neuroscience and Virtual Simulation in an Interactive Training Platform. 3rd Int’l Conference on Applied Human Factors and Ergonomics, Miami, FL