A Novel Method of Emergency Situation Detection for a Brain-controlled Vehicle by Combining EEG Signals with Surrounding InformationL. Bi, H. Wang, T. Teng, and C. Guan.IEEE Transactions on Neural Systems and Rehabilitation Engineering:2018,online摘要下载全文In this paper, to address the safety of braincontrolled vehicles under emergency situations, we propose a novel method of emergency situation detection by fusing driver electroencephalography (EEG) signals with surrounding information. We first build a novel EEG-based detection model of driver emergency braking intention. We then recognize emergency situations by fusing the result of the proposed EEGbased intention detection model with that of the obstacle detection model based on surrounding information. The real-time detection system of driver emergency braking intention is implemented on an embedded system, and the driver-and-hardware-in-the-loopexperiment of the proposed detection method of emergency situations is performed. Experimental results show that the proposed method can detect emergency situations with the system accuracy of 94.89%, false alarm rate of 0.05%, and response time of 540 ms. This study has important values in the future development of brain-controlled vehicles, human-centric advanced driver assistant systems, and self-driving vehicles and opens a new avenue on how cognitive neuroscience may be applied to human-machine integration.
Mathematical Modeling of EEG Signals-Based Brain-Control BehaviorY. Lu, L. Bi, J. Lian, and H. Li.IEEE Transactions on Neural Systems and Rehabilitation Engineering:2018,vol. 26, no. 8,pp. 1535-1543摘要Brain-control behaviors (BCBs) are behaviors of humans that communicate with external devices by means of the human brain rather than peripheral nerves or muscles. In this paper, to understand and simulate such behaviors, we propose a mathematical model by combining a queuing network-based encoding model with a brain-computer interface model. Experimental results under the static tests show the effectiveness of the proposed model in simulating real BCBs. Furthermore, we verify the effectiveness and applicability of the proposed model through the dynamic experimental tests in a simulated vehicle. This paper not only promotes the understanding and prediction of BCBs, but also provides some insights into assistive technology on brain-controlled systems and extends the scope of research on human behavior modeling.
EEG-Based Detection of Driver Emergency Braking Intention for Brain-controlled VehiclesT. Teng, L. Bi, and Y. Liu.IEEE Transactions on Intelligent Transportation Systems:2018,vol. 19, no. 6,pp. 1766 – 1773摘要下载全文In this paper, we propose a new approach of detecting emergency braking intention for brain-controlled vehicles by interpreting electroencephalography (EEG) signals of drivers. Regularization linear discriminant analysis with spatial-frequency features is applied to build the detection model. These spatial-frequency features are selected from the powers of frequency points across sixteen channels by using the sequential forward floating search. Experimental results from twelve subjects show that on average, the proposed method can detect emergency braking intentions 420 ms after the onset of emergency situations with the system accuracy of over 94%, showing the feasibility of developing a practical system of detecting driver emergency braking intention with the power spectra of EEG signals for brain-controlled vehicles.
Queuing Network Modeling of Driver EEG Signals-Based Steering ControlL. Bi, Y. Lu, X. Fan, J. Lian, and Y. Liu.IEEE Transactions on Neural Systems and Rehabilitation Engineering:2017,vol. 25, no. 8,pp. 1117-1124摘要下载全文Directly using brain signals rather than limbs to steer a vehicle may not only help disabled people to control an assistive vehicle, but also provide a complementary means of control for a wider driving community. In this paper, to simulate and predict driver performance in steering a vehicle with brain signals, we propose a driver brain-controlled steering model by combining an extended queuing network-based driver model with a brain-computer interface (BCI) performance model. Experimental results suggest that the proposed driver brain-controlled steering model has performance close to that of real drivers with good performance in brain-controlled driving. The brain-controlled steering model has potential values in helping develop a brain-controlled assistive vehicle. Furthermore, this study provides some insights into the simulation and prediction of the performance of using BCI systems to control other external devices (e.g., mobile robots).
Using the Support Vector Regression Approach to Model Human PerformanceLuzheng Bi, Omer Tsimhoni, Yili Liu.IEEE Transactions on Systems Man and Cybernetics Part A-Systems and Humans:May 2011,vol. 41, no. 3,pp. 410-417摘要下载全文Empirical data modeling can be used to model human performance and explore the relationships between diverse sets of variables. A major challenge of empirical data modeling is how to generalize or extrapolate the findings with a limited amount of observed data to a broader context. In this paper, we introduce an approach from machine learning, known as support vector regression (SVR), which can help address this challenge. To demonstrate the method and the value of modeling human performance with SVR, we apply SVR to a real-world human factors problem of night vision system design for passenger vehicles by modeling the probability of pedestrian detection as a function of image metrics. The results indicate that the SVR-based model of pedestrian detection shows good performance. Some suggestions on modeling human performance by using SVR are discussed.
Effects of Symmetry and Number of Compositional Elements on Chinese Users＼ Aesthetic Ratings of Interfaces: Experimental and Modeling InvestigationsLuzheng Bi, Xin-an Fan, Yili Liu.International Journal of Human-Computer Interaction:02 Feb 2011,vol. 27, no. 3,pp. 245-259摘要下载全文This article reports two experiments and the corresponding modeling research on the effects of symmetry and the number of compositional elements on Chinese users＼ aesthetic ratings of interfaces composed with abstract black-and-white geometric images and realistic-looking web pages. Symmetry and the number of compositional elements are the two independent variables, each with three levels. The dependent variable is subjective ratings of aesthetic appeal. The results from both experiments show that symmetry of compositional elements significantly affects aesthetic ratings of Chinese users, whereas the number alone does not have a significant effect on aesthetic ratings. However, subjects preferred realistic web page images with few elements when they lack symmetry. We also describe our development and evaluation of a computational model of aesthetic ratings based on symmetry and the number of compositional elements to predict aesthetic ratings, which can be used to evaluate and support aesthetic design of interfaces.
A Head-Up Display-based P300 Brain-Computer Interface for Destination SelectionLuzheng Bi, Xin-an Fan, Nini Luo, Ke Jie, Yun Li, and Yili Liu.IEEE Transactions on Intelligent Transportation Systems:Dec. 2013,vol. 14, no.4,pp. 1996-2001摘要下载全文In this paper, we propose a P300 brain-computer interface (BCI) with visual stimuli presented on a head-up display and we apply this BCI for selecting destinations of a simulated vehicle in a virtual scene. To improve the usability of the selection system, we analyze the effects of the number of electroencephalogram (EEG) rounds on system performance. Experimental results from eight participants show that the BCI-based model of destination selection can be built with EEG data from eight channels, and participants can use this BCI to select a desired destination with an accuracy value of 93.6% ± 1.6% (mean value with standard error) in about 12 s of selection time. This paper lays a foundation for developing vehicles that use a BCI to select a desired destination from a list of predefined destinations and then use an autonomous navigation system to reach the desired destination.
Using Queuing Network and Logistic Regression to Model Driving with a Visual Distraction TaskLuzheng Bi, Guodong Gan, and Yili Liu.International Journal of Human-Computer Interaction:2014,vol. 30, no. 1,pp. 32-39摘要下载全文Computational dual-task models of driving with a secondary task can help compute, simulate, and predict driving behavior in dual task situations. These models can thus help improve the process of developing in-vehicle devices by reducing or eliminating the need for conducting driver experiments in the early stage of the development. Further, these models can help improve traffic flow simulation. This article develops a dual-task model of driving with a visual distraction task using the Queuing Network model of driver lateral control and a logistic regression model. The comparison between the model simulation data and the human data from drivers in a driving simulator shows that this computational model can perform driving with a secondary visual task well and its performance is consistent with the driver data.
Detecting Driver Normal and Emergency Lane-changing Intentions With Queuing Network-based Driver ModelsLuzheng Bi, Cui-e Wang, Xuerui Yang, Mingtao Wang, and Yili Liu.International Journal of Human-Computer Interaction:Feb. 2015,vol.31,no. 2,pp. 139-145摘要下载全文Driver intention detection is an important component in human-centric driver assistance systems. In this paper, we propose a novel method for detecting driver normal and emergency left- or right-lane-changing intentions by using driver models based on the queuing network (QN) cognitive architecture. Driver lane-changing and lane-keeping models are developed and used to simulate driver behavior data associated with five kinds of intentions (i.e., normal and emergency left- or right-lane-changing and lane-keeping intentions). The differences between five sets of simulated behavior data and the collected actual behavior data are computed, and the intention associated with the smallest difference is determined as the detection outcome. The experimental results from fourteen drivers in a driving simulator show that the method can detect normal and emergency lane-changing intentions within 0.325 s and 0.268 s of the steering maneuver onset, respectively, with high accuracy (98.27% for normal lane changes and 90.98% for emergency lane changes) and low false alarm rate (0.294%).
A Speed and Direction-based Cursor Control System with P300 and SSVEPLuzheng Bi, Jinling Lian, Ke Jie, Ru Lai, and Yili Liu.Biomedical Signal Processing and Control:9 August 2014,Vol. 14,pp. 126–133摘要下载全文In this paper, we propose a novel two-dimension (2-D) cursor control system by using steady state visual evoked potential (SSVEP) and P300 signals to control the direction and speed of the cursor, respectively. A hybrid stimulus interface is developed, where P300 visual stimuli are distributed at the top and bottom edges, representing accelerating and decelerating commands, respectively, and SSVEP stimuli, representing turning clockwise and counterclockwise, respectively, are located at the right and left sides. A classifier built with support vector machine is first used to distinguish between the direction and speed control commands, and if the classification result is the speed control command, a linear classifier is then applied to classify accelerating and decelerating commands. A real-time cursor control system is developed and tested with eight participants. The experimental results show that the cursor control system has satisfactory efficiency and accuracy, and the cursor movement is smooth and continuous. More importantly, it can work for the users, who are unable to produce event-related desynchronization/event-related synchronization (ERD/ERS) well but can produce P300 and SSVEP potentials compared to other 2-D cursor control systems. The proposed system can be used as a complementary, sometimes an alternative, system to the existing 2-D cursor control systems.
A Brain-computer Interface-based Vehicle Destination Selection System Using P300 and SSVEP SignalsXin-an Fan, Luzheng Bi, Teng Teng, Hongsheng Ding, and Yili Liu.IEEE Transactions on Intelligent Transportation Systems:Feb. 2015,vol.16, no.1,pp. 274-283摘要下载全文In this paper, we propose a novel driver–vehicle interface for individuals with severe neuromuscular disabilities to use intelligent vehicles by using P300 and steady-state visual evoked potential (SSVEP) brain–computer interfaces (BCIs) to select a destination and test its performance in the laboratory and real driving conditions. The proposed interface consists of two components: the selection component based on a P300 BCI and the confirmation component based on an SSVEP BCI. Furthermore, the accuracy and selection time models of the interface are built to help analyze the performance of the entire system. Experimental results from 16 participants collected in the laboratory and real driving scenarios show that the average accuracy of the system in the real driving conditions is about 99% with an average selection time of about 26 s. More importantly, the proposed system improves the accuracy of destination selection compared with a single P300 BCI-based selection system, particularly for those participants with relatively low level of accuracy in using the P300 BCI. This study not only provides individuals with severe motor disabilities with an interface to use intelligent vehicles and thus improve their mobility, but also facilitates the research on driver–vehicle interface, multimodal interaction, and intelligent vehicles. Furthermore, it opens an avenue on how cognitive neuroscience may be applied to intelligent vehicles.
Queuing Network Modeling of Driver Lateral Control With or Without a Cognitive Distraction TaskLuzheng Bi, Guodong Gan, Junxing Shang, and Yili Liu.IEEE Transactions on Intelligent Transportation Systems:Dec. 2012,vol. 13, no. 4,pp. 1810-1820摘要下载全文In this paper, we propose a computational model of driver lateral control based on the queuing network cognitive architecture and the driver preview model about driver lateral control activities. This computational model was applied to model the dual tasks of driving with a cognitive distraction task. The comparison between human driver data and model simulation data shows that this computational model can perform vehicle lateral control well, and its performance is consistent with that of drivers under single- and dual-task driving conditions. Furthermore, we examine the effectiveness of some parameters of the model in representing different styles of driving and discuss the value of this computational model in facilitating the evaluation of vehicle dynamics and driver assistant systems and providing new insights into research on unmanned vehicle control techniques.
Using Image-Based Metrics to Model Pedestrian Detection Performance With Night-Vision SystemsLuzheng Bi, Omer Tsimhoni, Yili Liu.IEEE Transactions on Intelligent Transportation Systems:March 2009,vol. 10, no. 1,pp. 155-164摘要下载全文The primary purpose of night-vision systems in civilian vehicles is to help drivers detect pedestrians. Pedestrian detection distance with night-vision systems has been modeled based on image metrics. However, the probability of pedestrian detection, in particular considering the factor of distance, has not been modeled based on image metrics. In this paper, we first describe a model of the probability of pedestrian detection, which compares several combinations of image-based clutter, contrast, and pedestrian size metrics using a simple mathematical equation. Next, we describe a model of the probability of pedestrian detection as a function of distance and image-based metrics by combining the model of pedestrian-detection probability and a model that represents the relationship between the distance to a pedestrian and an image-based pedestrian size metric. In the final model, image-based metrics are used to predict pedestrian-detection performance and can also be used to evaluate and support the development of night-vision systems in vehicles.
Using a Head-up Display-Based Steady-State Visually Evoked Potential Brain–Computer Interface to Control a Simulated VehicleLuzheng Bi, Xin-an Fan, Ke Jie, Teng Teng, Hongsheng Ding and Yili Liu.IEEE Transactions on Intelligent Transportation Systems:June 2014,vol. 15, no. 3,pp. 959-966摘要下载全文In this paper, we propose a new steady-state visually evoked potential (SSVEP) brain-computer interface (BCI) with visual stimuli presented on a windshield via a head-up display, and we apply this BCI in conjunction with an alpha rhythm to control a simulated vehicle with a 14-DOF vehicle dynamics model. A linear discriminant analysis classifier is applied to detect the alpha rhythm, which is used to control the starting and stopping of the vehicle. The classification models of the SSVEP BCI with three commands (i.e., turning left, turning right, and going forward) are built by using a support vector machine with frequency domain features. A real-time brain-controlled simulated vehicle is developed and tested by using four participants to perform a driving task online, including vehicle starting and stopping, lane keeping, avoiding obstacles, and curve negotiation. Experimental results show the feasibility of using the human “mind” alone to control a vehicle, at least for some users.
Development of a Driver Lateral Control Model by Integrating Neuromuscular Dynamics Into the Queuing Network-Based Driver ModelLuzheng Bi, Mingtao Wang, Cuie Wang and Yili Liu.IEEE Transactions on Intelligent Transportation Systems:2015,PP,pp. 1-8摘要下载全文This paper describes the development of a novel driver lateral control model by integrating the driver＼s neuromuscular dynamics into the queuing network (QN)-based driver lateral control model. Experimental results from 16 participants in a driving simulator show that, compared to the QN-based model, the proposed model performs better, and its performance is closer to that of drivers when a vehicle runs at a relatively high speed. The proposed model not only has the advantages of the models based on a cognitive architecture but also captures the dynamic interaction between the vehicular steering system and the driver＼s neuromuscular system. Thus, it can better represent driver lateral control and has greater value in supporting the development of driver assistance systems.
EEG-Based Brain-controlled Mobile Robots: A SurveyLuzheng Bi, Xin-an Fan, Yili Liu.IEEE Transactions on Human-Machine Systems:March 2013,vol. 43, no. 2,pp. 161-176摘要下载全文EEG-based brain-controlled mobile robots can serve as powerful aids for severely disabled people in their daily life, especially to help them move voluntarily. In this paper, we provide a comprehensive review of the complete systems, key techniques, and evaluation issues of brain-controlled mobile robots along with some insights into related future research and development issues. We first review and classify various complete systems of brain-controlled mobile robots into two categories from the perspective of their operational modes. We then describe key techniques that are used in these brain-controlled mobile robots including the brain-computer interface techniques and shared control techniques. This description is followed by an analysis of the evaluation issues of brain-controlled mobile robots including participants, tasks and environments, and evaluation metrics. We conclude this paper with a discussion of the current challenges and future research directions.