Biology and Medicine

Today we can see the increasing of a life expectancy of the world population and advances in medicine. But at the same time we have the increasing of a prevalence of injuries and various diseases, mainly of the cardiovascular and neurological diseases. Therefore, an important objective of healthcare organizations is to find an effective way to maintain a high working ability and intellectual activity of the population and further increase the life expectancy and improve the quality of life. For attaining this it requires a deep understanding of the mechanisms of functioning of the body, primarily the physiology of the nervous system, which regulates the activity of the whole organism. In this regard, the development of a modern medicine takes place in close cooperation with technical and engineering sciences. Millions of people with disabilities, including stroke patients, various injuries, amputation, and others are hoping for the success of this collaboration.

In recent years, the applied biorobotechnique made a fantastic career and made its way from a science fiction to commercial products, such as exoskeletons, prostheses, remotely operated robots, and robotic intelligent wheelchairs. Thanks to advances in technology along with increasing availability and computing power of such devices, today they are used for rehabilitation, enhancing physical capabilities, and replacement of lost body parts. The most important in designing such devices is the choice of control systems, which ensure accuracy, security, and stability for a long time. Today there are solutions based on the registration of biopotentials of the human body, such as electromyography (EMG), electroencephalography (EEG), electrooculography (EOG), and electrocardiography (ECG) [1-5]. At the same time, the available designs and products are based on EMG. There are some disadvantages that inhibit their active implementations in a social sector, in cases like, when a person is unable to generate a sufficient muscular signal, for example, in the case of a post-stroke disability, after spinal cord injury, etc. Violation of motor functions leads to the cessation of flow of information to the brain from the motor units (violation of reverse afferentation). As a result, the neurons of sensory-motor cortex undergo apoptotic death.

The development of a technology for brain-computer interfaces (BCI), capable of tracking the brain activity and transforming the intentions of a person to a command for the external devices, has already reached the level where they can be applied in the framework of biorobotechnique. So, in 2013, a work was published, which demonstrated the effectiveness of BCI in a system controlling the neuroprosthesis in a person with tetraplegia with relatively preserved sensory, emotional, and cognitive sphere [6]. This was the impetus for the development of medical products based on the knowledge of physiology of the nervous system.

Among the variety of ways of recording the brain signals, one of the most common methods is EEG, because of its low cost, availability, mobility, and security. It is known that the brain consists of 1010-1011 neurons that transfer information to the brain and from the brain back through a multifaceted neurotransmitter mechanism. Like a patchwork quilt, the cerebral cortex is divided into many areas with specialized functions [7]. Modern knowledge of physiology of the central nervous system and EEG allows us to introduce technologies based on biological feedback (biofeedback) with increasing control and efficiency with the help of EEG. For example, in the occipital lobe of the brain is the center of vision. This is the target for the detection of alpha waves by EEG, which normally occur in awake humans with closed eyes, and in a case of a distress the activity of the alpha rhythm fades away. Training and correction of the alpha rhythm, as well as the general biofeedback- EEG training, allows to improve the adaptation to stress, to reduce and sometimes to treat anxiety, depression, and various obsessive- compulsive disorders, and even helps to improve cognitive function in operated cancer patients [8-10].

Today there is no doubt that it is possible to distinguish the EEG signal depending on the various movements of the limbs. So, an approach for the creation of decoders on the basis of the EEG signal was proposed, which can reconstruct the corners at the joints of the fingers in these movements such as reaching out a hand for the object and the object hand capture the accuracy of the EEG signal in this case was 76% [11]. Contrary to the firm opinion about the necessity of using multiple channels of EEG for accurate diagnosis, J. Yang et al., by discarding noise, selected 6 out of 32 channels and introduced an upgraded classification of EEG signals for artificial neural network and control of the robot, while the classification accuracy when performing certain motor tasks reached 86% [12].

For the correction of inaccuracies of an isolated EEG signal in a low-signal stimulus, measured by using EMG, it is possible to use a combination of these approaches [13,14]. To compensate the lack of EMG signals in case of paresis or lack of limbs can be additionally used the EEG signal. This will allow to provide the movement of the prosthetic or exoskeleton by the “power of thought”. With a normal EMG signal, the EEG can reduce or neutralize tremor, artifacts, and adverse effects of fatigue.

With this in mind, a team of scientists has proposed the hybrid EEG-EMG system, and its effectiveness was assessed in 6 healthy volunteers [15]. The task for the volunteers was the performance of periodic motions by the left or the right hand (the command is executed on the color bar) for 5 s (total of 60 reps). Brain activity was recorded with 16 sensors, placed according to the standard 10/20 system. Muscle activity was recorded from the flexor and extensor of the left and right forearms. The obtained EMG signals were straightened and were averaged (time interval was 0.3 s). The results of the two classifiers were merged to obtain the control signal. Despite the fact that these EMG signals were quite informative (the classification accuracy averaged 83%), a hybrid approach has shown a greater efficacy (91%), especially in the increasing of muscle fatigue. Another group of scientists conducted a study of the hybrid EEG-EMG system with the assessment of the efficiency in the movement of the upper limbs by means of robotic devices [16]. The robot was equipped with a camera and was able to recognize the position of the hands through the sensors of the angle of rotation and force. The managing system used a 16-channel EMG interface, reading data from the arms and shoulder girdle. The experiment was conducted with the participation of four healthy, young people, who wore the exoskeleton and a device for EEG monitoring. In the first experiment, subjects performed movements of flexion and extension of hands in an elbow joint, at the same time the robot performed the opposite action (the robot tried to resist the movements of the subject). In the second experiment, on the table were placed two empty cups and a full one. While the subject took a full cup, the robot activated an algorithm of help, which evaluated the position of the empty cups with the camera and randomly chose one of them, and then assisted in the pouring of the liquid. The evaluation of correctness of the selection was carried out using the EEG and EMG signals. If the subject did not resist, the target was selected correctly; otherwise the robot changed its selection. The results of the experiment showed an improvement in the accuracy of interpretation of the human actions by the robot.

In addition to the EMG, signals from the cardiovascular system may enhance the efficiency of neurotechnology, recorded along with the EEG. Today the areas of invasive and noninvasive monitoring of blood circulation system, primarily the monitoring of blood pressure and heart rate (HR), are actively developing. The physiological parameters of the cardiovascular system have a high level of correlation with brain activity, including the performance of motional tasks [17,18]. The results of the study on heart’s working in the presence of a changing mental activity measured by using EEG, tell us about the prospects of using such hybrid systems [19,20]. The technology of hybridization of EEG and EOG is very promising. A developed EEG-EOG system demonstrated an improvement of the reliability of the long-term use of the arm exoskeleton during the performance of grasping movements [21].

Thus, the obtained data of different research groups demonstrated the prospects of using a hybrid approach for the control systems for external devices. The simultaneous fusion of several different biosignals (EEG, EMG, EOG, ECG, etc.) has the potential to significantly improve the performance of the robot, providing an increasingly complex movement in a safe mode. Currently, the results of such experiments are not available in a scientific literature; however, taking into consideration the speed of development of these technologies, the appearance of a high-precision neurodevice on the market in the near future is highly possible.

In this regard, at the Chemistry and Biology Institute of BFU named after I. Kant, a study was carried out, whose goal was the creation of multifunctional neurodevices and the exploration of opportunities of long-term monitoring of EEG, EMG, EOG, heart rate, saturation of oxygen (SpO2), body temperature, and motor activity with a remote transfer of data to and from an exoskeleton device.

Research work was carried out in stages. To date, was designed and manufactured the layout device to detect electrophysiological signals. Was carried out a testing of this device, as well as the study of hybridization of EEG and EMG signals.

Using EEG, the electrical activity of the brain was measured. In the present work, at the EEG registration, alpha rhythm was given attention, which normally is the most stable electrophysiological signal. Using EMG, muscle potentials were measured, and using EOG, potentials of the eye movement. A temperature test was conducted by a method of thermometry. The evaluation of cardiac activity was carried out using photoplethysmography from the skin of the index finger, where pulse oximetry was simultaneously done. The locomotor activity was studied by means of exercises with a change of the Euler angles with bending and turning of the head to the left, right, forward, backward, with a fixed neurodevice on it. All received data were recorded in a graphical form.

To conduct an objective analysis of the efficiency of the developed device, reference devices were used, which were proven on the market. The results of EEG, EMG, and EOG signals were compared with the results of the device KARDi3 (“Medical computer systems” firm, Russia), the device is positioned as a polygraph for lie detection tool. To compare the accuracy of measurements of body temperature, a multimeter Fluke 17b with a plug-in thermistor was used (Fluke Corporation, USA).

In the course of the work, the following studies were conducted. In the first stage, tests of the developed neurodevice were carried out. To do this, first a study of a physical activity was conducted. Then the registration and a comparative evaluation of neurodevice and device KARDi3 were carried out, using the following electrophysiological studies: EEG (100 tests), EMG (thigh muscles, 100 tests), and EOG (100 tests). A study of the heart work (heart rate) through photoplethysmographic tests using neurodevice and ECG instrument module KARDi3 (100 tests) was conducted. At the conclusion of the first phase of tests, a study to determine the temperature using the neurodevice and the device Fluke 17b (136 tests) was conducted.

For the registration of data from the EEG, EMG, and EOG on the neurodevice under study, our own software was written. For the registration of data from the device KARDi3, the program “Neocortex” was used. To study heart rate on the device KARDi3, the program “Neocortex,” was used, and for the neurodevice, the program “Heart Rate Monitor Demo” was used.

For the EEG registration, bipolar electrodes were placed according to the standard system—10/20, and the subject was constantly in the reclining position for a maximum relaxation of the muscles of the head and neck and to avoid EMG artifacts. For the registration of EOG, the electrodes were attached at the temples, around the right eye and on the forehead, and the subject was in a sitting position in the front of a board with graphic symbols.

The subject performed the following exercises: looking in the middle (yellow dot), looking up (red circle), in the middle (yellow dot) and down (blue circle), in the middle (yellow dot), to the left (red crosses), in the middle (yellow dot), to the right (blue cross), in the middle (yellow dot). The EMG signal was recorded by using a single electrode with a bipolar connection in the contraction of the hip muscles during the movement of the right foot forward, imitating the step. While the left leg remained motionless, the support of the right foot with the electrodes was not realized. The subject was in a standing position, the fulcrums were the left leg and the right arm, and the right leg with the electrodes was in a relaxed state. For the evaluation of the heart work, first R-R interval was recorded by placing on the wrists the electrodes of the device KARDi3. Further, a finger was placed
in the photoplethysmogram’s module of the neurodevice, and in the output, a signal occurring as a result of pulsation was obtained. A heart rate was measured with the help of neurodevice. It was also provided a pulse oximetry (SpO2). In the thermometry a temperature sensor was placed on the forehead of the subject by means of an adhesive ring. The temperature data were transmitted to the computer via Bluetooth for every second.

The abovementioned tests were carried out on two healthy men (22 and 23 years old, height 175 cm and 177 cm, weight 70 kg and 75 kg).

In the second phase of tests, the feasibility of the coregistration of EEG and EMG signals taken from the neurodevice. A union of these
two methods helped in a further work in a combination with robotic equipment, including the exoskeleton.

In this study 10 healthy males (right-handed) were included, aged 22-29 years (mean age [median] was 25 years). According to the instruction, the subjects were asked to visualize the movement of the left leg and then perform a flexion and extension of the hip. To assess the accuracy of the classification, at first only the signals of EMG was used, and then EMG1EEG. Each participant performed the experiment for 10 sessions for each case of the visualization of the movement and performing the movement. An assessment of the physiological parameter was performed continuously both during a movement and during a rest between the orders (5 s). For the classification, a linear discriminant analysis of Fisher was used.

In the study of motion activity while performing a series of exercises for tilting and turning a head to the left, right, forward, and backward, the results were obtained. They indicated the high accuracy and precision of the data obtained using the motion sensor of neurodevice (Figure 1). The graph shows the change in the Euler angle (X-axis— time, Y axis—angle): (1) The pitch—movement around the transverse axis (green); (2) Declivity—movement around the longitudinal axis (blue); (3) The yaw—movement around the vertical axis (red).

Figure 1: (a) An exercise by tilting a head to the left; (b) a reset (the angles of declivity and yaw, the pitch angle do not change, the gyroscope returns to its original position)

In some of the measurements, a “swimming” of gyro associated with a change in a magnetic field from the battery was observed. This is due to the relatively soft fixing of a battery to the model. With the rigid fixation of the battery to the neurodevice, this aspect was eliminated. In case of insufficient fixation of the neurodevice on the patient’s body, sometimes a simultaneous change of more than two angles while performing a single exercise was observed. This influence was eliminated by a more rigid fixation of the experimental sample of neurodevice. There was also a change of two angles at the same time during the performing of one exercise associated with an error due to the multilayered electronics. Components of a gyroscope were placed on the movement module, the movement module was placed on the neurodevice, and the neurodevice was placed on the head of the subject. In addition, it was established that a substantial contribution gives a relatively smooth movement of the neck of the subject. In the manufacturing of an experimental sample of neurodevice, it would be taken into account and would be used a software-hardware autocalibration of its position relative to the patient. In the experimental sample the basic parameters of a locomotor activity would be the linear acceleration of the accelerometer, angular acceleration of the gyroscope, and the magnetic field vector of the magnetometer.

EEG data obtained by the layout of neurodevice showed the same artifacts, and also the EEG data obtained from the device KARDi3. The number of artifacts was comparable. This demonstrates the build quality of the layout and the competitiveness of the developed neurodevice in this direction. Artifacts of a muscle activity are associated with the participant’s fatigue and, as a consequence, a small movement of the head and neck. On many of the EEG, an ECG artifact that may be associated with individual characteristics of the vascular system of the subject was present, as well as with the imposition of the electrode on the saphenous artery (Figure 2).

Figure 2: The image of EEG obtained (1) with KARDi3 (j: artifacts of a muscle tension, and -j: the ECG artifact); (2) from the layout of the neurodevice (g: artifacts of a muscle tension, and -j: the ECG artifact)

During the further stages of the working muscle (movement), artifacts on the EEG and EOG signals would be eliminated in the experimental sample by the additional notch filters or programmatically. The ECG artifact would be resolved by changing the static fixing of the electrodes to the dynamic, with the possibility of movement not less than 10 mm. This will help to move the electrode in case of a contact with a vessel. In addition, the improvement of a snap for attaching the electrodes will also contribute to the reduction of artifacts.

The EOG data of neurodevice, at least, were comparable in terms of informativeness with the results obtained from KARDi3. The number of artifacts was less with the recording of EOG by means of our developed model. This demonstrates the reliability of the results, their precision, and a high quality of neurodevice assembling (Figure 3). The most common artifact was blinking, manifested in the form of a sharp increase in the amplitude of the EOG. Artifacts of facial muscles that occur because of the participant’s fatigue were also recorded.

Figure 3: The electrooculogram of the subject, obtained using: (1) KARDi3 device (d, e, f—artifacts of a muscle activity); (2) layout of a neurodevice (no artifacts of a muscle activity)

In the course of the experiment for assessing the quality of EMG signal, it was determined that EMG data obtained by using the neurodevice have the same information content as the EMG data obtained from KARDi3 device (Figure 4). This demonstrates the reliability of the results, their accuracy and quality. No artifacts were found during this exercise.

Figure 4: The electromyogram obtained by: (1) KARDi3; (2) neurodevice (artifacts were not discovered)

The obtained results of tests in the research aimed for determining of a temperature indicated that the temperature sensor of neurodevice had a high stability and accuracy of the measurements. The measurement results were comparable with the referent device—Fluke 17b. The range of temperatures measured on both devices amounted to an average of 0.3%.

The detectedpulse signal was also obtainedintwo ways: by obtaining electrocardiograms by means of KARDi3 and photoplethysmogram by the module of neurodevice. Data of the R-R interval estimation of ECG obtained by using the device KARDi3, was characterized by the same values of heart rate as the heart rate measured by photoplethysmography module of a developed neurodevice. In average (median) the heart rate was 78 beats/min on KARDi3 and 77 beats/min on neurodevice in the first participant, and in the second—72 and 71 beats/min, respectively. Artifacts affecting the results were not detected (Figure 5).

Figure 5: Study of cardiac activity: (1) placement of electrodes and ECG obtained by KARDi3; (2) placement of electrodes and obtaining a photoplethysmogram and SpO2 saturation by using neurodevice

It should be noted that the developed module for the assessment of the cardiovascular system has the advantage, since it includes the assessment of blood oxygen saturation, which can also provide valuable information for the exoskeleton control.

In the study of the hybridization efficiency of EEG and EMG signals were obtained the results testifying about the benefits of combining the several electrophysiological signals. The experiment showed that the accuracy of the EMG averaged 74.3%, and hybridization of EMG and EEG allowed increasing in all subjects the classification accuracy on average by 12.5%, reaching the level of 86.8% (75-97%). The results of each test are presented in Table 1.

Table 1: Results of classification accuracy (%)

The obtained data allows us to conclude that the combined use of EEG and EMG signal improves the accuracy of the interpretation of the planned and actual physical activity. The EEG signals that do not dependent on a muscular work, are an optional identifier that can be used in a robotic technology. It is assumed that a further upgrade of the system and simultaneous use of different physiological studies will bring the diagnostic accuracy to 100%.

It is obvious that the establishment of high-precision multifunction neurodevice is of a great importance. It allows for the registration of the physiological signals for a long time and to transmit the information to an external device (exoskeleton). This will give the opportunity to return to an active social life and a home life for millions of people with disabilities.

Based on the results of this research we can conclude about the feasibility of using a hybrid approach to control systems for external devices. The merging of multiple modalities or switching between them in order to find those that better embodies the intention of a person not only increases the accuracy but also serves as an evidence of a potentially higher productivity of robotic equipment. The experimental results of the EEG and EMG signals hybridization confirmed the success of various research groups and the feasibility of different physiological signals merging [15,16,21]. A combination of multiple types of biological signals as the source of the control commands should be considered as promising. And a further research with a higher representative number of subjects, including those with different nosological forms, has been found to be justified. It is expected that further studies will allow us to proceed to the formulation of different technical solutions for the individual modules and interfaces of the developed platform, which will become the basis for the creation of a highly accurate and safe multifunctional neurodevice and be able to satisfy medical and social queries.

At the further stages it has been planned to improve the layout, its miniaturization and intellectualization. It is assumed that in the end, the developed device would be able to decode the intentions and emotions of a person and to adapt to the dynamic changes of the surrounding world, thus providing a social integration of people with disabilities.

This work was supported by The Ministry of Education and Science of the Russian Federation, grant number RFMEFI57815X0140 from Immanuel Kant Baltic Federal University. The authors would like to thank Danil Borchevkin, Alexey Belousov, and Vladimir Savinov for preparing the manuscript.