Functional electrical stimulation controlled by
artificial neural networks: pilot experiments with
simple movements are promising for rehabilitation applications
Elena De Momia
aNITlab TBM LAB, Bioengineering Department, Polytechnic of Milan, Italy
bFDG Bioengineering Centre, IRCCS Don C. Gnocchi Foundation ONLUS, Milan, Italy
Reprint requests to: Dr Alessandra Pedrocchi
Dipartimento di Bioingegneria
Politecnico di Milano
P.zza L. da Vinci, 32 - 20133 Milan - Italy
Accepted for publication: December 3, 2004
This study falls within the ambit of research on functional electrical stimulation for the design of rehabilitation training for spinal cord injured patients.
In this context, a crucial issue is the control of the stimulation parameters in order to optimize the patterns of muscle activation and to increase the duration of the exercises. An adaptive control system (NEURADAPT) based on artificial neural networks (ANNs) was developed to control the knee joint in accordance with desired trajectories by stimulating quadriceps muscles. This strategy includes an inverse neural model of the stimulated limb in the feedforward line and a neural network trained on-line in the feedback loop.
NEURADAPT was compared with a linear closed-loop proportional integrative derivative (PID) controller and with a model-based neural controller (NEUROPID).
Experiments on two subjects (one healthy and one paraplegic) show the good performance of NEURADAPT, which is able to reduce the time lag introduced by the PID controller. In addition, control systems based on ANN techniques do not require complicated calibration procedures at the beginning of each experimental session. After the initial learning phase, the ANN, thanks to its generalization capacity, is able to cope with a certain range of variability of skeletal muscle properties.
KEY WORDS: artificial neural networks, functional electrical stimulation, non-linear adaptive control systems, rehabilitation engineering.
Research in the field of functional electrical stimulation (FES) targets solutions which could guarantee effective and lasting recovery of lost or impaired capacity for movement. In the pursuit of this aim, technology is being improved through the study of electrodes (1) and implanted micro-stimulators (2). Moreover, control systems have been developed that allow stimulated limbs to achieve the desired movements. The study described here belongs to this latter area of research. Even a very simple and common movement, such as extending the knee, can be performed in very different ways by the muscle of our legs. The fine control system, which pilots the movement in healthy subjects, is able to reach the desired trajectory by tuning the recruitment of muscle fibres so as to reduce fatigue and modify muscle activation in response to fatigue or unexpected perturbations. The way we learn all these control mechanisms is a very sophisticated process based on experience. To succeed in mimicking this sophisticated controller (at least for very simple movements) when pathology has corrupted it, is one of the great challenges for bioengineering.
Although the long-term objective of this study is to achieve recovery of some motor functionality in spinal cord injured (SCI) patients, there is also a very interesting short-/medium-term objective that deserves consideration, i.e., the design of optimized rehabilitation training exercises to increase cardiovascular conditioning. Since, nowadays, SCI patients usually survive long after their trauma, the need for good conditioning of the paralyzed parts, to protect against cardiovascular disease, is very important. SCI patients are very susceptible to vascularization problems and 10-20 years after the trauma these problems start to become major invalidating factors that further impair their quality of life. The possibility of offering these patients FES-based rehabilitation training is a very interesting step forward. FES allows recovery of muscle tone, promotes vascularization of the paralyzed limbs, and offers the best prevention and recovery prospects for decubitus (3,4).
In this study the key point was that the developed strategy allowed the training exercise to be controlled and tailored, as far as possible, to the needs of the single patient and to the single rehabilitation session, monitoring muscular conditioning and fatigue. The main issue was that it offered the possibility of controlling on-line the stimulation level, taking into account the modifications of the muscular condition during each experimental session and the improvements made session by session. In this study, we chose a relatively simple movement in order to focus our attention on the actual control techniques, which were first evaluated through simulation trials and then validated through experimental sessions.
Proportional integrative derivative (PID) controllers are widely used in engineering, as well as in biomedical applications, including control of FES (5). Some of the studies on PID controllers for FES performed in the past (5,6) were aimed at comparing the performances of different controllers; others were related to the application of PID controllers to the recovery of FES-induced functional movements (3,4). In particular, it was shown (3) that the good tracking performance of PID controllers was offset by a considerable time lag between reference and actual joint angle, which became more marked when exercises were protracted in time. This time lag could be reduced by adding a feedforward inverse model, which also worked as a compensator for the non-linearity of the controlled system.
In addition to the traditional open- and closed-loop control schemes (7,8), model-based control systems have been used in FES (5,9). These included a neuro-musculo-skeletal model of the system to be controlled, which could be a forward or an inverse model, depending on its position in the control system. Unfortunately, the large quantity of parameters required for identification of the system to be controlled is difficult to determine experimentally. The ability to reproduce physiological control mechanisms is crucial to the development of high-performing controllers, because biological systems are non-linear, time-varying (muscle fatigue), redundant and very difficult to model analytically (10). Moreover, muscular properties change as the condition of the subject improves, which means that they require recalibration for each session in order to maintain optimal performance.
The physiological model could be replaced by a non-linear black box model, such as an artificial neural network (ANN). Like their biological counterparts, ANNs can learn from examples and generalize. They can successfully cope with the complexity and non-linearity of systems. Chang et al. (11) suggested combining an ANN (acting as an inverse model of the system to be controlled) with a closed-loop, fixed-parameter PID feedback controller, thereby adjusting for residual errors due to external disturbances or to erroneous model identification. The controlled system was the quadriceps. The performance of the neural controller was limited by model identification errors, by the time-varying system dynamics, and by the variability of the skeletal muscle properties.
Starting from a control scheme similar to Chang’s, we developed an adaptive control strategy (NEURADAPT) using an ANN trained on-line in the feedback loop. The rationale of the proposed control strategy was inspired by the way humans execute voluntary movement. A good model of the actuators and of the anthropometric and strength characteristics is stored in the central nervous system (CNS) as a long-term memory. On the basis of this knowledge, the CNS produces an initial feedforward motor programme. In order to optimize the motor command during the exercise, the CNS also uses a short-term memory. The NEURADAPT system was based on this scheme. First, it included a permanent model of the lower limb properties in the feedforward line, which represented the long-term memory and the internal model used by the CNS to send the open-loop motor command for any well-known voluntary movement. We called this the inverse model (IM). Then, the adaptive feedback controller was used to monitor the execution of the exercise and was trained and modified on-line according to the error found between the actual and the desired movement. Therefore, this component was necessary to reduce and compensate for impaired tracking of the desired angular trajectory due to muscle fatigue. As in physiological conditions, when the fatigue effect appears, a given contraction is no longer able to produce the desired movement and a larger contraction, which means more recruited fibres or higher frequency, is produced, modifying the delivered stimulation.
The NEURADAPT control system allowed optimal tracking of knee joint rotations during repeated flexion-extension movements produced by surface electrode stimulation of the quadriceps.
After a feasibility simulation study, the performance of this new control system was evaluated through experimental sessions in two subjects: a healthy subject and a paraplegic patient.
The control systems considered in this study were based on the generic scheme shown in figure 1(a), in which the control signal delivered to the plant was the sum of the stimulation signal predicted by the inverse model in the feedforward loop and the correction provided by the feedback controller.
Three control systems were compared: a traditional closed-loop PID controller (Fig. 1b), a model-based neural controller (NEUROPID) (Fig. 1c), in which the inverse model was identified by an ANN and the feedback control was calculated by a PID controller and the adaptive neural controller (NEUROADAPT) (Fig. 1d), which included the ANN inverse model in the feedforward line and the adaptive ANN trained on-line in the feedback loop.
The inverse model had to transform the kinematics of the desired movement into the stimulation pulse width. The adaptive neural control strategy proposed (NEURADAPT) (Fig. 1d) used an ANN feedback controller, which was trained on-line to produce a suitable and stable compensation signal in response to the changes occurring in the plant, which were not well described by the feedforward inverse model. The Appendix describes in detail the neural network topology, the building of the training sets, and the optimization of the neural network architectures.
The control systems were first evaluated through simulation trials and then experimentally tested on two subjects. Experiments were performed on the healthy subject first, in order to determine the optimal experimental setup and protocol, and then evaluated on a paraplegic patient.
In order to evaluate the control systems in simulations, a complex model (12), implemented in Simulink®, was used to simulate the lower limb of a paraplegic subject. Plant parameters were optimized according to the force measurements and anthropometric data of a paraplegic subject. The plant (Fig. 2a) was restricted to movements in the sagittal plane and the knee was assumed to be an ideal hinge joint, in accordance with previous studies that adopted the same experimental setup (13).
The model also represented muscle fatigue according to the equation proposed by Riener (14).
Inputs to the plant (Fig. 2b) were the pulse widths produced by the stimulator and delivered to the quadriceps through the surface electrodes. The plant output was the computed knee joint position as resulting from stimulation of different muscle groups or from passive oscillations. Muscle groups could be treated independently and they differed in activation parameters (contraction time, recruitment threshold, etc.) as well as contraction parameters (muscle length, isometric force, etc.). When the quadriceps was stimulated, other muscles, through their passive properties, still contributed to the limb dynamics (5). The dynamic block (Fig. 2b) in fact, took into account the elastic and the viscous torque.
Setup. Subjects sat on a bench (Fig. 3), which allowed the lower leg to swing freely whereas hip angle (g) was fixed at 60 degrees of flexion and ankle (a) at 0 degrees by an artificial foot orthosis.
Quadriceps muscles were stimulated by adhesive rectangular surface electrodes with the cathode placed proximally over the estimated motor point of the rectus femoris and the anode approximately 4 cm proximal to the patella. In this way, the other heads of the quadriceps, too, were activated.
Knee joint angle was measured by an electrogoniometer placed, as shown in figure 3, on the thigh and on the shank.
The linear potentiometer fixed on the joint of the two steel bars of the electrogoniometer, was interfaced with the PC through an A/D board with a sample rate of 100 Hz.
A computer-driven multi-channel stimulator delivered balanced bipolar rectangular pulses. The stimulating current was fixed at a value previously defined through preparatory tests carried out on each subject. The pulse width, which is the controlled variable, could nominally range between 20 and 500 ms; preparatory tests on each subject reduced the range, as described in the Appendix.
Procedure. Two volunteers, one healthy female subject, SF, and one 33-year-old male with paraplegia, CB, gave their consent to the study and took part in the experiments.
CB had a complete spinal cord lesion at T5-T6 level, thus his lower trunk and lower limbs were paralyzed. He was not pre-conditioned, thus the control sessions were limited by an excessive muscular fatigue effect.
The initial values of PID parameters were determined during the simulation session and the tuning was performed by a trial and error procedure in order to minimize angular trajectory errors.
Performance indices. The performances of the different control systems (PID, NEUROPID, NEURADAPT) were evaluated over repeated flexion-extension movements (5 in simulation and for the healthy subject, 3 for the paraplegic patient). In simulation and for the healthy subject, two different exercise conditions were evaluated: with and without an external 1 kg load at the ankle (free condition and weighted condition). The patient could not be submitted to the weighted condition because of the lack of muscular pre-conditioning.
The validation indices taken into consideration were those proposed by Ferrarin et al. (5): root mean square error (RMSE), time lag (TL) and tracking error (Tr), computed between the desired and actual trajectories.
The RMSEs between the desired and obtained trajectories are compared in figure 5 (a and b).
ANOVA comparison of RMSEs of the three control systems highlighted significant differences. The highest p value was 0.0001 in the free condition and 0.0055 in the weighted condition. In the free condition the PID TL was 280±42 ms, and the NEUROPID TL was 5±16 ms, whereas NEURADAPT showed no delay in the delivered control signal, indicating a huge improvement in performance with respect to the PID system. Analogously, in the weighted condition, the PID TL was 245±37 ms, the NEUROPID TL was 10±21 ms, and NEURADAPT showed no delay in control.
In the healthy subject, we compared the performances of the three control systems during repeated cycles of limb flexion-extension under two conditions (free and weighted swinging), whereas the paraplegic patient could not be submitted to the weighted experimental session because of poor muscular pre-conditioning.
In the healthy subject, the free swinging performance was not particularly significant because five swings were not sufficient to induce visible muscular fatigue. Indeed, the only positive effect was produced by the inverse model, thus NEUROPID and NEURADAPT showed similar behaviour.
The results obtained in the healthy subject in the weighted condition were instead more relevant. As shown in figure 6a (upper panel), the angle profiles produced by NEURADAPT revealed an evident reduction of the time lag between the desired and the actual trajectory. In addition, even at the fifth cycle, NEURADAPT, unlike the other two systems, still showed good tracking of the desired angle. Moreover, this good performance was associated with lower corresponding pulse width values (Fig. 6a, lower panel) – and low pulse width values help to reduce the velocity of the incoming fatigue phenomenon. As in the simulations, a large stimulation pulse width between consecutive oscillations was shown by the PID controller but not by the other systems.
The performance, in terms of RMSE, of the healthy subject confirmed the results obtained in the simulations (Fig. 6b). Comparison of the results revealed statistically significant differences between the three control systems (ANOVA p<0.05).
As reported in Table I, the discriminating factor between the systems was TL, whereas their Tr was always comparable.
Comparison of the performances of the different control systems, in terms of angular trajectories and pulse widths, for the paraplegic subject, is shown in figure 7a (over). A definitely worse ability to track the desired trajectory was evident for the PID controller (Fig. 7a, upper panel), while the angles produced by NEUROPID and NEURADAPT were similar.
One of the most important observations, as in the simulations and in the healthy subject experiments, was that the PID stimulated the muscle also during the oscillations between consecutive flexion-extension movements. This was attributable mainly to the PID controller delay. On the contrary, NEURADAPT kept the level of stimulation in between consecutive cycles below threshold, thus completely overcoming this problem.
The quantitative results (Fig. 7b, over) confirmed the potentiality of the NEUROADAPT system. NEUROADAPT was able adequately to compensate for the muscular fatigue effect.
NEURADAPT showed a similar Tr error to the other strategies but it reduced the TL and thus gave lower RMSE values (Table II).
The aim of this work was to design an ANN-based system able to control the electrical stimulation of the quadriceps muscle, in order to obtain repeated knee flexion-extensions, paying particular attention to the fatigue effect. A relatively simple movement was chosen in order to focus attention on the control systems, first implemented in simulation and then experimentally evaluated.
In order to provide SCI patients with efficient FES training, the exercise has to be prolonged as much as possible to allow conditioning of their paretic muscles. Therefore, in this study, the stimulation parameters (current amplitude and pulse width) used to achieve the desired movement had to be kept as low as possible. Indeed, it was crucial to avoid over-stimulation, which would have over-fatigued the muscles. It is also very important to remember that the muscles of paraplegic patients are usually very poorly conditioned, which amplifies the fatigue effect. In addition, FES first activates the fast fibres (with larger diameter and lower threshold) and, accordingly, these fatigue very quickly. Unfortunately, at least with surface electrodes, this recruitment order cannot be reversed, thus the control systems were crucial in order to keep the level of stimulation as low as possible.
Model-based control systems combined with a closed-loop, adaptive strategy (NEURADAPT) are a very promising technique in the FES field, as they allow stimulus modulation suited to a functional movement and to the conditions of the subject. In the literature, numerous linear, non-linear and physiological models for control applications have been proposed (15,16). The most sophisticated physiological models are difficult to identify because of their considerable non-linearity. This explains the difficulty of applying model-based control strategies.
In particular, the sheer quantity of experimental data needed precluded their use in paraplegic subjects, and especially in clinical applications, which instead ought to be the objective of any meaningful study in this field.
In the literature, numerous papers (10,17,18), demonstrated the ability of neural networks to identify non-linear dynamic systems. The ANN architecture proposed in this study, the multilayer perceptron, has already been used for the identification of the muscle skeletal system (17,19) and shown considerable accuracy and good generalization ability.
To compare and evaluate the performance of the new controller, we also implemented a traditional closed-loop control system (PID) and a hybrid control system (NEUROPID), both already proposed in the literature (11).
As regards the simulation results, NEURADAPT gave the best performance. This new control system, in fact, benefited from the simultaneous presence of a block that included non-linearity (neural inverse model) and of a block able to correct the error due to time-varying behaviour (adaptive neural network).
Comparison of the results obtained in a healthy subject and in a patient showed NEURADAPT to produce the best performance in experiments, too. The improvement, already present during the free swinging movements of the healthy subject, was more evident in the weighted condition, because it was only in this condition that the healthy subject felt the effects of muscular fatigue and on-line parameter adaptation became essential in order to offset this phenomenon. During trials on the paraplegic subject, who was strongly affected by muscular fatigue, NEURADAPT showed the best performance when this was evaluated in terms of RMSE. It had a tracking ability superior to that of the PID, especially in the last cycle thanks to its adaptation capacity and in particular to its ability to reduce the delay in control shown by the PID controller. Indeed, this time lag produced passive oscillations of the leg which were out of phase with respect to the desired movement. For this reason, the PID controller produced undesired pulse width peaks that were over the activation threshold between consecutive extensions and caused excessive muscular fatigue. This aspect is very important if the controlled subject is paraplegic, because this phenomenon could slow down his recovery.
The different control performance between the traditional controllers and NEURADAPT was due to two different effects. The first was the delay introduced by the feedback controller (particularly evident in the PID simulations), which was partly reduced by the presence of the inverse model (NEUROPID delay was always under 50 ms), and partly by the adaptive feedback neural network action. The second was related to a sort of predictive ability of the adaptive ANN. In this way, there was an improvement in the ability to produce the desired trajectory, especially coinciding with the passive oscillations present between consecutive cycles of movement. These effects, already visible in the free simulations, were emphasized during the weighted trials. NEURADAPT bettered the performance of the traditional PID controller because it exploited the PID system only when it was working well. What is more, even though the feedback network was trained to give the PID output, it was able to accomplish this more efficiently than the PID controller because it showed deeper exploration of the input space and it worked as a sort of short-term memory which self-updated (in particular its weights) according to the different muscular conditions during the exercise.
Another important aspect to be emphasized in all the FES applications for the rehabilitation of SCI patients was the time required for tuning the whole-system parameters. Indeed, the use of ANNs can render the identification phase shorter and simpler than the identification phase required by traditional model-based controllers. With NEURADAPT the experimental session devoted to inverse model identification was simpler than with the PID controller because it consisted of a predetermined number of single flexion-extension movements, which could be positive for patient muscular tone rehabilitation. Optimization of the PID parameters required, instead, a variable number of trials, this number depending on several factors, such as limb properties and muscular conditions. In this study NEURADAPT did not show a short identification phase because the desired output on the adaptive feedback neural network depended on the PID, but the development of a totally ANN-based control strategy could exploit this advantage. Another aspect supporting the use of neural networks in the control of FES is that in the case of the paraplegic patient, in spite of the reduced training data collection (due to low muscle conditioning), the neural network preserved the ability to determine a bijective mapping between pulse width and knee kinematics since it was still trained on a wide range of variability. Moreover, an appropriate choice of training set could allow, in a future work, the construction of a general inverse model that would fit different subjects, healthy and paraplegic. In this context, correct muscular response identification, after the selection of the adequate individual current amplitude, could allow reduction or elimination of the identification phase, which, if performed a long time before the control session, could imply a variation in the subject’s muscular properties.
Our work shows that ANNs could be utilized to control time-varying non-linear systems and in particular that they could effectively control biological systems. Although the capabilities of neural networks still have to be completely explored, they already seem to be valid in the realization of FES control systems.
Future research efforts should be focused on the development of a new control strategy totally free from the PID controller.
The extension of this methodology to more complex stimulation setups, e.g., a multi-input multi-output (MIMO) system, is theoretically possible but has not yet been investigated with a view to specific applications. In this way it may prove possible to control simultaneously different muscles involved in the generation of movement. This would make it possible to mimic the muscular synergies and, in particular, the contribution of antagonist muscles. MIMO neural network architectures have been successfully proposed for different applications (20). The use of MIMO controllers will open up the problem of the competitive strategies normally used in human movements: similar movements could be produced by different muscular strategies. The taking into account of this variability, which is crucial to the recovery of muscular fatigue, will require more complex control schemes, with the possible introduction of stochastic mechanisms.
Subsequent developments may the realization of different actions such as cycling or standing up and the control of the upper limb.
The ability of neural networks to adapt their parameters on-line seems to be an important aspect of muscular system control by FES. The possibility of capturing changes due to muscular fatigue, in order to correct the stimulation signal, appears to be a crucial starting point for the development of highly efficient control systems.
These aspects, very interesting if the stimulation is delivered via surface electrodes, must, in the future, also be considered for implanted electrodes.
The inverse model of the controlled plant was modelled by a multilayer ANN, shown in figure 8, similar to the scheme proposed by Chang (11).
A multilayer perceptron with a single hidden layer and external dynamic was used. Neural network inputs included a time window (4 lags indicated by z-1 in figure 8) of knee angle and velocity samples. Activation functions of neurons were the hyperbolic tangent for the hidden layer and the logistic function for the output layer. In this way the non-linearity intrinsic in the plant could be well learned by the inverse model. The neural network computed the pulse width of the stimulation normalized between 0 and 1. Neuron weights were initialized according to the Nguyen-Widrow method (21).
The feedback ANN used in the NEURADAPT control system was also a multilayer perceptron, with a single hidden layer. Inputs were five delayed samples of the angular error and the output was the correction of the control signal, i.e., the extra pulse width that had to be added to the one given by the inverse model in order to compensate for the errors occurring. The activation functions were the hyperbolic tangents both for the hidden and the output layer, in order to provide a better identification of the non-linearities of the system to be identified and controlled and in order to produce a positive or negative error adjustment, representing, respectively, excitation and inhibition effects.
1) Training sets of the inverse model
Approaching a control problem with ANNs, the crucial issue was to build the training set, i.e., the group of input/output examples used to train the network weights. The training set was required to describe well the working space and thus represented the behaviour that we wanted to teach to the network. The data samples used to train the inverse model were obtained by stimulating the rectus femoris and vasti muscles of the subject/plant in order to perform single flexion-extension movements without time-varying effects due to muscular fatigue.
With the aim of exploring the whole space of the possible system behaviour, the training set was made up of an enlarged and diversified set of stimulation patterns. The waveforms used both in the simulations and in the two experimental sessions were rectified sinusoids and triangles. Levemberg-Marquardt training algorithm was used for the inverse model neural network (22).
In order to optimize the inverse model, several architectures were trained and tested varying the number of the hidden layer neurons and the number of delayed input samples. The different architectures were evaluated based on three factors, first the value of the RMSE between the desired output of the network and the one produced. Second, we wanted the network to minimize the difference between RMSE computed on training and testing data (23); this would indicate that the network was generalizing well over the whole working space and not overfitting the training set. Third, we wanted to minimize the complexity of the network avoiding long computing times. Thus, the smallest network architecture that gave good RMSE and similar performance between training and testing data was chosen.
2) Training sets of the inverse model: simulations
In simulations, all the combinations of rectified sinusoids and triangles with durations of 2, 4, 6, 8 and 10 seconds, with pulse width ranging from 200 to 450 ms and with 2, 4 and 6 samples of rest between two consecutive waveforms composed the training set, as depicted in Table III (a). The training set was randomly given to the neural networks in order to avoid the polarization toward the last learned samples. The whole training set counted 180 different waves repeated three times. The test set instead included 36 waveforms lasting 5 seconds, with pulse width ranging from 200 to 450 ms and with 2, 4 and 6 samples of rest between two consecutive waveforms. The testing set was within the working space but not included in the training set, so as to provide information on the capacity of the network to generalize over the whole working space.
3) Training sets of the inverse model: experimental sessions
The protocol for the experimental identification of the inverse model was the following: first positioning the stimulation electrodes, second assessing the muscle contraction to verify the correct positioning of the electrodes. Preparatory tests were used to define the individual stimulating current and range of pulse width. Parameters of stimulation (current amplitude and pulse width range) were selected in order to achieve the required maximal (J=180°) and minimal (J=110°) extension. Each stimulation induced only one single flexion-extension cycle in order to capture the non-linearity of the muscle, avoiding fatigue effects and passive oscillation between two consecutive movements. Pulse width ranges were from 170 ms to 220 ms for the healthy subject and from 170 ms to 320 ms for the paraplegic patient. The stimulating current for the healthy subject was fixed at 30 mA and for the paraplegic at 110 mA.
To build the training set for inverse model identification the subject’s muscles were stimulated with pulse width in the whole range. This allowed a good spanning of the whole parameter space, both in pulse width and angle (training set input/output).
The stimulation waves were rectified sinusoids and triangles for both the subjects; the duration and the pulse width are reported in Table III (b) and (c). The training set for the healthy subject consisted of 5700 samples, and for the paraplegic patient of only 2040 samples, because of the anticipated occurrence of fatigue.
The trained ANNs had the same number of layers and activation functions as adopted in simulation and described in the previous sections. In any case, the architecture of the network, i.e., the number of neurons in the hidden layer and the number of delay samples in the input, was chosen in order to optimize the performance of the ANNs trained with the experimental data.
4) Inverse model architecture optimization: simulation results
The network architecture for simulation was chosen according to the three factors described in subsection 1) of this appendix.
The network with 4 delays and 18 neurons in the hidden layer (Fig. 9) proved to be the smallest one giving good performances on both the training and the testing sets.
5) Inverse model architecture optimization: experimental results
According to the same criterion described in the previous section, the optimal architecture of the inverse model was identified for each subject. For the healthy subject the chosen network had 4 delayed angular samples and 12 neurons in the hidden layer (Fig. 10a). For the patient the chosen inverse model presented 4 delays and 16 neurons in the hidden layer (Fig. 10b).
6) Adaptive ANN feedback controller architecture and training sets
Neural network inputs were the angular error samples at the current time and in the four previous instants, whereas the desired output was the pulse width correction obtained by the PID controller in response to the actual error.
In order to train this network, Levemberg Marquardt algorithm was used. Being computationally faster than backpropagation, it updated the weights compatibly with the sample rate of the current impulses generator (20 Hz).
If the error was zero, this feedback module had no significant inputs and it did not contribute to correct the pulse width, which was well estimated by the inverse model. This meant that the feedforward control was predicting well the plant/subject behaviour. Otherwise, when the inverse model pulse width was not actually making the plant/subject perform the desired movement and an error occurred, the feedback neural network was trained on the error so that it produced a considerable extra pulse width contribution. The desired output used to train the feedback neural network was calculated by the PID controller, but in the future, methods of reinforcement learning would be used in order to optimize the control action.
The PID controller general form in the time domain was given by:
u(t) = Kp e(t) + Ki ∫e(t)dt + Kd ———
where: e(t) was the difference between the reference and the actual value of the controlled variable, and Kp, Ki, Kd, were the proportional, integrative and derivative parameters respectively.
The PID controller parameters were first identified by simulations using an iterative procedure based on the minimization of RMSE (6), where the initial estimate of the optimization was derived from the Ziegler-Nichols rules (24).
Figure 1 - Block diagrams of the control strategies implemented: (a) general model-based closed loop control, (b) feedback PID control, (c) the combination of a feedforward controller, neural inverse model, and a feedback controller, NEUROPID, (d) the adaptive control system, NEURADAPT.
Abbreviations: PID=proportional integrative derivative; ANN=artificial neural network.
Figure 2 - (a) Mucle-skeletal model adopted in the present study. The model takes into account 5 muscles: 1 rectus femoris, 2 vasti, 3 hamstrings, 4 biceps femoris, short head, 5 gastrocnemius. The bold line indicates stimulated muscles. In our setup, hip angle g and ankle angle a were fixed respectively at 60° and 0°. (b) The plant simulator scheme.
Figure 3 - Experimental setup.
Figure 4 - Sequence of free (a) and weighted (b) flexion-extension movements controlled by the different strategies in simulations. The upper panels show the angular values, the lower panels show the pulse widths.
Figure 5 - Comparison of the performance (based on root mean square error, RMSE) of the three control system for each swing cycle of the repeated sequence. Results are shown in two different conditions: free (a) and weighted (b). *ANOVA comparison of the three strategies revealed statistically significant differences (p<0.05).
Figure 6 - Weighted swinging movements of the healthy subject controlled by the different strategies (a) and comparison of the controller performances (based on root mean square error, RMSE) for each swing cycle (b).
Figure 7 - (a) The sequence of swinging movements performed by the paraplegic subject in free condition with the different control systems: the angular values are shown in the upper panel and the pulse widths in the lower panel. (b) Comparison of the performances of the control systems (in terms of RMSE) for each swing cycle.
Figure 8 - Architecture of the neural inverse model: qd was the desired angular trajectory of the knee. The z-1 blocks indicated the time lags. Pulse width (PW) was the learned pulse width. The number of neurons in the hidden layer depended on network architecture optimization procedure.
Figure 9 - RMSE for training and testing data, calculated for networks with different number of delays (p2=2 delays, p3=3 delays, p4=4 delays) and with different number of neurons in hidden layer.
Figure 10 - RMSE for training and testing data, calculated for networks with different numbers of delays (p2=2 delays, p3=3 delays, p4=4 delays) and with different numbers of neurons in the hidden layer. Panel (a) shows the network architecture optimization for the healthy subject, panel (b) for the paraplegic patient.