Mathematical Modeling of a Self-Learning Neuromorphic Network Based on Nanosized Memristive Elements with a 1T1R-Crossbar-Architecture

Artificial neural networks play an important role in the modern world. Their main field of application is the tasks of recognizing and processing images, speech, robotics, and unmanned systems. The use of neural networks is related to high computational costs. In part, it was this fact that held back their progress, and only with the advent of high-performance computing systems did the active development of this area begin. Nevertheless, the issue of speeding up the work of neural network algorithms is still relevant. One of the promising areas is the creation of analog implementations of artificial neural networks, since analog calculations are performed orders of magnitude faster than digital ones. The memristor acts as the base element on which such systems are built. A memristor is a resistor whose conductivity depends on the total charge passed through it. Combining memristors into a matrix (crossbar) allows one layer of artificial synapses to be implemented at the hardware level. Traditionally, the Spike Timing Dependent Plasticity (STDP) method based on Hebb’s rule has been used as an analog learning method. A two-layer fully connected network with one layer of synapses is modeled. The memristive effect can manifest itself in different substances (mainly in different oxides), so it is important to understand how the characteristics of memristors affect the parameters of the neural network. Two oxides are considered: titanium oxide (TiO2) and hafnium oxide (HfO2). For each oxide, a parametric identification of the corresponding mathematical model is performed for the best agreement with the experimental data. The neural network is tuned depending on the oxide used and the process of learning it to recognize five patterns is simulated.


INTRODUCTION
Artificial neural networks are used in many areas of modern life and allow solving urgent, important, and practically significant problems that often cannot be solved using classical approaches. To speed up the operation of neural network algorithms, special processors are being developed, based on the principles of the human brain and representing a hardware implementation of pulse (spike) neural networks. The use of analog calculations instead of digital ones seems to be promising in this direction, since they are performed orders of magnitude faster. In relation to this, the creation of analog neuromorphic systems is an urgent task.
A memristor is a resistor whose conductivity changes depending on the total electric charge flowing through it and which is an elementary cell of longterm nonvolatile memory [1,2]. Combining memris-tors into a matrix (crossbar) allows us to perform a fast analog matrix-vector product [3,4]. Due to a certain similarity of memristive elements with a biological synapse, it seems promising to use them for the analog implementation of self-learning pulse neural networks.
Previously, the authors have already investigated the possibility of using memristors for analog implementation of second-generation convolutional neural networks [5]. In the work [5], estimates of the characteristics of memristive elements were obtained, at which they could be used for the hardware implementation of the corresponding neural network algorithms.
In this study, we simulate a two-layer fully connected pulse network with one layer of memristor elements (synapses). The 1T1R crossbar architecture is used, in which one transistor corresponds to each memristor. Thanks to this combination, it is possible to train the network at the hardware level using the Spike Timing Dependent Plasticity (STDP) method [6][7][8][9][10][11]. The memristive effect can manifest itself in different substances (mainly in different oxides), so it is important to understand how the characteristics of memristors will affect the parameters of the neural network.
The aim of this study is to simulate the process of a neuromorphic network functioning with memristive elements as synaptic weights based on different oxides and to study the possibilities of adapting the used model of a neuromorphic network to the case of using different types of memristors.
Several existing mathematical models of memristors are considered and their characteristics are compared with the experimental data on memristors based on titanium oxide (TiO 2 ) and hafnium oxide (HfO 2 ), respectively. In the third section, a mathematical model of a circuit design is formulated that implements a single-layer self-learning pulse neural network with memristive elements as synaptic weights. In the fourth section, the operation of five interconnected neurons with 320 synapses for two different oxides is numerically modeled. The main results of the study are formulated in the conclusion.

MEMRISTOR MATHEMATICAL MODELS
The memristive effect, as a rule, arises due to the movement of ions in an ultrathin dielectric layer when an electric field is applied. As applied to various oxides, the movement of oxygen vacancies and the formation/destruction of conducting filaments is often mentioned. Most of the known memristor models are formulated as a dynamic system with respect to the state of the memristor. The state parameter of a memristor is a quantity that corresponds to the position of the boundary separating the regions with low and high concentration of oxygen vacancies, the thickness of the conducting layer, or the thickness of the nonconducting barrier in which the tunneling current of electrons occurs. Depending on the law of changing the state parameter of the memristor, several mathematical models can be distinguished, in particular, the linear [12] and nonlinear drift [13], a model based on the Simmons barrier [14]. To limit the state variable, special window functions are introduced [15][16][17][18]. The experimental data show that a change in state occurs does not occur at any voltage, but starting from a certain threshold, and in relation to this, the threshold conditions are added to the model [19][20][21].
Several memristor models are considered. The first model is a variable-resistor thin-film memristor model based on the exponential dopant drift model [22]: (1) where x ∈ [0, 1] is the state variable; R on and R off are the minimal and maximal memristor resistance; I M , V M , and R are the current value of the current, voltage, and resistance of the memristor; V p and V n are the voltage values in which the state switches; μ v is the coefficient of alloying mobility; and D is the thickness of the semiconductor film.
The memristor's operation is simulated with the following parameter values: R on = 205 Ohm, R off = 2.13 kΩ, μ v = 6 × 10 -10 , V p = 0.65 V, V n = −0.87 V, D = 620 nm, x(0) = 0.1, t ∈ [0, 16] ms, and V M (t) is shown in Fig. 1b. Such a choice of parameters and waveforms V M (t) is due to obtaining characteristics of the memristor, similar to the experimental characteristics for titanium oxide, given in [12]. Figure 1a shows an experimental volt-ampere characteristic and a model one.
There is close agreement on the right side of the graphs and satisfactory agreement on the left side. Next, a model with a nonlinear voltage dependence is considered. In general, the equation describing the state of the memristor can be represented as follows: where x ∈ [0, 1] is the state variable; a is a constant determined by the properties of the material; V is the current voltage value; s is an odd integer; and f(x) is the window function used to approximate the nonlinear effects of the ion drift and boundary constraints. In this study, we use the memristor model of this class proposed in [23]: (2) where I and V are the current values of current and voltage; v thr is the activation voltage threshold; n, β, α M , χ, and γ are the adjustable parameters in the The memristor's operation is simulated with the following parameter values: n = 5, β = 7.069 × 10 -5 V, Fig. 2b. Some of the values correspond to the values in the original work [23], and some were selected for the best correspondence with the experimental data on hafnium oxide (HfO 2 ) given in the same article [23]. Here, there is a satisfactory agreement between the simulation results and experimental data.
An important feature of memristor elements is their possible imperfection [24], leading to an uncontrolled change in the conductivity level during the system's operation or initiation. Note that in this case, the memristor model can be described as a dynamic   system with uncertainties and appropriate methods can be used to study it [25,26].

MATHEMATICAL MODEL OF A NEUROMORPHIC NETWORK
Let us consider the operation of a circuit solution of a single-layer self-learning analog pulse neural network with memristive elements as synaptic weights (Fig. 3). The pulses arriving at the input V g open the corresponding transistors, which leads to the flow of currents through the memristors with their subsequent summation in neurons. The schematic model of a neuron is represented by a parallel RC circuit and abstract pulse generator G (Fig. 4). As soon as the value of the potential on the capacitor exceeds a certain threshold, its potential is reset, and the pulse generator gives an output signal V out and feedback signal V te . In addition, some small potential is constantly maintained in the feedback, which is necessary for the network to function in the normal mode.
The process of learning the network occurs according to the STDP rule (those synaptic connections that led to the activation of the neuron are strengthened, and others are weakened). This learning mechanism is implemented by feedback from neurons (V te ). At the moment of neuron activation, two pulses of the oppo-site sign arrive through the feedback channel with delays. If there is activity at the synapse and a positive feedback pulse arrives, then the conductivity value of the corresponding memristor increases, and if a negative feedback pulse arrives, then the memristor's conductivity decreases. Figure 5 illustrates the process of changing synaptic weights using the example of one synapse.
The network is trained in the following way: either arbitrary noise or a predetermined pattern is fed to the network input with equal probability. After some time, the network adapts to pattern recognition. In the case of several templates, the output layer will contain several neurons with additional connections (in Fig. 3 marked with letter α). When one neuron is activated, it suppresses the rest of the neurons (decreases the value of their potential). The templates are distributed among neurons during the learning process.
Let us formulate a complex mathematical model of a single-layer self-learning pulse neural network (Fig. 3). The mathematical model is given by the following relations [27]: and is supplemented by the initial conditions where n are the number of inputs; m is the number of neurons; is the current value of the voltage on the ith neural network input; is the current value of the feedback voltage of the jth neuron; is the current value of the output voltage of the jth neuron; τ j is the time elapsed since the last activation of the jth neuron; is the voltage across the capacitor of the jth neuron; R int and C int are the values of resistance and capacitance in the neurons; , , and are the values for the amplitude of the feedback pulses and the default voltage value; is the amplitude of the output pulse; V th is the level of the neuron activation voltage; R i,j is the value of the memristor resistance of the ith synapse of the jth neuron; x i,j is the state of the memristor of the ith synapse of the jth neuron, x i,j ∈ [0, 1]; τ r is the duration of the feedback signal after neuron activation; τ s is the duration of one pulse in the feedback signal, 2τ s < τ r ; τ out is the duration of one pulse at the network output; α is the suppression ratio; and is responsible for ensuring that the feedback pulses do not contribute to the accumulation of potential within the neuron; δ ij is the Kronecker symbol; δ(x) is the delta function; and is the Heaviside function. Ratios (3) set the memristor model. Function F X (I, v, x) determines the rate of change of the state variable depending on the current (I), voltage (v), and the current state (x). Function F R (x, v) determines the dependence of the memristor resistance on the state and the applied voltage. Ratio (4) specifies the model of a neuron, which is a parallel RC chain (Fig. 4) connected in series with a resistor (all synapse memristors at the neuron level can be considered as one resistor). Equation (5) implements the mechanism of the time counter after the last activation of the neuron. Once the tension on the capacitor reaches the threshold V th , variable τ j is reset to zero. The same happens in the equation (4): after activation of a neuron, the potential accumulated by it is reset, and in other neurons, it decreases in direct proportion to coefficient α. Equations (6) and (7) determine the shape of the pulses in the feedback and at the output of the neural network. At the initial moment of time, variable τ j is selected in such a way as to avoid the premature appearance of pulses in the feedback and at the output.  Note that the given mathematical model describes only one layer of the neural network. To simulate a multilayer network, we simply connect the output of the kth layer with input (k + 1) of the layer i = j. Thus, the scope of the described mathematical model is not limited to a single-layer network.
Voltage has a discrete distribution: = 0 V with probability 0.73 and = 2 V with probability 0.27. Figure 7 shows the process of adaptation of the synaptic weights to the recognized patterns. The color corresponds to the value of the state variable of the corresponding memristor: the darker the color the greater the conductivity; and the lighter the color the lower the conductivity. At the initial moment of time, all weights are initialized with random values, and gradually change during the operation of the network. From about the 700th epoch, patterns began to be seen, the Fig. 6. Recognizable patterns. Next, we consider the memristor model (2), which corresponds to hafnium oxide (HfO 2 ). The parameters of the mathematical model of the neural network are as follows: R int = 1 kΩ, C int = 45 μF, = 1.55 V, , = -1.6 V, = 10 mV, , V th = 2.5 mV, τ r = 15 ms, τ s = 1 ms, and τ out = 7.5 ms. The parameters were adjusted by analogy with the titanium oxidebased memristor model: = 0 V with a probability of 0.85 and = 2 V with a probability of 0.15. Figure 8 shows the change in the synaptic weights of the network during training.
Unlike the previous example, here the network took longer to train. Note that the given sets of neural network parameters are one of the possible sets and are not optimal in terms of the network learning rate. In the learning process, the patterns are distributed among the neurons in an arbitrary way; therefore, in the two examples given, the neurons responsible for the same pattern are different.

CONCLUSIONS
This study is devoted to the mathematical modeling of a self-learning neuromorphic network based on nanoscale memristive elements with a 1T1R crossbar architecture. Several mathematical models describing V memristors based on titanium oxide and hafnium oxide are considered. The characteristics of the models are compared with the experimental data. A complex mathematical model of an pulse neuromorphic network with a learning mechanism according to the STDP rule has been formulated. The operation of two neural networks with memristive elements as synaptic weights based on different oxides, consisting of five neurons with 320 synapses, has been modeled. The parameters of the neuromorphic network were adjusted depending on the oxide used, which indicates the versatility and flexibility of the used mathematical model. In the process, neural networks have successfully learned to recognize certain patterns.