Simulation modeling of an analog impulse neural network based on a memristor crossbar using parallel computing technologies

The work is devoted to the issues of simulation modeling of an analog impulse neural network based on memristive elements within the framework of the problem of pattern recognition. Simulation modeling allows you to configure the network at the level of a mathematical model, and subsequently use the obtained parameters directly in the process of operation. The network model is given as a dynamic system, which can consist of tens and hundreds of thousands of ordinary differential equations. Naturally, there is a need for an efficient and parallel implementation of an appropriate simulation model. OpenMP (Open Multi-Processing) is used as a technology for parallelizing calculations, since it allows you to easily create multi-threaded applications in various programming languages. The efficiency of parallelization is evaluated on the problem of modeling the process of learning the network to recognize a set of five images of size 128 by 128 pixels, which leads to the solution of about 80 thousand differential equations. On this problem, more than a sixfold acceleration of calculations was obtained.
According to experimental data, the character of memristor operation is stochastic, as evidenced by the spread in the current-voltage characteristics during switching between high-resistance and low-resistance states. To take this feature into account, a memristor model with interval parameters is used, which gives upper and lower limits on the quantities of interest, and encloses the experimental curves in corridors. When modeling the operation of the entire analog self-learning impulse neural network, each epoch of training, the parameters of the memristors are set randomly from the selected intervals. This approach makes it possible to do without the use of a stochastic mathematical apparatus, thereby further reducing computational costs.

1. Merolla P.A., Arthur J.V., Alvarez-Icaza R., Cassidy A.S., Sawada J., Akopyan F., Jackson B.L., Imam N., Guo Ch., Nakamura Y., Brezzo B., Vo I., Esser S.K., Appuswamy R., Taba B., Amir A., Flickner M.D., Risk W.P., Manohar R., Modha Dh.S. Artificial brains. A million spiking-neuron integrated circuit with a scalable communication network and interface. Science. 2014; 345(6197): 668—673. https://doi.org/10.1126/science.1254642

2. Wong H.-S.P., Lee H.-Y., Yu Sh., Chen Y.-Sh., Wu Y., Chen P.-Sh., Lee B., Chen F.T., Tsai M.-J. Metal-oxide RRAM. Proceedings of the IEEE. 2012; 100(6): 1951—1970. https://doi.org/10.1109/JPROC.2012.2190369

3. Yang J.J., Strukov D.B., Stewart D.R. Memristive devices for computing. Nature Nanotechnology. 2013; 8(1): 13—24. https://doi.org/10.1038/nnano.2012.240

4. Li C., Hu M., Li Y., Ge N., Montgomery E., Zhang J., Song W., Dávila N., Graves C.E., Li Zh., Strachan J.P., Lin P., Wang Zh., Barnell M., Wu Q., Williams R.S., Yang J.J., Xia Q. Analogue signal and image processing with large memristor crossbars. Nature Electronics. 2018; 1: 52—59. https://doi.org/10.1038/s41928-017-0002-z

5. Morozov A.Yu., Abgaryan K.K., Reviznikov D.L. Mathematical model of a neuromorphic network based on memristive elements. Chaos, Solitons & Fractals. 2021; 143: 110548. https://doi.org/10.1016/j.chaos.2020.110548

6. Morozov A.Yu., Abgaryan K.K., Reviznikov D.L. Mathematical modeling of a self-learning neuromorphic network based on nanosized memristive elements with 1T1R crossbar architecture. Izvestiya Vysshikh Uchebnykh Zavedenii. Materialy Elektronnoi Tekhniki = Materials of Electronics Engineering. 2020; 23(3): 186—195. (In Russ.). https://doi.org/10.17073/1609-3577-2020-3-186-195

7. Diehl P., Cook M. Unsupervised learning of digit recognition using spike-timing-dependent plasticity. Frontiers in Computational Neuroscience. 2015; 9: 99. https://doi.org/10.3389/fncom.2015.00099

8. Ambrogio S., Balatti S., Milo V., Carboni R., Wang Zh., Calderoni A., Ramaswamy N., Ielmini D. Neuromorphic learning and recognition with one-transistor-one-resistor synapses and bistable metal oxide RRAM. IEEE Transactions on Electron Devices. 2016; 63(4): 1508—1515. https://doi.org/10.1109/TED.2016.2526647

9. Guo Y., Wu H., Gao B., Qian H. Unsupervised learning on resistive memory array based spiking neural networks. Frontiers in Neuroscience. 2019; 13: 812. https://doi.org/10.3389/fnins.2019.00812

10. OpenMP. https://www.openmp.org/ (accessed on 02.04.2021).

11. PVS-Studio is a static analyzer on guard of code quality, security (SAST), and code safety. https://pvs-studio.com/ru/a/0057/ (accessed on 02.04.2021).

12. Rodriguez-Fernandez A., Cagli C., Perniola L., Miranda E., Suñé J. Characterization of HfO2-based devices with indication of second order memristor effects. Microelectronic Engineering. 2018; 195: 101—106. https://doi.org/10.1016/j.mee.2018.04.006

13. Teplov G.S., Gornev E.S. Multilevel bipolar memristor model considering deviations of switching parameters in the Verilog-A language. Russian Microelectronics. 2019; 48(3): 131—142. (In Russ.). https://doi.org/10.1134/S0544126919030104

14. Vasil’ev V.A., Chernov P.S. Mathematical modeling of memristor in the presence of noise. Mathematical Models and Computer Simulations. 2014; 26(1): 122—132. (In Russ.)

15. Morozov A.Yu., Abgaryan K.K., Reviznikov D.L. Simulation of the neuromorphic network operation taking into account stochastic effects. In: Short paper proceed. of the VI Inter. conf. on information technologies and high-performance computing (ITHPC 2021). Khabarovsk, September 14–16, 2021. CEUR Workshop Proceedings; 2021. P. 84—91.

16. Morozov A.Y., Abgaryan K.K., Reviznikov D.L. Mathematical modeling of an analogue self-learning neural network based on memristive elements taking into account stochastic switching dynamics. Nanobiotechnology Reports. 2021; 16(6): 767—776. (In Russ.). https://doi.org/10.1134/S1992722321060157

17. Morozov A.Yu., Abgaryan K.K., Reviznikov D.L. Interval model of a memristor crossbar network. Physica Status Solidi (B). 2022; 259(11): 2200150. https://doi.org/10.1002/pssb.202200150

18. Morozov A.Yu., Reviznikov D.L. Interval approach to solving problems of parametric identification of dynamical systems. Differential Equations. 2022; 58(7): 962—976. (In Russ.). https://doi.org/10.31857/S0374064122070081

19. Mladenov V. Analysis of memory matrices with HfO2 memristors in a PSpice environment. Electronics. 2019; 8(4): 383. https://doi.org/10.3390/electronics8040383

20. Zheng G., Mohanty S.P., Kougianos E., Okobiah O. Polynomial metamodel integrated Verilog-AMS for memristor-based mixed-signal system design. Proceed. IEEE 56th Inter. midwest symposium on circuits and systems (MWSCAS). Columbus, OH, August 04, 2013. Demand Purchase at Partner; 2013. P. 916—919. https://doi.org/10.1109/MWSCAS.2013.6674799

21. Martyshov M.N., Emelyanov A.V., Demin V.A., Nikiruy K.E., Minnekhanov A.A., Nikolaev S.N., Taldenkov A.N., Ovcharov A.V., Presnyakov M.Yu., Sitnikov A.V., Vasiliev A.L., Forsh P.A., Granovsky A.B., Kashkarov P.K., Kovalchuk M.V., Rylkov V.V. Multifilamentary character of anticorrelated capacitive and resistive switching in memristive structures based on (Co-Fe-B)x(LiNbO3)100-x nanocomposite. Physical Review Applied. 2020; 14(3): 034016. https://doi.org/10.1103/PhysRevApplied.14.034016

22. Rylkov V., Nikolaev S., Demin V., Emelyanov A.V., Nikiruy K.E., Levanov V.A., Presnyakov M.Y., Taldenkov A.N., Vasiliev A.L., Chernoglazov K.Y., Tugushev V.V., Sitnikov A.V., Kalinin Y.E., Bugaev A.S., Granovsky A.B., Vedeneev A.S. Transport, magnetic, and memristive properties of a nanogranular (CoFeB)x(LiNbOy)100-x composite material. Journal of Experimental and Theoretical Physics. 2018; 126(3): 353—367. https://doi.org/10.1134/S1063776118020152

23. Photo hosting Pinterest. (In Russ.). https://ru.pinterest.com/pin/351912463120005/ (accessed on 02.09.2022).