University of Twente Student Theses

Login

Sparse artificial neural networks : Adaptive performance-based connectivity inspired by human-brain processes

Lapshyna, V. (2020) Sparse artificial neural networks : Adaptive performance-based connectivity inspired by human-brain processes.

[img] PDF
675kB
Abstract:Artificial Neural Networks are powerful machine learning systems. However, a high number of weights close to zero make networks unnecessary large and heavy. Sparse models remove redundant weights, aiming to decrease the number of parameters with minimal loss in accuracy. Sparse Evolutionary Training procedure adaptively evolves weights of the Artificial Neural Network topology. This technique proves to remove a vast number of weights and achieve higher accuracy than its non-evolutionary or densely connected counterparts, although the connection addition and removal follows a relatively simple algorithm. Inspired by the synaptic pruning in the human brain, we propose an advanced approach for weight evolution in the Sparse Evolutionary Training algorithm. We suggest gradually removing connections during the training phase as the accuracy increases. We show that the number of parameters can be significantly reduced with almost no loss in accuracy and negligible additional computational complexity. We demonstrate the performance of the algorithm on the Multilayer Perceptron trained on benchmark image and tabular datasets. This research contributes to the understanding of Sparse Artificial Neural Networks and makes a step towards more efficient models.
Item Type:Essay (Bachelor)
Faculty:EEMCS: Electrical Engineering, Mathematics and Computer Science
Subject:54 computer science
Programme:Business & IT BSc (56066)
Link to this item:https://purl.utwente.nl/essays/80559
Export this item as:BibTeX
EndNote
HTML Citation
Reference Manager

 

Repository Staff Only: item control page