Efficient Sparse Neural Network Training
| dc.contributor.advisor | Choi, Jee | |
| dc.contributor.author | Shvedov, Konstantin | |
| dc.date.accessioned | 2022-10-04T19:43:35Z | |
| dc.date.available | 2022-10-04T19:43:35Z | |
| dc.date.issued | 2022-10-04 | |
| dc.description.abstract | Developments in neural networks have led to advanced models requiring large amounts of training time and resources. To reduce the environmental impact and to decrease the training times of models, acceleration techniques have been developed. One method is neural network pruning, which removes insignificant weights and preempts the generation of sparse models. This paper attempts to improve and explore a method of training sparse neural networks efficiently processing only non-zero values using optimized just-in-time kernels from the Libsxmm library while randomly pruning network layers at initialization. The algorithms explored within this paper show a proof of concept and the possibility of improving training time beyond what the highly optimized PyTorch library is currently capable of. Through the work in this paper algorithm's processing times are sped up over 100-fold. Further, this work provides additional evidence that advanced pruning algorithms and other improvements can significantly reduce training times and resources. | en_US |
| dc.identifier.uri | https://hdl.handle.net/1794/27618 | |
| dc.language.iso | en_US | |
| dc.publisher | University of Oregon | |
| dc.rights | All Rights Reserved. | |
| dc.subject | JIT | en_US |
| dc.subject | Matrices | en_US |
| dc.subject | Neural Networks | en_US |
| dc.subject | Pruning | en_US |
| dc.subject | Sparse | en_US |
| dc.title | Efficient Sparse Neural Network Training | |
| dc.type | Electronic Thesis or Dissertation | |
| thesis.degree.discipline | Department of Computer and Information Science | |
| thesis.degree.grantor | University of Oregon | |
| thesis.degree.level | masters | |
| thesis.degree.name | M.S. |
Files
Original bundle
1 - 1 of 1
Loading...
- Name:
- Shvedov_oregon_0171N_13358.pdf
- Size:
- 431.1 KB
- Format:
- Adobe Portable Document Format