Оцінка методу прунінгу SNIP на сучасній моделі детекції обличчя
dc.citation.epage | 22 | |
dc.citation.issue | 1 | |
dc.citation.journalTitle | Обчислювальні проблеми електротехніки | |
dc.citation.spage | 18 | |
dc.contributor.affiliation | National Technical University of Ukraine “Igor Sikorsky Kyiv Polytechnic Institute” | |
dc.contributor.author | Мельниченко, Артем | |
dc.contributor.author | Шалденко, Олексій | |
dc.contributor.author | Melnychenko, Artem | |
dc.contributor.author | Shaldenko, Oleksii | |
dc.coverage.placename | Львів | |
dc.coverage.placename | Lviv | |
dc.date.accessioned | 2024-04-11T09:15:11Z | |
dc.date.available | 2024-04-11T09:15:11Z | |
dc.date.created | 2023-02-28 | |
dc.date.issued | 2023-02-28 | |
dc.description.abstract | Завдяки швидкому розвитку машинного навчання та як наслідок глибокого навчання глибокі нейронні мережі досягли помітних результатів у різних сферах. Однак зі збільшенням точності навчених моделей нові архітектури нейронних мереж створюють нові виклики, оскільки потребують великої кількості обчислювальних потужностей для навчання та подальшого використання. Ця стаття має на меті переглянути підходи до зменшення обчислювальних потужностей та часу, потрібних для навчання нейронних мереж, оцінити та вдосконалити один із таких методів на моделі для детекції облич. Результати показали, що пропонований метод може усунути 69 % параметрів, втрачаючи лише 1,4 % точності. Його можна додатково покращити, зменшивши втрату точності до 0,7 %, вилучивши контекстні модулі мережі із методу. | |
dc.description.abstract | With rapid development of machine learning and subsequently deep learning, deep neural networks achieved remarkable results in solving various tasks. However, with increasing the accuracy of trained models, new architectures of neural networks present new challenges as they require significant amount of computing power for training and inference. This paper aims to review existing approaches to reducing computational power and training time of the neural network, evaluate and improve one of existing pruning methods for a face detection model. Obtained results show that the presented method can eliminate 69% of parameters while accuracy being declined only by 1.4%, which can be further improved to 0.7% by excluding context network modules from the pruning method. | |
dc.format.extent | 18-22 | |
dc.format.pages | 5 | |
dc.identifier.citation | Мельниченко А. Оцінка методу прунінгу SNIP на сучасній моделі детекції обличчя / Артем Мельниченко, Олексій Шалденко // Обчислювальні проблеми електротехніки. — Львів : Видавництво Львівської політехніки, 2023. — Том 13. — № 1. — С. 18–22. | |
dc.identifier.citationen | Melnychenko A. Evaluating Snip Pruning Method on the State-of-the-Art Face Detection Model / Artem Melnychenko, Oleksii Shaldenko // Computational Problems of Electrical Engineering. — Lviv : Lviv Politechnic Publishing House, 2023. — Vol 13. — No 1. — P. 18–22. | |
dc.identifier.doi | doi.org/10.23939/jcpee2023.01.018 | |
dc.identifier.issn | 2224-0977 | |
dc.identifier.uri | https://ena.lpnu.ua/handle/ntb/61717 | |
dc.language.iso | uk | |
dc.publisher | Видавництво Львівської політехніки | |
dc.publisher | Lviv Politechnic Publishing House | |
dc.relation.ispartof | Обчислювальні проблеми електротехніки, 1 (13), 2023 | |
dc.relation.ispartof | Computational Problems of Electrical Engineering, 1 (13), 2023 | |
dc.relation.references | [1] G. Hinton et al., “Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research Groups”, in IEEE Signal Processing Magazine, Vol. 29, No. 6, pp. 82–97, Nov. 2012. | |
dc.relation.references | [2] K. He, X. Zhang, S. Ren and J. Sun, “Deep Residual Learning for Image Recognition”, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 2016, pp. 770-778. | |
dc.relation.references | [3] K. Zhang, Z. Zhang, Z. Li and Y. Qiao, “Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks”, in IEEE Signal Processing Letters, Vol. 23, No. 10, pp. 1499–1503, Oct. 2016. | |
dc.relation.references | [4] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding”, Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Vol. 1 (Long and Short Papers), pp. 4171–4186, Minneapolis, Minnesota. | |
dc.relation.references | [5] Brown, T et al. “Language models are few-shot learners”, Advances in neural information processing systems, 33, pp. 1877–1901. | |
dc.relation.references | [6] Schwartz, Roy, Jesse Dodge, Noah Smith and Oren Etzioni. “Green AI.” Communications of the ACM 63, 2019, pp. 54–63. | |
dc.relation.references | [7] Ben Taylor, Vicent Sanz Marco, Willy Wolff, Yehia Elkhatib, and Zheng Wang. “Adaptive deep learning model selection on embedded systems”, in Proc. 19th ACM SIGPLAN/SIGBED International Conference on Languages, Compilers, and Tools for Embedded Systems, New York, USA, pp. 31–43, 2018. | |
dc.relation.references | [8] Han, Song, Huizi Mao and William J. Dally. “Deep Compression: Compressing Deep Neural Network with Pruning, Trained Quantization and Huffman Coding”, arXiv: Computer Vision and Pattern Recognition, 2015. | |
dc.relation.references | [9] S. Teerapittayanon, B. McDanel and H. T. Kung, “Distributed Deep Neural Networks Over the Cloud, the Edge and End Devices”, 2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS), Atlanta, USA, pp. 328–339, 2017. | |
dc.relation.references | [10] Misha Denil, Babak Shakibi, Laurent Dinh, Marc'Aurelio Ranzato, and Nando de Freitas. “Predicting parameters in deep learning”, in Proc. 26th International Conference on Neural Information Processing Systems, vol. 2 (NIPS'13). Curran Associates Inc., Red Hook, USA, pp. 2148–2156, 2013. | |
dc.relation.references | [11] Max Jaderberg, Andrea Vedaldi, and Andrew Zisserman. “Speeding up Convolutional Neural Networks with Low Rank Expansions”, In Proceedings of the British Machine Vision Conference. BMVA Press, September 2014. | |
dc.relation.references | [12] Novikov, A., Podoprikhin, D., Osokin, A. and Vetrov, D.P.. “Tensorizing neural networks”, Advances in neural information processing systems, 28, 2015. | |
dc.relation.references | [13] Song Han, Jeff Pool, John Tran, and William J. Dally. “Learning both weights and connections for efficient neural networks”, In Proceedings of the 28th International Conference on Neural Information Processing Systems, Vol. 1 (NIPS'15), MIT Press, Cambridge, USA, pp. 1135–1143, 2015. | |
dc.relation.references | [14] S. Park, J. Lee, S. Mo and J. Shin, , “Lookahead: A far-sighted alternative of magnitude-based pruning”, arXiv preprint arXiv:2002.04809, 2020. | |
dc.relation.references | [15] B. Hassibi and D. G. Stork. “Second order derivatives for network pruning: optimal brain surgeon”, in Proc. 5th International Conference on Neural Information Processing Systems (NIPS'92), Morgan Kaufmann Publishers Inc., San Francisco, USA, pp. 164–171, 1992. | |
dc.relation.references | [16] J. Frankle, and M. Carbin, “The lottery ticket hypothesis: Finding sparse, trainable neural networks”, in Proc. 7th International Conference on Learning Representations, New Orleans, USA, May 6–9, 2019. | |
dc.relation.references | [17] N. Lee, T. Ajanthan and P.H. Torr, “The lottery ticket hypothesis: Finding sparse, trainable neural networks”, in Proc. 7th International Conference on Learning Representations, New Orleans, USA, May 6–9, 2019. | |
dc.relation.references | [18] J. Deng, J. Guo, E. Ververas, I. Kotsia, Stefanos Zafeiriou, “RetinaFace: Single-stage Dense Face Localisation in the Wild“,in Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5203–5212, 2020. | |
dc.relation.referencesen | [1] G. Hinton et al., "Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research Groups", in IEEE Signal Processing Magazine, Vol. 29, No. 6, pp. 82–97, Nov. 2012. | |
dc.relation.referencesen | [2] K. He, X. Zhang, S. Ren and J. Sun, "Deep Residual Learning for Image Recognition", 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 2016, pp. 770-778. | |
dc.relation.referencesen | [3] K. Zhang, Z. Zhang, Z. Li and Y. Qiao, "Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks", in IEEE Signal Processing Letters, Vol. 23, No. 10, pp. 1499–1503, Oct. 2016. | |
dc.relation.referencesen | [4] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding", Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Vol. 1 (Long and Short Papers), pp. 4171–4186, Minneapolis, Minnesota. | |
dc.relation.referencesen | [5] Brown, T et al. "Language models are few-shot learners", Advances in neural information processing systems, 33, pp. 1877–1901. | |
dc.relation.referencesen | [6] Schwartz, Roy, Jesse Dodge, Noah Smith and Oren Etzioni. "Green AI." Communications of the ACM 63, 2019, pp. 54–63. | |
dc.relation.referencesen | [7] Ben Taylor, Vicent Sanz Marco, Willy Wolff, Yehia Elkhatib, and Zheng Wang. "Adaptive deep learning model selection on embedded systems", in Proc. 19th ACM SIGPLAN/SIGBED International Conference on Languages, Compilers, and Tools for Embedded Systems, New York, USA, pp. 31–43, 2018. | |
dc.relation.referencesen | [8] Han, Song, Huizi Mao and William J. Dally. "Deep Compression: Compressing Deep Neural Network with Pruning, Trained Quantization and Huffman Coding", arXiv: Computer Vision and Pattern Recognition, 2015. | |
dc.relation.referencesen | [9] S. Teerapittayanon, B. McDanel and H. T. Kung, "Distributed Deep Neural Networks Over the Cloud, the Edge and End Devices", 2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS), Atlanta, USA, pp. 328–339, 2017. | |
dc.relation.referencesen | [10] Misha Denil, Babak Shakibi, Laurent Dinh, Marc'Aurelio Ranzato, and Nando de Freitas. "Predicting parameters in deep learning", in Proc. 26th International Conference on Neural Information Processing Systems, vol. 2 (NIPS'13). Curran Associates Inc., Red Hook, USA, pp. 2148–2156, 2013. | |
dc.relation.referencesen | [11] Max Jaderberg, Andrea Vedaldi, and Andrew Zisserman. "Speeding up Convolutional Neural Networks with Low Rank Expansions", In Proceedings of the British Machine Vision Conference. BMVA Press, September 2014. | |
dc.relation.referencesen | [12] Novikov, A., Podoprikhin, D., Osokin, A. and Vetrov, D.P.. "Tensorizing neural networks", Advances in neural information processing systems, 28, 2015. | |
dc.relation.referencesen | [13] Song Han, Jeff Pool, John Tran, and William J. Dally. "Learning both weights and connections for efficient neural networks", In Proceedings of the 28th International Conference on Neural Information Processing Systems, Vol. 1 (NIPS'15), MIT Press, Cambridge, USA, pp. 1135–1143, 2015. | |
dc.relation.referencesen | [14] S. Park, J. Lee, S. Mo and J. Shin, , "Lookahead: A far-sighted alternative of magnitude-based pruning", arXiv preprint arXiv:2002.04809, 2020. | |
dc.relation.referencesen | [15] B. Hassibi and D. G. Stork. "Second order derivatives for network pruning: optimal brain surgeon", in Proc. 5th International Conference on Neural Information Processing Systems (NIPS'92), Morgan Kaufmann Publishers Inc., San Francisco, USA, pp. 164–171, 1992. | |
dc.relation.referencesen | [16] J. Frankle, and M. Carbin, "The lottery ticket hypothesis: Finding sparse, trainable neural networks", in Proc. 7th International Conference on Learning Representations, New Orleans, USA, May 6–9, 2019. | |
dc.relation.referencesen | [17] N. Lee, T. Ajanthan and P.H. Torr, "The lottery ticket hypothesis: Finding sparse, trainable neural networks", in Proc. 7th International Conference on Learning Representations, New Orleans, USA, May 6–9, 2019. | |
dc.relation.referencesen | [18] J. Deng, J. Guo, E. Ververas, I. Kotsia, Stefanos Zafeiriou, "RetinaFace: Single-stage Dense Face Localisation in the Wild",in Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5203–5212, 2020. | |
dc.rights.holder | © Національний університет “Львівська політехніка”, 2023 | |
dc.subject | pruning | |
dc.subject | deep neural networks | |
dc.subject | inference | |
dc.subject | optimization | |
dc.subject | face detection | |
dc.title | Оцінка методу прунінгу SNIP на сучасній моделі детекції обличчя | |
dc.title.alternative | Evaluating Snip Pruning Method on the State-of-the-Art Face Detection Model | |
dc.type | Article |
Files
License bundle
1 - 1 of 1