Pruning is a model compression method that removes redundant parameters and accelerates the inference speed of deep neural networks while maintaining accuracy. Most available pruning methods impose various conditions on parameters or features directly. In this talk, we introduce a simple and effective regularization strategy to improve the structured sparsity and structured pruning in DNNs from a new perspective of evolution of features. In particular, we consider the trajectories connecting features of adjacent hidden layers, namely feature flow. We propose feature flow regularization (FFR) to penalize the length and the total absolute curvature of the trajectories, which implicitly increases the structured sparsity of the parameters. The principle behind FFR is that short and straight trajectories will lead to an efficient network that avoids redundant parameters. We provide experiment results on CIFAR-10 and ImageNet datasets which show that FFR improves structured sparsity and achieves pruning results comparable to or even better than those state-of-the-art methods.

5月4日
9:00am - 10:00am
地點
https://hkust.zoom.us/j/93243000376 (Passcode: hkust)
講者/表演者
Miss Yue WU
HKUST
主辦單位
Department of Mathematics
聯絡方法
付款詳情
對象
Alumni, Faculty and staff, PG students, UG students
語言
英語
其他活動
10月10日
研討會, 演講, 講座
IAS / School of Science Joint Lecture - Use of Large Animal Models to Investigate Brain Diseases
Abstract Genetically modified animal models have been extensively used to investigate the pathogenesis of age-dependent neurodegenerative diseases, such as Alzheimer (AD), Parkinson (PD), Hunti...
7月14日
研討會, 演講, 講座
IAS / School of Science Joint Lecture - Boron Clusters
Abstract The study of carbon clusters led to the discoveries of fullerenes, carbon nanotubes, and graphene. Are there other elements that can form similar nanostructures? To answer this questio...