E2GC: Energy-efficient Group Convolution in Deep Neural Networks

Jha, Nandan Kumar and Saini, Rajat and Nag, Subhrajit and Mittal, Sparsh (2020) E2GC: Energy-efficient Group Convolution in Deep Neural Networks. In: Proceedings - 33rd International Conference on VLSI Design, VLSID 2020 - Held concurrently with 19th International Conference on Embedded Systems, 4 January -8 January 2020.

Full text not available from this repository. (Request a copy)

Abstract

The number of groups (g) in group convolution (GConv) is selected to boost the predictive performance of deep neural networks (DNNs) in a compute and parameter efficient manner. However, we show that naive selection of g in GConv creates an imbalance between the computational complexity and degree of data reuse, which leads to suboptimal energy efficiency in DNNs. We devise an optimum group size model, which enables a balance between computational cost and data movement cost, thus, optimize the energy-efficiency of DNNs. Based on the insights from this model, we propose an 'energyefficient group convolution' (E2GC) module where, unlike the previous implementations of GConv, the group size (G) remains constant. Further, to demonstrate the efficacy of the E2GC module, we incorporate this module in the design of MobileNet-V1 and ResNeXt-50 and perform experiments on two GPUs, P100 and P4000. We show that, at comparable computational complexity, DNNs with constant group size (E2GC) are more energy-efficient than DNNs with a fixed number of groups (Fg GC). For example, on P100 GPU, the energy-efficiency of MobileNet-V1 and ResNeXt-50 is increased by 10.8% and 4.73% (respectively) when E2GC modules substitute the Fg GC modules in both the DNNs. Furthermore, through our extensive experimentation with ImageNet-1K and Food-101 image classification datasets, we show that the E2GC module enables a trade-off between generalization ability and representational power of DNN. Thus, the predictive performance of DNNs can be optimized by selecting an appropriate G. The code and trained models are available at https://github.com/iithcandle/E2GC-re1ease.

[error in script]
IITH Creators:
IITH CreatorsORCiD
Jha, Nandan KumarUNSPECIFIED
Saini, RajatUNSPECIFIED
Nag, SubhrajutUNSPECIFIED
Mittal, Sparshhttp://orcid.org/0000-0002-2908-993X
Item Type: Conference or Workshop Item (Paper)
Uncontrolled Keywords: Classification datasets; Computational costs; Data movements; Energy efficient; Fixed numbers; Generalization ability; Group size; Predictive performance;Classification (of information); Complex networks; Computational complexity; Convolution; Deep neural networks; Economic and social effects; Embedded systems; Energy efficiency; Program processors; VLSI circuits
Subjects: Physics > Mechanical and aerospace
Physics > Mechanical and aerospace > Transportation Science & Technology
Physics
Divisions: Department of Mechanical & Aerospace Engineering
Depositing User: . LibTrainee 2021
Date Deposited: 24 Jun 2021 09:21
Last Modified: 24 Jun 2021 09:21
URI: http://raiithold.iith.ac.in/id/eprint/7995
Publisher URL: http://doi.org/10.1109/VLSID49098.2020.00044
Related URLs:

Actions (login required)

View Item View Item
Statistics for RAIITH ePrint 7995 Statistics for this ePrint Item