E2GC: Energy-efficient Group Convolution in Deep Neural Networks

Jha, Nandan Kumar and Saini, Rajat and Nag, Subhrajit and Mittal, Sparsh (2020) E2GC: Energy-efficient Group Convolution in Deep Neural Networks. In: Proceedings - 33rd International Conference on VLSI Design, VLSID 2020 - Held concurrently with 19th International Conference on Embedded Systems, 4 January -8 January 2020.

Full text not available from this repository. (Request a copy)

Abstract

The number of groups (g) in group convolution (GConv) is selected to boost the predictive performance of deep neural networks (DNNs) in a compute and parameter efficient manner. However, we show that naive selection of g in GConv creates an imbalance between the computational complexity and degree of data reuse, which leads to suboptimal energy efficiency in DNNs. We devise an optimum group size model, which enables a balance between computational cost and data movement cost, thus, optimize the energy-efficiency of DNNs. Based on the insights from this model, we propose an 'energyefficient group convolution' (E2GC) module where, unlike the previous implementations of GConv, the group size (G) remains constant. Further, to demonstrate the efficacy of the E2GC module, we incorporate this module in the design of MobileNet-V1 and ResNeXt-50 and perform experiments on two GPUs, P100 and P4000. We show that, at comparable computational complexity, DNNs with constant group size (E2GC) are more energy-efficient than DNNs with a fixed number of groups (Fg GC). For example, on P100 GPU, the energy-efficiency of MobileNet-V1 and ResNeXt-50 is increased by 10.8% and 4.73% (respectively) when E2GC modules substitute the Fg GC modules in both the DNNs. Furthermore, through our extensive experimentation with ImageNet-1K and Food-101 image classification datasets, we show that the E2GC module enables a trade-off between generalization ability and representational power of DNN. Thus, the predictive performance of DNNs can be optimized by selecting an appropriate G. The code and trained models are available at https://github.com/iithcandle/E2GC-re1ease.

[error in script]

IITH Creators:

IITH Creators	ORCiD
Jha, Nandan Kumar	UNSPECIFIED
Saini, Rajat	UNSPECIFIED
Nag, Subhrajut	UNSPECIFIED
Mittal, Sparsh	http://orcid.org/0000-0002-2908-993X

Item Type:

Conference or Workshop Item (Paper)

Uncontrolled Keywords:

Classification datasets; Computational costs; Data movements; Energy efficient; Fixed numbers; Generalization ability; Group size; Predictive performance;Classification (of information); Complex networks; Computational complexity; Convolution; Deep neural networks; Economic and social effects; Embedded systems; Energy efficiency; Program processors; VLSI circuits

Subjects:

Physics > Mechanical and aerospace
Physics > Mechanical and aerospace > Transportation Science & Technology
Physics

Divisions:

Department of Mechanical & Aerospace Engineering

Depositing User:

. LibTrainee 2021

Date Deposited:

24 Jun 2021 09:21

Last Modified:

24 Jun 2021 09:21

URI:

http://raiithold.iith.ac.in/id/eprint/7995