Submodular Batch Selection for Training Deep Neural Networks

Joseph, K J and Vamsi, Teja R and Singh, Krishnakant and Balasubramanian, Vineeth N (2019) Submodular Batch Selection for Training Deep Neural Networks. arXiv.org.

[img]
Preview
Text
1906.08771.pdf

Download (845kB) | Preview

Abstract

Mini-batch gradient descent based methods are the de facto algorithms for training neural network architectures today. We introduce a mini-batch selection strategy based on submodular function maximization. Our novel submodular formulation captures the informativeness of each sample and diversity of the whole subset. We design an efficient, greedy algorithm which can give high-quality solutions to this NP-hard combinatorial optimization problem. Our extensive experiments on standard datasets show that the deep models trained using the proposed batch selection strategy provide better generalization than Stochastic Gradient Descent as well as a popular baseline sampling strategy across different learning rates, batch sizes, and distance metrics.

[error in script]
IITH Creators:
IITH CreatorsORCiD
Balasubramanian, Vineeth NUNSPECIFIED
Item Type: Article
Subjects: Computer science
Divisions: Department of Computer Science & Engineering
Depositing User: Team Library
Date Deposited: 28 Jun 2019 04:10
Last Modified: 28 Jun 2019 04:10
URI: http://raiithold.iith.ac.in/id/eprint/5581
Publisher URL:
Related URLs:

Actions (login required)

View Item View Item
Statistics for RAIITH ePrint 5581 Statistics for this ePrint Item