Bayesian Generative Adversarial Nets with Dropout Inference

Palakkadavath, Ragja and Srijith, P K (2020) Bayesian Generative Adversarial Nets with Dropout Inference. In: 3rd ACM India Joint International Conference on Data Science and Management of Data, CODS-COMAD 2021, 2 January 2021through 4 January 2021, Virtual, Online.

Text
ACM.pdf - Published Version
Restricted to Registered users only
Download (2MB) | Request a copy

Abstract

Generative adversarial networks are one of the most popular approaches to generate new data from complex high-dimensional data distributions. They have revolutionized the area of generative models by creating quality samples that highly resemble the true data distribution. However, these samples often cover only few high density areas of the true data distribution. As some of the modes are missing in the generated data, this issue is referred to as mode collapse. Bayesian GANs (BGANs) can address this to a great extend by considering Bayesian learning principles. Instead of learning point estimates of parameters in the network, BGANs learn a probability distribution over these parameters and make use of the posterior distribution over parameters to make prediction. As these models are huge neural networks, analytical inference is not feasible due to the intractable likelihood and evidence terms. Hence, BGANs perform an approximate inference based on stochastic gradient Hamiltonian Monte Carlo (SGHMC) sampling which is computationally expensive and displays convergence problems. We propose a simple and effective Bayesian GAN model based on Monte Carlo dropout based inference (BDGAN). We establish theoretical connection between variational inference in Bayesian GANs and Monte Carlo dropout in GANs. The effectiveness of the proposed model in overcoming mode collapse is demonstrated on various synthetic and real-world data sets. Additionally, we analyse the training time and memory usage to show case the proposed method's advantages over Bayesian GAN. © 2021 ACM.

[error in script]

IITH Creators: