Sync-DRAW: Automatic Video Generation using Deep Recurrent Attentive Architectures

Mittal, Gaurav and Marwah, Tanya and Balasubramanian, Vineeth N (2017) Sync-DRAW: Automatic Video Generation using Deep Recurrent Attentive Architectures. arXiv. pp. 1-9.

Preview

Text (arXiv copy)
Sync-DRAW.pdf - Accepted Version
Download (5MB) | Preview

Abstract

is paper introduces a novel approach for generating videos called Synchronized Deep Recurrent A�entive Writer (Sync-DRAW). Sync- DRAW can also perform text-to-video generation which, to the best of our knowledge, makes it the �rst approach of its kind. It com- bines a Variational Autoencoder (VAE) with a Recurrent A�ention Mechanism in a novel manner to create a temporally dependent sequence of frames that are gradually formed over time. �e recur- rent a�ention mechanism in Sync-DRAW a�ends to each individual frame of the video in sychronization, while the VAE learns a latent distribution for the entire video at the global level. Our experiments with Bouncing MNIST, KTH and UCF-101 suggest that Sync-DRAW is e�cient in learning the spatial and temporal information of the videos and generates frames with high structural integrity, and can generate videos from simple captions on these datasets.

[error in script]

IITH Creators: