Beyond VQA: Generating multi-word answers and rationales to visual questions

Dua, R. and Kancheti, S.S and Balasubramanian, V.N (2021) Beyond VQA: Generating multi-word answers and rationales to visual questions. In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 19 June 2021 through 25 June 2021, Virtual, Online.

Full text not available from this repository. (Request a copy)

Abstract

Visual Question Answering is a multi-modal task that aims to measure high-level visual understanding. Contemporary VQA models are restrictive in the sense that answers are obtained via classification over a limited vocabulary (in the case of open-ended VQA), or via classification over a set of multiple-choice-type answers. In this work, we present a completely generative formulation where a multi-word answer is generated for a visual query. To take this a step forward, we introduce a new task: ViQAR (Visual Question Answering and Reasoning), wherein a model must generate the complete answer and a rationale that seeks to justify the generated answer We propose an end-to-end architecture to solve this task and describe how to evaluate it. We show that our model generates strong answers and rationales through qualitative and quantitative evaluation, as well as through a human Turing Test.

[error in script]

IITH Creators:

IITH Creators	ORCiD
Balasubramanian, Vineeth N	https://orcid.org/0000-0003-2656-0375

Item Type:

Conference or Workshop Item (Paper)

Uncontrolled Keywords:

End to end; Multi-modal; Multi-word; Multiple choice; Qualitative evaluations, Quantitative evaluation, Question Answering, Turing tests, Visual query

Subjects:

Computer science

Divisions:

Department of Computer Science & Engineering

Depositing User:

Mrs Haseena VKKM

Date Deposited:

18 Nov 2021 04:53

Last Modified:

07 Mar 2022 11:02

URI:

http://raiithold.iith.ac.in/id/eprint/8982