Video Quality Prediction Using Voxel-Wise fMRI Models of the Visual Cortex

Mahankali, Naga Sailaja and Channappayya, Sumohana S. (2021) Video Quality Prediction Using Voxel-Wise fMRI Models of the Visual Cortex. In: 2021 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2021, 6 June 2021 through 11 June 2021, Virtual,Toronto.

Text
ICASSP.pdf - Accepted Version
Restricted to Registered users only
Download (927kB) | Request a copy

Abstract

In this work, we address the problem of full-reference video quality prediction. To address this problem, we rely on deep learning based spatio-temporal representations of natural videos. Specifically, we use feature representations derived from a per-voxel deep learning regression model. This model predicts the functional Magnetic Resonance Imaging (fMRI) responses of the visual cortical regions to natural video stimuli. We construct a rudimentary full-reference spatio-temporal quality feature that is simply the L1-norm of the error between the voxel model’s response to the reference and test video stimuli. This feature is shown to correlate well with subjective quality scores. Additionally, we rely on the Multi-Scale Structural Similarity (MS-SSIM) index as the spatial quality feature. We show that the combination of the proposed spatio-temporal feature and the spatial (MS-SSIM) feature delivers competitive performance for both Quality of Experience (QoE) prediction and Video Quality Assessment (VQA) tasks. This finding not only provides corroborative evidence to previous results based on electroencephalograph (EEG) signals on the role of the visual cortex in quality prediction but also opens up interesting directions for perceptually inspired design of objective video quality metrics. ©2021 IEEE

[error in script]

IITH Creators:

IITH Creators	ORCiD
Channappayya, Sumohana S.	https://orcid.org/0000-0002-5880-4023

Item Type:

Conference or Workshop Item (Paper)

Uncontrolled Keywords:

Blood Oxygen Level-Dependent (BOLD); Functional Magnetic Resonance Imaging (fMRI); Haemodynamic Response Function (HRF); Quality of Experience (QoE); Time Varying Subjective Quality (TVSQ); Video Quality Assessment (VQA); Visual Cortex

Subjects:

Electrical Engineering

Divisions:

Department of Electrical Engineering

Depositing User:

. LibTrainee 2021

Date Deposited:

07 Oct 2022 05:11

Last Modified:

07 Oct 2022 05:11

URI:

http://raiithold.iith.ac.in/id/eprint/10838