public-projects
/
vlcn
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Update README.md

This commit is contained in:
Adnen Abdessaied 2022-08-10 14:32:52 +00:00
parent dcd62491c4
commit f1b80cfb8b
1 changed files with 4 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
README.md
View File

@ -1,7 +1,7 @@
# VLCN
This repository contains the official code of the paper:
## Video Language Co-Attention with Fast-Learning Feature Fusion for VideoQA [[PDF](https://aclanthology.org/2022.repl4nlp-1.15.pdf)]
**Video Language Co-Attention with Fast-Learning Feature Fusion for VideoQA** [[PDF](https://aclanthology.org/2022.repl4nlp-1.15.pdf)]
[Adnen Abdessaied](https://adnenabdessaied.de), [Ekta Sood](https://perceptualui.org/people/sood/), [Andreas Bulling](https://perceptualui.org/people/bulling/)
**Poster**
@ -18,6 +18,9 @@ If you find our code useful or use it in your own projects, please cite our pape
pages = {1--12}
}
```
# Abstract
We propose the Video Language CoAttention Network (VLCN) – a novel memory-enhanced model for Video Question Answering (VideoQA). Our model combines two original contributions: A multimodal fast-learning feature fusion (FLF) block and a mechanism that uses selfattended language features to separately guide neural attention on both static and dynamic visual features extracted from individual video frames and short video clips. When trained from scratch, VLCN achieves competitive results with the state of the
art on both MSVD-QA and MSRVTT-QA with 38.06% and 36.01% test accuracies, respectively. Through an ablation study, we further show that FLF improves generalization across different VideoQA datasets and performance for question types that are notoriously challenging in current datasets, such as long questions that require deeper reasoning as well as questions with rare answers
# Overview
<p align="center"><img src="assets/overview_project_one.png" alt="drawing" width="600" height="400"/></p>