Retrieval-Augmented Generation (RAG)
This video explains the Retrieval-Augmented Generation (RAG) model! This approach combines Dense Passage Retrieval with a Seq2Seq BART generator. This is tested out on knowledge intensive tasks like open-domain QA, jeopardy question generation, and FEVER fact verification. This looks like a really interesting paradigm for building language models that produce factually accurate generations!
Thanks for watching! Please Subscribe!
Paper Links:
Original Paper: https://arxiv.org/pdf/2005.11401.pdf
FB Blog Post (Animation used in Intro): / retrieval-augmented-generation-streamlinin...
HuggingFace RAG description: https://huggingface.co/transformers/m...
Billion-scale similarity search with GPUs: https://arxiv.org/pdf/1702.08734.pdf
Language Models as Knowledge Bases? https://arxiv.org/abs/1909.01066
REALM: Retrieval-Augmented Language Models: https://arxiv.org/pdf/2002.08909.pdf
Dense Passage Retrieval: https://arxiv.org/pdf/2004.04906.pdf
FEVER: https://arxiv.org/pdf/1803.05355.pdf
Natural Questions: https://storage.googleapis.com/pub-to...
TriviaQA: https://arxiv.org/pdf/1705.03551.pdf
MS MARCO: https://arxiv.org/pdf/1611.09268.pdf
Thanks for watching!
Time Stamps
0:00 Introduction
2:05 Limitations of Language Models
4:10 Algorithm Walkthrough
5:48 Dense Passage Retrieval
7:44 RAG-Token vs. RAG-Sequence
10:47 Off-the-Shelf Models
11:54 Experiment Datasets
15:03 Results vs. T5
16:16 BART vs. RAG - Jeopardy Questions
17:20 Impact of Retrieved Documents zi
18:53 Ablation Study
20:25 Retrieval Collapse
21:10 Knowledge Graphs as Non-Parametric Memory
21:45 Can we learn better representations for the Document Index?
22:12 How will Efficient Transformers impact this?
Watch video Retrieval-Augmented Generation (RAG) online without registration, duration hours minute second in high quality. This video was added by user Connor Shorten 07 October 2020, don't forget to share it with your friends and acquaintances, it has been viewed on our site 32,454 once and liked it 585 people.