Retrieval-Augmented Generation (RAG)

Опубликовано: 07 Октябрь 2020
на канале: Connor Shorten

32,454

585

This video explains the Retrieval-Augmented Generation (RAG) model! This approach combines Dense Passage Retrieval with a Seq2Seq BART generator. This is tested out on knowledge intensive tasks like open-domain QA, jeopardy question generation, and FEVER fact verification. This looks like a really interesting paradigm for building language models that produce factually accurate generations!

Thanks for watching! Please Subscribe!

Paper Links:
Original Paper: https://arxiv.org/pdf/2005.11401.pdf
FB Blog Post (Animation used in Intro): / retrieval-augmented-generation-streamlinin...
HuggingFace RAG description: https://huggingface.co/transformers/m...
Billion-scale similarity search with GPUs: https://arxiv.org/pdf/1702.08734.pdf
Language Models as Knowledge Bases? https://arxiv.org/abs/1909.01066
REALM: Retrieval-Augmented Language Models: https://arxiv.org/pdf/2002.08909.pdf
Dense Passage Retrieval: https://arxiv.org/pdf/2004.04906.pdf
FEVER: https://arxiv.org/pdf/1803.05355.pdf
Natural Questions: https://storage.googleapis.com/pub-to...
TriviaQA: https://arxiv.org/pdf/1705.03551.pdf
MS MARCO: https://arxiv.org/pdf/1611.09268.pdf

Thanks for watching!

Time Stamps
0:00 Introduction
2:05 Limitations of Language Models
4:10 Algorithm Walkthrough
5:48 Dense Passage Retrieval
7:44 RAG-Token vs. RAG-Sequence
10:47 Off-the-Shelf Models
11:54 Experiment Datasets
15:03 Results vs. T5
16:16 BART vs. RAG - Jeopardy Questions
17:20 Impact of Retrieved Documents zi
18:53 Ablation Study
20:25 Retrieval Collapse
21:10 Knowledge Graphs as Non-Parametric Memory
21:45 Can we learn better representations for the Document Index?
22:12 How will Efficient Transformers impact this?

Смотрите видео Retrieval-Augmented Generation (RAG) онлайн без регистрации, длительностью часов минут секунд в хорошем качестве. Это видео добавил пользователь Connor Shorten 07 Октябрь 2020, не забудьте поделиться им ссылкой с друзьями и знакомыми, на нашем сайте его посмотрели 32,454 раз и оно понравилось 585 людям.