Product quantization in Faiss and from scratch
In this video, we talk about a vector compression technique called Product quantization. We first explain conceptually, what the main ideas are and then show how one can use an existing implementation of it from Faiss (IndexPQ). Finally, we also implement the algorithm from scratch. Last but not least, we run some experiments and compare different methods.
Paper: https://lear.inrialpes.fr/pubs/2011/J...
Code: https://github.com/jankrepl/mildlyove...
Faiss: https://github.com/facebookresearch/f...
Nanopq: https://github.com/matsui528/nanopq
00:00 Intro
00:45 Overview: Similarity search [slides]
01:50 Overview: Main idea [slides]
02:18 Overview: How exactly codes created [slides]
03:25 Overview: Similarity search with PQ [slides]
04:24 PQ in Faiss [code]
09:40 Custom PQ - Constructor [code]
12:25 Custom PQ - Training [code]
13:49 Custom PQ - Encoding and add method [code]
15:11 Custom PQ - Computing distance [code]
17:36 Custom PQ - Search [code]
18:56 Making sure things work [code]
20:48 Results: CustomPQ vs Faiss? [demo]
22:59 Results: Number of segments tradeoff [demo]
24:17 Outro
If you have any video suggestions or you just wanna chat feel free to join the discord server: / discord
Twitter: / moverfitted
Credits logo animation
Title: Conjungation · Author: Uncle Milk · Source: / unclemilk · License: https://creativecommons.org/licenses/... · Download (9MB): https://auboutdufil.com/?id=600
Watch video Product quantization in Faiss and from scratch online without registration, duration hours minute second in high quality. This video was added by user mildlyoverfitted 22 June 2022, don't forget to share it with your friends and acquaintances, it has been viewed on our site 7,371 once and liked it 213 people.