NLP & LLM
NLP & LLM
Gain a solid understanding of how tokenization is done and how to optimize vector search in your RAG systems.
Learn about tokenization and vector search optimization for large-scale customer-facing RAG applications. You'll learn about the technical details of how vector search works and how to optimize it for better performance.
Here's what you'll learn, in detail:
Understand the internal workings of embedding models and how your text turns into vectors.
Explore how different tokenization techniques like Byte-Pair Encoding, WordPiece, and Unigram, work and affect search relevance.
Learn how to measure the quality of your search across several quality metrics.
Understand how the main parameters in HNSW, a graph-based algorithm, affect the relevance and speed of vector search and how to optimally adjust these parameters.
Experiment with the three major quantization methods –product, scalar, and binary – and learn how they impact memory requirements, search quality, and speed.
Take your RAG applications to the next level!