Dense vs Sparse Retrieval

Dense Retrieval refers to a category of document retrieval techniques where the document and query representations are dense vectors, that is, vectors where none or a very small number of entries are zeros. This is in contrast to Sparse Retrieval methods, where those representations are sparse, that is, most of the vector representation entries are zeros. A typical sparse representation is tf-idf (Term Frequency - Inverse Document Frequency), while most dense representations are based on Transformer models (such as the Longformer, for example).
Related concepts:
tf-idfTransformerVector Database
Related video:
https://youtube.com/shorts/7w-wtp-RqCE