GPU-Accelerated Indexing in LanceDB

by yurisagalovon 11/2/2023, 5:31 PMwith 4 comments

by gardnron 11/3/2023, 6:29 AM

If you are just getting into this tech, like me, then this article leads down the rabbit hole of different index types supported in different vector stores. Most easy-to-use vector stores only use HNSW indices, which is "good enough".

A few offer different types of indices. Milvus[1] and FAISS[2] are a couple that support IVF indices. It's great to see another lightweight tool that does this with GPU acceleration.

1. https://objectbox.io/vector-database/

2. https://www.pinecone.io/learn/series/faiss/vector-indexes/

by esafakon 11/3/2023, 6:41 AM

They do predicate pushdown for filtering too. Noice! https://lancedb.github.io/lance/read_and_write.html#filter-p...

by schneehertzon 11/3/2023, 6:19 AM

LanceDB is one of the few options for embeddable vector databases, and I have used it in my Electron application. If they could choose a less confusing npm package name than "vectordb," maybe I would be more forgiving towards them. Moreover, the documentation for LanceDB is also poorly written.