When I started building Vettore, the goal was fairly simple: I wanted vector search inside my Elixir applications without installing another database or running a separate service just to compare embeddings.
Rust looked like the sensible choice. I needed fast numerical operations, SIMD, HNSW, and perhaps GPU acceleration later. There wasn't much point forcing every calculation into pure Elixir.
So in the first versions, Rust didn't just do the calculations. It owned the data too.
It worked. But as I added features, an uncomfortable thought kept coming back: I was building a little database inside Rust, with Elixir reduced to something resembling a remote control.
That wasn't the system I wanted.
Speed wasn't the thing bothering me
Keeping vectors in native memory has obvious advantages. You avoid conversions, lay out data in the most convenient format for the algorithms, and don't keep crossing the BEAM–Rust boundary.
But there was a cost I hadn't given enough attention to.
If records live exclusively inside the native engine, their lifecycle belongs to that engine as well. Inspecting them, recovering them, or rebuilding an index all mean going through native APIs.
More importantly, I was giving up one of the reasons I enjoy working in Elixir: explicit processes, clear ownership, and state whose responsibilities I can actually see.
I started wondering whether I'd put too much into one component.
An HNSW index exists to find nearby vectors quickly. Why should it also be the only owner of the documents those vectors refer to?
That's where I changed direction: ETS owns the records; Rust accelerates the expensive operations.
I'm not claiming ETS is faster than Rust. I'm saying a native index no longer has to double as my database.
The test I care about most is rebuilding everything
What I like most about the new architecture is that I can save a collection and load those very same records with a different index.
Here's a real example using Vettore's API:
{:ok, collection} =
Vettore.new(
dimensions: 3,
metric: :cosine,
index: :hnsw
)
:ok =
Vettore.put(collection, %{
id: "document-1",
vector: [1.0, 0.0, 0.0],
metadata: %{lang: "en"}
})
:ok = Vettore.snapshot(collection, "documents.ets")
# Same data, different search strategy
{:ok, restored} =
Vettore.load_snapshot(
"documents.ets",
index: :flat
)The snapshot keeps the records and collection configuration. It doesn't need to preserve HNSW's native graph as the only representation of the data; the requested index is built again on load.
I also wrote a test that simulates a native index becoming unusable. Searching through the broken index fails, but the canonical records can still be saved and used to reconstruct a new one.
That's an important distinction. It doesn't make Rust immune to crashes, and snapshots still need to be written and managed properly. It does mean the recoverability of my records isn't tied to one particular search structure.
Of course, the complexity didn't disappear. It moved. ETS and the native index now have to stay consistent. If a vector is inserted into ETS but the index update fails, I can't simply shrug and leave them disagreeing.
That's why the implementation has coordinated updates and rollback paths. I gained control over state, not simplicity for free.
The fun part: searching with bits
Separating the records from the algorithms also gave me more freedom to experiment with different search strategies.
One of my favorites is binary quantization.
A 384-dimensional embedding stored as 32-bit floats needs 1,536 bytes for its numerical values alone. Its sign signature represents those components with 384 bits—48 bytes of packed information. That's a theoretical encoded size, not the actual ETS memory footprint: Vettore holds the 64-bit words in an Elixir list, which adds overhead.
For example:
Embedding: [0.8, -0.3, 0.2, -0.9] Signs: + - + - Bits: 1 0 1 0
Two signatures can be compared with XOR and popcount. The number of differing bits gives the Hamming distance.
It's cheap. It's also very much not an exact vector search.
Take [1.0, 0.1] and [0.1, 1.0]: both produce the same sign signature, [1, 1]. But if the first vector is the query, cosine similarity is 1.0 with itself and about 0.198 with the second. The sign bits alone cannot distinguish those directions.
So Vettore compares sign signatures using Hamming distance to shortlist candidates, then reranks only those candidates using the original full-precision vectors. HNSW takes a different route by exploring a graph index:
{:ok, results} =
Vettore.quantized_search(
collection,
[1.0, 0.0, 0.0],
candidates: 100,
limit: 10
)And that introduces another tradeoff. Fewer candidates mean less computation, but also a greater chance that a good result gets discarded before the precise comparison ever happens.
To decide whether the optimization is worth it, I measure latency alongside recall@k: how many of the exact search's top k neighbors the approximate search also finds. If exact search returns ten neighbors and the faster method finds only eight of those ten in its own top ten, recall@10 is 80%. A faster benchmark is not enough if it discards too many relevant results.
The repository includes benchmarks for Flat and HNSW across different vector dimensions.
What I gave up along the way
The new architecture isn't necessarily the fastest possible one. ETS has memory overhead, maintaining two structures costs work, and rebuilding indexes takes time.
Process ownership matters too. A Vettore collection is closed when the process that created it terminates. Creating a long-lived collection inside a short-lived Task is therefore a bad idea.
But the result fits much better with the rest of my Elixir code.
If I decide to replace HNSW, change the search policy, or experiment with another native algorithm, I don't necessarily have to rewrite the part that owns the records.
I might have squeezed out more performance by leaving everything in Rust. I don't rule that out.
But at some point I realized that the speed of an algorithm and the quality of the architecture around it are two different problems. With Vettore, I wanted to take both seriously.
