I built this to give developers a single-binary deployment with native ML inference (via a built-in service called Termite), meaning you don't need external API calls for vector search unless you want to use them.
Some things that might interest this crowd:
Capabilities: Multimodal indexing (images, audio, video), MongoDB-style in-place updates, and streaming RAG.
Distributed Systems: Multi-Raft setup built on etcd's library, backed by Pebble (CockroachDB's storage engine). Metadata and data shards get their own Raft groups.
Single Binary: antfly swarm gives you a single-process deployment with everything running. Good for local dev and small deployments. Scale out by adding nodes when you need to.
Ecosystem: Ships with a Kubernetes operator and an MCP server for LLM tool use.
Native ML inference: Antfly ships with Termite. Think of it like a built-in Ollama for non-generative models too (embeddings, reranking, chunking, text generation). No external API calls needed, but also supports them (OpenAI, Ollama, Bedrock, Gemini, etc.)
License: I went with Elastic License v2, not an OSI-approved license. I know that's a topic with strong feelings here. The practical upshot: you can use it, modify it, self-host it, build products on top of it, you just can't offer Antfly itself as a managed service. Felt like the right tradeoff for sustainability while still making the source available.
Happy to answer questions about the architecture, the Raft implementation, or anything else. Feedback welcome!
Discussion (40 Comments)Read Original on HackerNews
I’ve got a project right now, separate vector DB, Elasticsearch, graph store, all for an agent system.
When you say Antfly combines all three, what does that actually look like at query time? Can I write one query that does semantic similarity + full-text + graph traversal together, or is it more like three separate indexes that happen to live in the same binary?
Does it ship with a CLI that's actually good? I’m pivoting away from MCP. Like can I pipe stuff in, run queries, manage indexes from the terminal without needing to write a client? That matters more to me than the MCP server honestly.
And re: Termite + single binary, is the idea that I can just run `antfly swarm`, throw docs and images at it, and have a working local RAG setup with no API keys? If so, that might save me a lot of docker-compose work.
Who's actually running this distributed vs. single-node? Curious what the typical user experience looks like.
Exactly the use case I built it for! I wanted a world where you could build your indexes and the query planner could just be smart enough to use them in a single query. I've not quite nailed down the agentic query planner side 100% (it's getting there), but the JSON query DSL allows you to pipeline, join, fuse all the full-text, semantic, graph, reranking, pruning (score/token pruning) all in one query.
The CLI is my primary development tool with antfly, I am definitely looking for feedback on what people would like to see there, it's a little chonky with the flags --pruner e.g. requires writing the JSON for the config because I didn't want users to have to memorize 1000 subflags. It's definitely a first class citizen.
With respect to "Termite + single binary" that's exactly right, Termite handles chunking, multimodal chunking, embeddings (sparse + dense), reranking, fused chunking/embedding models, and we're excitedly getting more support for a variety of onnx based llms/ner models to help with data extraction use cases (functiongemma/gliner2/etc) so you don't have to setup 10 different services for testing vs deployment.
We run Antfly ourselves for our https://platform.searchaf.com (cheeky search AntFly) Algolia style search product in a distributed setup, and some users run Antfly in single node with large instances (more at the Postgres size datasets with millions of documents vs. large multitenant depoys). But we really wanted to build something with a more seamless experience of going back and forth between a distributed vs single node instance than elasticsearch or postgres can offer.
Hope that helps! Let me know if I can help you with anything!
On a parallel note, It would be nice to put an architecture diagram in the github repo. Are there particular aspects of the current implementation which you want to actively improve/rearchitect/change?
I agree with the goals set out for the project and can testify that elasticsearch's DX is pretty annoying. Having said that, distributed indexing with pluggable ingestion/query custom indexes may be a good goal to aim for. - Finite State Transducers (FST) or Finite state automata based memory efficient indexes for specific data mimetypes - adding hashing based search semantic search indexes.
And even changing the indexer/reranker implementation would help make things super hackable.
Yes good call, I tried to start that on the website with a react-flows based architectural flow chart a little bit but it's a bit high level, and not consumable directly in github markdown files but I'll work on that!
That's exactly the direction I've been working on, the reranking, embedders and chunkers are all plugable and the schema design (using jsonschema for our "schema-ish" approach allows for fine-grained index backend hints for individual data types etc.) I'll work on getting a good architecture doc up today and tomorrow!
Curious about resource contention though: if a heavy indexing job saturates Termite, does that affect query latency on the Raft side? And how does Termite handle model cold starts in single-process mode?
On the license: the ELv2 framing is honest and the "can't offer as managed service" carve-out is pretty standard at this point. Won't bother most people reading this.
In regards to contention, the answer is definitely dependent on how you host. We've had a lot of experience running different ML workloads and from an SRE perspective we knew you'd need a variety of different styles of hosting the models depending on read/write patterns of your usage. Termite and the proxy service/operator allow for all styles of model loading, either preloading and compiling to prevent cold starts or lazy loading to protect memory, with different pooling strategies and caching strategies for bundling multiple models running in the same Termite container.
If a heavy indexing job is running on a CPU only single-node deployment, it won't be using Raft (no replication). If it's running with GPU it doesn't share resources with the DB anyways really significantly there. If it's running distributed, also no issue with contention really.
Let us know if you have any other questions!
We implemented chaos testing suites akin to Jepsen to cover as many scenarios as possible and are currently implementing TigerBeetle style simulation tests on top of that for harder to reproduce scenarios!
I've long wished for QUIC with Nomad! [1] We've always used a weird QUIC-over-TCP multiplexer called yamux. [2]
[1] https://github.com/hashicorp/nomad/issues/23848
[2] https://github.com/hashicorp/yamux (I'm fairly certain libp2p's fork is actually better)
Thanks for the links! I hadn't seem yamux before!
Comment on the Pause method indicates that waits for in flight Batch operations (by obtaining the lock) but Batch doesn’t appear to hold the lock during the batch operation. Am I missing something?
Upon another look it looks like we were actually missing the pause lock for the backfill operation too during a shard split though, I also went ahead and added it to batch for good measure although that case should be caught by the manager! Thank you for the report!
I've seen it on a few products and it doesn't click with me how people are using it.
Anyway, that's just one example of why you might want to use a knowledge graph. I'm sure there are literally hundreds, of more examples.
I can't figure out what the graph part of the equation unlocks.
For fun I am making hybrid search too and would love to see how you merge the two list (semantic and keyword) and rerank the importance score.
Did you build this for yourself?
https://arxiv.org/abs/2410.14452 spfresh, https://arxiv.org/abs/2111.08566 spann, https://arxiv.org/abs/2405.12497 rabitq, https://arxiv.org/abs/2509.06046 diskann,
I have a variety of blogs that I used too and reference implementations!
It's a Rabit[Q]uantized Hierchical Balanced Clustering algorithm we use for the vector index and we use a chunked segment index for the sparse index if you're curious! Happy to discuss more!