Vector databases are becoming part of the infrastructure that sysadmins, DevOps teams and developers need to understand and operate. Behind many Retrieval-Augmented Generation (RAG) systems, semantic search engines and AI agents sits a retrieval layer responsible for storing embeddings, indexing them and finding relevant information quickly. Choosing that layer involves much more than query performance: memory, storage, replication, backups, observability, scaling and operational complexity all matter.
The key facts about vector databases for sysadmins in 30 seconds
- A vector database adds similarity search to the infrastructure behind AI applications.
- HNSW and IVF are widely used to accelerate Approximate Nearest Neighbor (ANN) searches.
- PostgreSQL, Redis, MongoDB, Elasticsearch and OpenSearch can add vectors to platforms many teams already operate.
- Pinecone, Weaviate, Milvus and Qdrant provide more specialized vector architectures.
- RAM, storage, filtering, replication, backups and reindexing can matter as much as raw query latency.
- The best option is often the one that meets requirements without introducing unnecessary infrastructure.
For a developer, a vector database can initially look like another backend receiving queries. For the infrastructure team, the problem starts several steps earlier.
Someone has to decide where it runs, how large the indexes will become, how much memory they require, how nodes are replicated, what happens when a server fails and how millions of embeddings will be rebuilt if the underlying embedding model changes.
Vector workloads can also behave differently from conventional SQL databases.
An Approximate Nearest Neighbor index may require substantial RAM to deliver low latency. Building indexes consumes CPU and I/O. Generating embeddings may require GPUs or external inference services, while replacing an embedding model can trigger a large-scale reindexing operation.
That is why a vector database should not be selected independently from the rest of the infrastructure.
HNSW, IVF and exact search: what is actually being operated?
An embedding is essentially a numerical representation of some characteristics of the original content.
A system can store millions of these vectors and search for those closest to the vector generated from a user query.
The most accurate approach would compare the query against every vector. That can work with small datasets, but the cost increases quickly as the collection grows.
Large systems therefore commonly use ANN techniques.
One of the most widely deployed approaches is Hierarchical Navigable Small World (HNSW). It creates a multi-layer graph that makes it possible to navigate quickly towards areas likely to contain the nearest neighbors.
The speed comes with trade-offs.
HNSW requires additional data structures and can consume substantial memory. Configuration choices affect index size, query latency, build time and recall.
Another approach is Inverted File Index (IVF). Instead of searching the entire vector space, vectors are divided into groups and only selected groups are examined for each query.
FAISS, pgvector, Milvus and other technologies support variations of these techniques.
For sysadmins, that introduces parameters that can easily remain invisible during development:
| Infrastructure decision | Operational impact |
|---|---|
| Embedding dimensions | Size of every stored vector |
| Number of vectors | RAM and storage requirements |
| HNSW | Higher memory use for fast retrieval |
| IVF | Requires tuning of partitions and probes |
| Quantization | Lower memory use with possible accuracy trade-offs |
| Replication | Better availability but higher resource consumption |
| Metadata filtering | Can affect retrieval performance |
| Reindexing | CPU, memory, storage I/O and operational time |
This is why an application that works perfectly with 100,000 documents on a developer laptop can behave very differently when deployed with hundreds of millions of chunks.
16 vector technologies and what they mean for infrastructure
Not everything described as a vector database is actually the same kind of product.
FAISS is a library. pgvector is a PostgreSQL extension. Pinecone is a managed service. Milvus can become a substantial distributed infrastructure platform. Elasticsearch and OpenSearch are search engines that have added vector retrieval.
From an operations perspective, that distinction should come before benchmark results.
| Technology | Model | Operational overhead | Particularly suitable for |
|---|---|---|---|
| Pinecone | Managed Vector DB | Low | Managed RAG |
| Weaviate | Vector DB | Medium | Hybrid search and RAG |
| Milvus | Distributed Vector DB | Medium/high | Very large datasets |
| Qdrant | Vector DB | Medium | Performance and filtering |
| pgvector | PostgreSQL extension | Low if PostgreSQL exists | Infrastructure consolidation |
| Redis | Data platform + vectors | Low/medium | Low-latency workloads |
| FAISS | ANN library | Application dependent | Custom search engines |
| Chroma | AI retrieval platform | Low | Development and RAG |
| OpenSearch | Search engine | Medium/high | Search + RAG |
| Elasticsearch | Search platform | Medium/high | Enterprise search |
| MongoDB Vector Search | Documents + vectors | Low/medium | Existing MongoDB applications |
| Vespa | Search and ranking | High | Complex real-time ranking |
| LanceDB | Vector/multimodal platform | Low/medium | Multimodal AI |
| Vald | Distributed ANN | High | Kubernetes environments |
| Marqo | AI Search | Low/medium | Multimodal search |
| Azure AI Search | Managed search | Low | Microsoft Azure environments |
Pinecone: outsourcing much of the operational layer
Pinecone represents the managed-service approach. Its serverless architecture is intended to remove much of the work associated with provisioning and operating traditional vector database nodes.
That can simplify production considerably, particularly for development teams that do not want to operate another distributed database.
There is a trade-off. Infrastructure teams exchange operational responsibility for dependence on an external managed platform, its availability model and its pricing structure.
Weaviate: vectors and lexical search together
Weaviate supports both managed and self-hosted deployments and places considerable emphasis on hybrid retrieval.
Its hybrid search can combine vector similarity with BM25F lexical search. This is useful because not every query should be solved through semantic similarity alone.
A CVE identifier, hostname, product SKU or function name may require exact lexical matching, while a natural-language question may benefit more from embeddings.
For infrastructure teams, Weaviate provides the option of keeping the platform under their own control or consuming it as a managed service.
Milvus: when millions become billions
Milvus is designed specifically for vector search and provides several deployment models.
Milvus Lite targets smaller applications, Standalone runs on a single machine, while Milvus Distributed is intended for larger deployments and uses Kubernetes.
The distributed architecture is designed for workloads that can grow into billions of vectors.
That scalability comes with the usual infrastructure consideration: a distributed database is another distributed system that has to be deployed, upgraded, observed and recovered.
Qdrant: filtering and vector retrieval with Rust underneath
Qdrant is a vector search engine written in Rust and available both as open-source software and as a managed service.
It supports dense and sparse vectors, metadata filtering, quantization and distributed deployment.
Qdrant is particularly interesting when applications need to combine semantic retrieval with extensive metadata filtering. Enterprise RAG rarely searches every document indiscriminately; results frequently need to be restricted by tenant, department, user permissions, date or document type.
pgvector: check PostgreSQL before deploying another cluster
For many sysadmins, pgvector should be among the first options evaluated.
The extension brings vector similarity search directly into PostgreSQL and supports exact nearest-neighbor search as well as HNSW and IVFFlat indexes.
This allows users, permissions, transactional data, metadata and embeddings to remain inside the same database.
There is another operational advantage: existing PostgreSQL knowledge remains useful.
Replication, pg_dump, Point-in-Time Recovery (PITR), SQL monitoring, transactions, access control and established backup procedures do not suddenly disappear because an application starts using embeddings.
There can also be fewer synchronization problems.
If the source documents live in PostgreSQL but embeddings are stored in a separate vector database, the application has to keep two systems consistent. With pgvector, application data and vectors can potentially participate in the same transactional workflow.
A dedicated Vector DB may perform better or scale more naturally for particular workloads. But adding another distributed database to store a few hundred thousand or several million embeddings is not automatically better architecture.
Redis: vectors in an existing low-latency data layer
Redis provides vector search alongside its existing data structures.
Vectors can be stored in Hash or JSON objects and indexed for K-nearest neighbor and range queries. Redis also provides different indexing strategies for vector workloads.
This can be attractive where Redis already provides caching, sessions, queues or real-time application data.
Memory planning deserves attention. Adding millions of vectors and ANN indexes to a Redis deployment can substantially change its capacity requirements, so production sizing should use representative embeddings rather than extrapolating from a small proof of concept.
FAISS: a powerful engine, but not a complete database
FAISS requires an important distinction.
FAISS is primarily a library for efficient similarity search and clustering of dense vectors, originally developed by Meta AI Research. It includes C++ implementations, Python wrappers and algorithms that can use GPUs.
It is not a drop-in distributed database equivalent to Pinecone, Qdrant or Milvus.
Teams building around FAISS may need to provide persistence, APIs, authentication, metadata handling, high availability, replication, deployment and monitoring themselves.
That makes FAISS extremely useful when developers want control over their retrieval engine, but potentially unnecessary work when the requirement is simply to add reliable RAG retrieval to an application.
Chroma: a low barrier to RAG development
Chroma is designed around AI application development and provides a relatively simple way to store and retrieve embeddings.
It can be used locally during development and also offers deployment options beyond a developer workstation.
Its accessibility makes it useful for prototypes. As with any development-friendly datastore, however, moving into production should trigger a review of persistence, concurrency, backup, scaling and recovery requirements.
OpenSearch, Elasticsearch and MongoDB: use the infrastructure already running
A common production scenario is that the organization already operates a platform capable of vector search.
OpenSearch supports vector, semantic and hybrid search. It can combine traditional lexical retrieval with semantic results, making it suitable for search systems where exact matches and meaning both matter.
Elasticsearch has followed a similar path, combining full-text retrieval, kNN search, dense and sparse vectors and hybrid ranking strategies.
For organizations already operating Elasticsearch or OpenSearch clusters, extending the existing platform can be simpler than introducing a dedicated Vector DB.
That does not mean vectors are free.
ANN indexes can significantly change memory and storage requirements. Running observability, logs, enterprise search and a high-volume RAG application on the same cluster without capacity planning can create resource contention.
MongoDB Atlas Vector Search addresses the same architectural question from the document database side.
Applications can maintain documents, metadata and embeddings in the MongoDB environment and execute semantic retrieval with filtering.
From an operational perspective, reducing the number of systems containing different representations of the same data can be valuable in itself.
Vespa, LanceDB, Vald and Marqo
Vespa is aimed at applications where retrieval and ranking need to be treated as part of the same serving system.
Nearest-neighbor search can be combined with textual queries, filters and multiple ranking phases. That makes Vespa interesting for recommendation, search and large-scale serving workloads where retrieving candidates is only the first step.
LanceDB is increasingly focused on multimodal workloads. Its architecture is designed to work with vectors alongside original data and metadata, making it relevant when AI applications handle images, documents, audio or other large unstructured datasets.
Vald takes a strongly cloud-native approach. It provides distributed ANN search around Kubernetes components.
That can fit organizations with mature Kubernetes platforms. Deploying Kubernetes solely because an application needs vector retrieval, however, is unlikely to simplify a small infrastructure.
Marqo is better understood today as an AI search platform rather than simply a generic vector datastore. Its approach includes semantic and multimodal retrieval, ranking and other search capabilities.
Azure AI Search: vector retrieval inside the Microsoft stack
Azure AI Search provides vector, lexical and hybrid search as a managed Azure service.
Microsoft allows applications to work with existing embeddings or integrate vectorization into ingestion pipelines. Hybrid queries can combine full-text and vector retrieval before ranking the resulting candidates.
For organizations already standardized on Azure, this reduces the amount of infrastructure that must be operated directly.
The corresponding trade-offs are familiar: service pricing, cloud dependency and architectural portability need to be considered alongside functionality.
High availability and backups still matter in the vector world
A vector database does not stop being a data platform just because it stores embeddings.
One of the first questions should be whether vectors are primary data or derived data.
If every embedding can be recreated from documents safely stored elsewhere, losing an index may cause an availability incident without causing irreversible data loss.
The situation changes if the vector platform also contains metadata, user feedback, manual corrections or information that cannot be reconstructed.
Rebuilding time matters too.
Regenerating 20,000 embeddings may be straightforward. Regenerating hundreds of millions requires rereading the source dataset, running an embedding model and rebuilding ANN indexes.
Something can therefore be technically reproducible and still require backups because reconstruction would violate the application’s Recovery Time Objective (RTO).
High availability raises similar questions.
How many replicas exist? What happens when a node disappears? Where is cluster metadata stored? Can the service survive losing an availability zone or region? What are the Recovery Point Objective (RPO) and RTO?
Those questions should have answers before a RAG platform becomes a production dependency.
Re-embedding can become a major infrastructure migration
Vector systems introduce another operation that deserves planning: changing the embedding model.
Vectors generated by different embedding models do not necessarily inhabit compatible vector spaces.
Replacing a model can therefore require re-embedding the complete dataset.
Changing vector dimensions makes the issue even more obvious because the existing index structure may no longer accept the new representation.
Production teams should treat this operation more like a database migration than a routine model configuration change.
One strategy is to maintain two indexes temporarily.
The existing index continues handling production queries while a second index is populated using the new embedding model. The new version can then be evaluated before traffic switches over, while the previous index remains available for rollback.
It is effectively a blue-green migration for the retrieval layer.
At large scale, the infrastructure impact can be substantial: GPU or inference API consumption, CPU load, network traffic, storage requirements and ANN index construction all need capacity.
The best Vector DB may be the one that avoids another Vector DB
Infrastructure engineering has a natural tendency to look for the fastest technology.
But an ANN benchmark does not measure the total cost of running a production service.
A system that answers a query several milliseconds faster may require another Kubernetes cluster, additional monitoring, new backup procedures and engineers who know how to troubleshoot it at 3 a.m.
Meanwhile, PostgreSQL, Redis, MongoDB, OpenSearch or Elasticsearch may already be deployed, monitored and backed up.
The decision should therefore begin with four questions: how much data will exist, what latency does the application require, what availability must be guaranteed and what platforms can the team already operate reliably?
Only then should benchmark numbers and feature comparisons decide the winner.
For a developer, adding another dependency can mean one more line in a configuration file.
For a sysadmin, that same line can mean patching, CVEs, metrics, alerts, capacity planning, storage, replication, backups, restores and an additional service to troubleshoot when production stops responding.
Vector databases do not change those rules. They simply introduce a new type of workload into the infrastructure behind modern AI applications.
Frequently asked questions
Is pgvector better than a dedicated vector database?
It depends on scale and workload. When PostgreSQL is already part of the infrastructure and vector requirements are manageable, pgvector can considerably simplify the architecture; specialized platforms can make more sense for very large or demanding vector workloads.
How much RAM does a vector database need?
It depends on vector count, dimensionality, data representation and index type. HNSW adds memory overhead, while quantization can reduce resource requirements at the cost of potential retrieval-quality trade-offs.
Should embeddings be backed up?
That depends on reconstruction cost and recovery objectives. Even if embeddings can be regenerated, rebuilding hundreds of millions of vectors and indexes may take too long to meet the required RTO.
What should sysadmins monitor in a Vector DB?
CPU, RAM, storage and network remain important, but teams should also monitor query latency, index size and build times, ingestion queues, vector counts, replication health, errors and reindexing operations.